Testing is a fundamental technique of direct marketing, but mass media marketers rarely had the opportunity to test. They have been slow to acquire the skill as some of their marketing has moved onto the web. Marketing programs can be much improved by skilled testing; it’s a skill that needs to be acquired.
In fact, many marketers are foggy about the difference between marketing research and testing. Both have their place in a well-run marketing program. However, in an interactive environment, I argue strongly that testing should come first. There are several reasons:
•Testing answers the question, “what works?” and gives a strong basis for good marketing decisions.
•Testing is faster.
•Testing is cheaper.
Testing, however, cannot answer the “why” questions--why customers think and do what they do. That’s where marketing research comes in. It’s also the reason that I argue to test first and find out what works, then ask “why” if and only if you need the answer to improve decision making.
![](http://bp0.blogger.com/_OhWWsR1afKg/SHY2TAFkfGI/AAAAAAAAA0g/JXKwxvKlU70/s0-d/testing+pauls.jpg)
![](http://bp3.blogger.com/_OhWWsR1afKg/SHY2wV4yBrI/AAAAAAAAA0w/tz3OSu8bOQw/s0-d/testing+mine.jpg)
Let me give just one example—the free membership trial from the Marketing Sherpa site. The test was conducted by Marketing Experiments, and you can access commentary and the full presentation via their blog. This test is a simple A/B split in which existing sign-up page A is tested against new sign-up page B. In this case A is the sign-up page currently in use, so it becomes the control (baseline) and the question is, “Does B work better than A?” A control (no treatment in experimentation parlance) is not strictly required in an A/B split. Then the question is, “Does A or B work better?”
![](http://bp1.blogger.com/_OhWWsR1afKg/SHY3O6LaIFI/AAAAAAAAA04/8oJk3tPNYTc/s0-d/testing+control.jpg)
![](http://bp1.blogger.com/_OhWWsR1afKg/SHY3cgwbTXI/AAAAAAAAA1A/i5gNX1RifIA/s0-d/testing+treatment.jpg)
In this case the control included a fairly generic paragraph about the benefits of membership and requested a minimal amount of information for signup. The treatment page included a feature matrix that detailed the content available free to newsletter subscribers and additional content available to members of Marketing Sherpa. The information required was the same; the difference was in the amount of information given to the potential registrant.
The test was run over a 30-day period and the metric used was conversion rate. More
![](http://bp3.blogger.com/_OhWWsR1afKg/SHY315XcqJI/AAAAAAAAA1I/p86-3wctjK8/s0-d/testing+results.jpg)
What Marketing Sherpa has done with it is interesting. Instead of simply using the features listing, it has a page for each feature as part of the process. That would be deadly if you had to go all the way through in order to sign up for the free trial membership. The sign-up box is on each page, so the minute the visitor is convinced, she can enter her information. If not yet convinced, she goes on to the next page. Clever!
I’d encourage you to view you can access Dr. McGlaughlin’s entire presentation. It gives two other good examples. You can also find testing articles on GrokDotCom and examples of various types of testing on Omniture’s Offermatica site.
Whether the marketer uses marketing support services or chooses to develop testing expertise in house, it is an essential skill for the Internet age.
Great blog Mary Lou, you bring up so many valid points. We have been split testing headlines and contents for some time now and while it is time consuming in the end it brings about the desired results!
ReplyDeleteDenise
http://www.DubLi-Network.blogspot.com
I think testing also plays an important role for good marketing decisions so it will be helpful for marketing our product in a right way.
ReplyDeleteHi Dr. Roberts,
ReplyDeleteWe use the Google A/B and multivariate testing and it has proven invaluable and it's FREE!
We have more than doubled our "Goals" conversion using testing.
Testing Saves Money! It's worth the effort!
I hope to take your Social Media class when it's available online.
Thanks for all the supportive comments! Testing is important in achieving many different types of marketing objectives. More marketers, especially those trained in traditional marketing, need to understand and use it.
ReplyDeleteThanks again!
ML