Testing Critical to Internet Marketing Success

Last week I wrote about diy display advertising and pointed out that all interactive advertising and promotion offers marketers a splendid opportunity to test various approaches. That can include many marketing issues from competing email subject lines to competing messages or creative in a display ad to alternative landing pages and more.

Testing is a fundamental technique of direct marketing, but mass media marketers rarely had the opportunity to test. They have been slow to acquire the skill as some of their marketing has moved onto the web. Marketing programs can be much improved by skilled testing; it’s a skill that needs to be acquired.

In fact, many marketers are foggy about the difference between marketing research and testing. Both have their place in a well-run marketing program. However, in an interactive environment, I argue strongly that testing should come first. There are several reasons:
•Testing answers the question, “what works?” and gives a strong basis for good marketing decisions.
•Testing is faster.
•Testing is cheaper.

Testing, however, cannot answer the “why” questions--why customers think and do what they do. That’s where marketing research comes in. It’s also the reason that I argue to test first and find out what works, then ask “why” if and only if you need the answer to improve decision making.

Testing also requires a certain degree of technical skill to ensure the test is run properly and read correctly. In the earlier post I recommended Paul Berger’s chapter on testing in our direct marketing book. Paul is a superb statistician and his approach is based on classical hypothesis testing. You may find the outline familiar from your own statistics course. I developed a companion concept for my Internet marketing text that emphasis the managerial aspects of testing. The truth is that both perspectives are key to conducting valid tests.

Let me give just one example—the free membership trial from the Marketing Sherpa site. The test was conducted by Marketing Experiments, and you can access commentary and the full presentation via their blog. This test is a simple A/B split in which existing sign-up page A is tested against new sign-up page B. In this case A is the sign-up page currently in use, so it becomes the control (baseline) and the question is, “Does B work better than A?” A control (no treatment in experimentation parlance) is not strictly required in an A/B split. Then the question is, “Does A or B work better?”

In this case the control included a fairly generic paragraph about the benefits of membership and requested a minimal amount of information for signup. The treatment page included a feature matrix that detailed the content available free to newsletter subscribers and additional content available to members of Marketing Sherpa. The information required was the same; the difference was in the amount of information given to the potential registrant.

The test was run over a 30-day period and the metric used was conversion rate. More information did work, in spite of the fact that it led to a longer, more complex page. It improved conversion by more than half a percent, which—given generally low conversion rates—is huge.

What Marketing Sherpa has done with it is interesting. Instead of simply using the features listing, it has a page for each feature as part of the process. That would be deadly if you had to go all the way through in order to sign up for the free trial membership. The sign-up box is on each page, so the minute the visitor is convinced, she can enter her information. If not yet convinced, she goes on to the next page. Clever!

I’d encourage you to view you can access Dr. McGlaughlin’s entire presentation. It gives two other good examples. You can also find testing articles on GrokDotCom and examples of various types of testing on Omniture’s Offermatica site.

Whether the marketer uses marketing support services or chooses to develop testing expertise in house, it is an essential skill for the Internet age.