Testing for Marketing ROI

Today seems to be testing day in marketing newsletters. I read an article in today’s SearchBuzz newsletter about multivariate tests and thought I should check out the tools. When I wrote about testing earlier I emphasized A/B splits (bivariate) because I think that’s an easier way to start. While I was working on that, along came this week’s chart from Marketing Sherpa that does a nice job of summarizing some of the strategy issues. But do you need a big budget to test?

In the earlier post I mentioned Offermatica (recently acquired by Omniture) as a testing tool. I really like their demos of both A/B splits and multivariate tests. They’re just a bit harder to find now. You have to look on Omniture’s product tours page and select Test & Target. But this is a hosted service, not DIY.

But the tools I wanted to take a look at are Google Website Optimizer and Widemile's Page Optimizer. Google’s is free. Google, of course, has a video.

Widemile’s business model is to provide technology to agencies and marketing services firms so they can offer testing services to their clients. They have a simple summary that describes the efficiency of A/B split testing against multivariate testing. You can test many variables at once when you use experimental designs (what we refer to as multivariate testing). That seems like stating the obvious. However, what may not be so obvious is that multivariate tests are much harder to “read.” The real question is, “Are these small differences statistically significant?” As the conceptual examples in my earlier post showed, that takes you right back to your statistics courses. And if you’re like me, you absorbed the fundamentals represented in the text box that starts “Choose the variables to be tested.” But you may share my reluctance to base serious marketing decisions on my personal statistical skills and my concern that the automated tools won't produce the right answers. Automated testing products need to take into account necessary sample sizes for statistical significance. They don’t want to burden users with that type of statistical detail (yes, I understand, it’s a bit scary), so you need to ask questions about statistical significance and confidence levels, even if it means you have to drag out an old stats textbook.

Since Google’s tool is DIY—and they don’t particularly want to talk about the significance issue either—I wondered how it was dealt with, if at all. They have a good blog that is well written and may help with some of your concerns. One of the entries I found was for a sample size estimation tool from one of their partners, a site called WebShare. If you followed my advice in the earlier post and read Paul Berger’s chapter on testing in our (free download) direct marketing text, you’ll find it follows the basic steps, although there are still issues. Specifically, it’s a one-tailed test; see the Improvement call-out on the tool page. It’s only looking for better results; not whether the results are either better or worse. That’s probably realistic for most marketing tests. They give you a nice comparison chart—various % improvement, various significance levels. Try it for yourself!

Most of us have a lot of questions about what will work best in our marketing programs. A/B splits will answer a few of these questions for us. Multivariate tests will answer many more questions in the same time period. Whether you are using marketing services of one kind or another, or whether you’re going to try it DIY, do it with care. I am reminded of the Mark Twain quote:

“It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so.”

Don’t accept insignificant results. And remember the old direct marketing rule to keep testing. If it’s an important marketing decision one test, especially one with a small sample size, is just not enough.

DIY Advertising Takes Off

When I wrote about AdReady several weeks ago I set up an account to keep an eye on this interesting self-service display advertising concept. We’ve gotten used to this process with Google AdWords, and AdReady has services like a library of standard ads that make DIY banner advertising a reality. Following the model of AdWords, you can set your own advertising budget and monitor campaign results.
Because the system is easy to use, it’s easy to test competing ad messages. In their June newsletter, AdReady gave an example of running a test on an advertising button of its own. It’s a nice, clean test, with the only change being the call to action. I’d have guessed that “Click Here Now” was a fairly weak call to action, but personally, I’d have thought “Build it Now” would outperform “Get Started.” That’s why marketers should run tests! If you’d like a direct-response testing primer, please read the free testing chapter from Paul Berger’s and my direct marketing text. There is additional testing material in my Internet marketing text; I’ll do a post on that soon, because it’s an important tool.

So—with AdReady you can create your own ads and test versions. Now you can do it on the New York Times site. It seems like a no-brainer. Small advertisers are not worth the time of the advertising department, but, in the aggregate, they could provide another significant revenue stream. Why not give them self-service capabilities? Using the AdReady platform, that’s just what the NYT has done.

For publisher sites that need to add revenue streams—and who doesn’t—this is an interesting option. For sites that are free to the user, like AdReady and so many others, it has something interesting to say about monetizing the site.

That leads me directly to what I’m planning to write about tomorrow. Please stay tuned!

Google Gadgets and White-Label Widgets

It’s no secret that I’m a fan of widgits. They offer useful content and functions. Even more important, they are completely controlled by the user and improve the web experience instead of interrupting and degrading it.

So last week when this video floated into my inbox, I took the time to watch it. It was filmed at the Google Developers conference and the quality reflects that. The content is well worth the 4.5 minutes it takes; it ranges from the new Gmail widget (available only for iGoogle pages at the moment), to a Google Maps applications that lets you follow a soapbox derby down city streets, to an app that lets you control the way you embed videos into blogs and web pages.

Most of these are entirely DIY; the YouTube video embed may require some knowledge of JavaScript. That means, for most of us marketers, that it may require a developer to install. That’s doable; the question we should ask is will we be able to maintain it without undue difficulty once the developer is finished? The answer is “probably yes,” but it’s a question that needs asking. If the freebies don’t work, you can get rid of them as easily as you acquired them; most of us want to exercise care when we actually spend money on these things. Do you have an iGoogle page—or any other personalized page (if you have Gmail, I think you automatically have an iGoogle page; you just may never have used it)? I’m surprised how many people don’t. It’s a useful experience to see how some of their widget collection works. Some is useful content—notice that weather is the first listed. Others provide an often-used function—I saw both a calendar and a calculator. Some are just plain fun—I loved the Yoda-speak gadget. Note the comments and that the developer is responding to his users. Don’t find one you like? Create your own from easy-to-use templates! The downside is that these are only for iGoogle pages, at least at this point.

There are alternatives—the so-called white label widget (or, more generally, marketing services) providers. TechCrunch defines the concept in terms of a social network; it applies equally to the creation and deployment of widgets. “The idea of white labeling a network is to make the platform provider as invisible as possible to the social network’s users and to brand the network with the builder’s identity or intent.” According to today’s Media Post one of the white-label widget makers, KickApps, is partnering with Clearspring Technologies to allow marketers to distribute their widgets through its network and track their use.

Whether you want a Google Gadget or you want to create your own branded widget from scratch (note the National Geographic widget in the Google list), there are platforms and services to meet your needs. Become an avid user of widgets and use your experience to think about how you can use them to provide branded content to your target audience. It’s content they have chosen and are likely to pay attention to. Can we say that about most advertising?

Have You Been Pingged?

I’ve been evited, but I’ve never been pingged. Presumably the name of this new site is a play on the technical term ping—clever!

Pingg came to my attention as the result of a story from DM News’ Sara Holoubek. It’s a good story that has an important marketing point. I was initially intrigued by the complete “party management” service she described, so I looked at the two sites and found a lot going on.


Evite has been around for awhile and according to the site has 18 million registered users. Evite has recently added the ability to send and receive invitations from your cell phone (assuming that you want to. . .). It also has a photo page to which you can upload pics of your parties, so they are clearly adding social networking apps.

Pingg is new, having been launched in March of this year. It is obviously designed with social networking in mind. It’s surroundSend functionality lets you get your invitation to the recipient, wherever he or she may be. The event management function provides all the basics and some cool Internet apps. The gift and charity registry allows you to embed your Amazon Wish List or to ask your guests to contribute to a charity instead. If you’re selling tickets to your event like Sara was, invitees can purchase their tickets with PayPal. If you are doing this for a worthy cause, you can show the fund-raising target on your personalized event page. Not only cool, but useful!

Back to Sara’s point. If your goal is to draw an audience, just putting something out there, even to your handpicked guest list may not be enough. “Searches and online transactions all originate within the physical world.” Right! And with all the clutter in both physical and cyber worlds, it usually takes more than one communications channel to get your marketing message to its target audience!

Mashups--Business and Otherwise

An interesting article that I accessed through the Knowledge @ Wharton newsletter led to another appealing DIY opportunity. The article discusses the changes in software application development that are occurring, partly to satisfy the demands of the Millennial generation for instant gratification—even in software application development. Read the article for yourself and see if you agree with the management perspective.

I was intrigued by the idea of DIY business mashups, so I investigated the Serena Software site. The site is currently in beta and offering its mashup software free. The FAQs say that you only pay when you publish (“in the cloud”) and start using the app. They don’t say how much or on what basis they charge for the working app.

The first video on their MashupTV page illustrates a situation that many of us are all too familiar with—we need programming support but IT has a seemingly endless backlog of requests. The young man must be a marketer; he wants to interface with Salesforce.com. In the video he promotes the idea of mashups in his unit and, in the process, makes his boss happy and confuses the personal relationships of colleagues who don’t understand “mashing.” Are software marketers developing a sense of humor?

Users can create their own app or customize a number of prebuilt ones that satisfy a lot of communications needs in enterprises. There’s a lot of material on the site to convince you that you can do it and show you how.

This slso ties in to an article in today's iMediaConnection. Max Zabrammy urges marketers to move beyond the desktop to give users access to the site or to specific functionality in other contexts including mobile and offline. One way of doing that is special-purpose apps, so we all better get good at creating them.

So back to where this blog started. Marketers need a lot of Internet firepower these days. It can take a long time for IT to build it for them; hence, the demand for DIY Marketing applications. This seems to be another attractive option.
Sphere: Related Content

Basho Builds B2B Community

In response to my post last week on BobKent.net, one of my students sent me a link to the Basho Technologies site—thanks, Noah! This sales training firm is actively working to create a community of “sales leaders” in a way that few other B2B firms are. Besides, I love the Sumo imagery!

They have a number of interactive channels, all of which are under the umbrella of the Basho Community. Their blog, newsletter and library are not unusual for a B2B firm. Podcasting is catching on—slowly—which is too bad because it seems such a natural in B2B markets. Theirs are well organized; for one thing when you go to the podcast page, you can easily make the choice to listen to the MP3 file on your computer or to download it to an iPod or other MP3 device. Making that clear is good for the non-iPod person or the novice user.
The most interesting channel is their LinkedIn group. When I looked, it had 2,927 members. I didn’t join, so I don’t know exactly how they network there, but it’s an impressive accomplishment in terms of size. The other thing that surprised me, considering the popularity of LinkedIn among professionals and the degree to which it is being used for job search and recruiting, is how few Corporate Groups there are. When you look at the corporate groups, most of them are corporate alumni groups. That’s a great way of staying in touch and meeting colleagues, but the paucity of real corporate groups suggests that a lot of B2B firms are missing a good (free!) bet.

They seem to be following the guidelines Jim Leach set down in a thoughtful iMediaConnection article a few days ago. Particularly note that he stresses the need for new metrics to measure the effectiveness of community building in both B2C and B2B markets.

I’m happy to see a firm that is building community around its brand in a strategic way. That seems such a natural in B2B where people have genuine involvement with the products and services they use and like to interact with colleagues who share their interests and often their problems. It is a channel marketers can use to get out their messages. More important, it is a channel in which they can listen to the issues that are important to their customers!
Sphere: Related Content

AdReady-A Boost for Display Ads?

It’s only been a few days since I wrote about a study that predicts a rapid decline in display advertising on the web. It came from Borrell Associates, a specialist in local digital advertising. Then today the SmartBrief newsletter linked me to a WSJ Online Story on a start-up that allows advertisers to create and place their own display ads. Having found creating (and I use the word “create” loosely) display ads with considerable difficulty in the past, I was eager to check it out.

I went to the AdReady site and tried it and it didn’t work. So I followed my usual strategy. I watched the instructional video.

Actually, there were two issues. I had to create an account before I could play with their customization tool. That’s ok. But I couldn’t get the customization page to load. I switched to Firefox and it worked fine. That probably was a momentary blip; wouldn’t be the first time IE 2007 has caused problems.
So I used a template called AdDonna and customized an ad. I’m not pretending that it’s great copy; it was just a trial. But I liked their image better than my own, so I didn’t change that. I changed the background color to my signature purple, added the blog logo, and I had an ad! No charge up to this point. I wasn’t interested in spending any money, so I didn’t go on to the next step of deploying my ad. Looking at this screen, though, it appears to work much like choosing your target audience on Google using audience demos instead of keywords. I couldn’t find out anything about their ad network. A blog post from the Seattle Post-Intelligencer suggests that the company is being coy about that subject at present.
It’s hard for a small local or regional business to break into online display advertising. AdReady and counterparts AdItAll and AdBrite intend to change that. Based on what I saw today, they’ve got something going here!
Sphere: Related Content

The General Motors Wiki

Last week GM launched a corporate history wiki as part of its centennial celebration. At first glance that appears to be a risky idea. Won’t it draw out every GM-hater who’s had a bad experience with a GM car or truck? Won’t it become a public relations nightmare? Won’t it suck up a lot of the time of GM executives who ought to be contributing to returning the corporation to profitability? The answer to the last question is that it will absorb some time. Having looked around a bit, the answer to the first 2 is not so obvious.

First, GM has a history with social media. The best-known effort is the Fast Lane blog, the voice of Vice-Chairman Bob Lutz and other GM execs. It’s been going strong since early 2005. Lutz started podcasting in the fall of 2005. GM now has several blogs, but it probably says something that the blogs have their own microsite, GM Blogs, that cannot easily (at all?) be located from the main GM site. Was GM initially skeptical about what kind of public commentary Fast Lane and the others would draw? Probably. I’m not a follower of car developments and I haven’t visited these blogs often. However, when you look around, you get the feeling that a lot of these people are pulling for GM, whatever their reasons. Some are critical, some compliment something specific, and a lot have suggestions to offer—everything from design to advertising. I took a look at some of the recent posts, and I see Bob Lutz and many other GM executives responding, not to individual comments, but to themes that have come up. I also see notes from two people identified as “blog editor.” It takes some organizational infrastructure to keep this sort of multi-authored blog going. Is this a better role for PR people than sending out endless press releases that no one reads? I’d judge the blog efforts so far to be successful. GM is not escaping criticism, but it has a voice in the discussion.

With this experience under their belts, they decided to ask the public to help them write the history of GM. They put it in an interesting context—GM Next. They want to honor their history, but they’re moving on into the future. Good spin. The GM Next initiative is featured on the corporate home page. Does that suggest a level of confidence about their ability to manage the program? Any of the links on the home page take you to GM Next. The conversation threads are design, tech, green, ideas and global. The wiki is prominently featured. The home page of the wiki itself shows a well-thought out structure and there’s lots of information to get you started if you want to contribute. It shows that it’s a serious wiki with View Source and History tabs to help HTML-knowledgeable contributors and those who want to see previous versions of the page. Categories that are well worth exploring are the “How to Use, FAQs and Meet the Experts” pages.
A strategy issue worth considering is that the wiki was open to the experts before it was made publicly available. There is already content that sets the tone. There are “Rules of the Road” to be followed, but they are pretty minimal. In several places it is made clear that all contributions will be moderated for adherence to those rules and that their team of experts (which will grow as the wiki becomes active) will moderate disputes about factual matters. They promise that contributions will not be edited, just screened to ensure they meet basic standards.

I’ll admit that I started this exploration with considerable skepticism, even though I was aware of Fast Lane. What I wasn’t aware of is how many GM people are active in the various social media efforts and how seriously they seem to take it. As a long time advocate of managers at all levels actually meeting and talking to customers, how can I not be impressed? My sense from the beginning has been that social media provide a way for managers to interact with customers in a way that is both effective (in communications terms) and efficient (in terms of managerial time).

GM’s wiki is an experiment in social media worth following. My sense is that their intentions are good, and I wish them well.
Sphere: Related Content

Visualizing the Social Graph

One of the fun things about doing research for a blog like this is that one thing leads to another. Yesterday I briefly mentioned the term “data visualization” in the context I think most marketers use—how can we transform mountains of raw data into graphics we can use for understanding and decision making? At least, that’s how I’ve thought about it ever since statistics and marketing research courses.

But I ran across a couple of sites that reminded me of something else—the “social graph.” With the rise of MySpace, Facebook and specialized social networks, it’s a concept that has been much discussed over the last year. My personal opinion is that it is not different from the idea of social networks that many of us remember from sociology courses; Wikipedia has an article that describes the sociological concept. Chris Brogan has a brief video that does a good job of tying the two concepts together and Mitch Joel has a thoughtful post on the subject. The Economist had a contrarian view last fall.

“Social graph” is clearly the term used in the social networking space, so I’ll stick with that. We are all part of multiple networks today, for better or for worse. Even for us as marketers, some of it is personal, as when I posted my travel photos on Flickr and invited friends to view them. Others are clearly work; probably most of the people who read this are on LinkedIn or some other professional network. Some would be best described as research. I continue to maintain that all marketers should have multiple accounts on sites like Facebook, MySpace, Second Life and other new media sites. We should join some groups and visit from time to time. How else do we know what’s going on? Having it filtered through employees or our teen-aged children is just not the same thing.

The really cool thing I ran into is a set of applications listed on Mashable.com that allow users to visualize networks of various kinds. These are only two of the 16 applications they list:


That lead me to check out similar applications on Facebook. I’m not a serious user, I’m a research user, so I was astounded to find that there are over 15,000 apps that Facebook users can put on their sites. Here are some of the currently most popular—notice how many of them connect users with one another. No wonder that WSJOnline reported this week (subscription required) that MySpace had opened its platform to allow developers to build apps for it! Lack of this “fun stuff” may be one reason MySpace has fallen behind Facebook. If you are into diy, Google recently announced a Social Graph API that allows users to build graphs from publicly-available data and connections.
Finally, it made me stop and read the Terms of Use and Privacy Police on Facebook. That’s a subject in and of itself; I’ll return to it next week.

The implications for marketers and advertisers are clear. In the past, we had to conduct segmentation studies in order to identify homogeneous groups of customers and prospects. Now users are doing it themselves. How far should we go to take advantage of the social groups that are forming on the web? That presents troubling issues of privacy and trust that marketers need to consider openly and carefully.
Sphere: Related Content

Are B2B Marketers Getting More Social?

A few days ago I complained about the difficulty of finding data about B2B use of podcasting. Maybe it was just because I was paying more attention (isn’t that a known psychological mechanism?), but I soon ran across some data suggesting that B2B marketers are making use of social media.

Studies by the ANA/BtoB Magazine and Direct Impact Marketing/Buzz Marketing for Technology agree that blogs and RSS are among the most used of the Web 2.0 tools. Add to that Web 1.0—websites, email, SEM and webinars. It suggests a rich menu of tools on which B2B marketers can spend the increasing proportion of their budgets that are going online (both eMarketer charts, newsletter, Jan 14, 2008).

According to Marketing Charts:

B2B marketers have adopted blogs and RSS more than other Web 2.0 tools such as wikis, according to the report; moreover, smaller marketers - the Davids among the Goliaths - are at the forefront: [emphasis mine] Some three-quarters of surveyed marketers that have deployed Web 2.0 tools are in companies of 10,000 or fewer people.

In the B2B arena where marketers usually know who their customers are and how to contact them, most companies still have a long way to go in establishing meaningful online dialog with their customers. There’s little doubt that B2B marketers are afflicted by the same concerns about new media as their B2C counterparts—lack of understanding of new media and desire to maintain control over their message in particular. The same arguments apply—marketers need to engage with their customers in as personal and relevant way as possible, and they must understand what is being said about their brands on all platforms.

The trend is in the right direction, but marketers in large corporations seem to be behind the curve. They need to reexamine the policies that inhibit employees from freely engaging with customers and provide incentives for meaningful use of new media to encourage relevant dialog. They’ll not only have happier customers, they’ll be making better use of those marketing budgets.
Sphere: Related Content

Podcasting for Marketers

In recent weeks I’ve been talking to two people in very different situations about adding podcasts to their marketing communications offering. The situations are different, but the common theme is the desire to provide detailed information and various perspectives to identifiable niche markets. I’ve been looking at both the mechanics and best practices for those who wish to podcast. For marketers who don’t want to podcast themselves, podcasts that successfully reach their target audiences offer another advertising opportunity.

A 2005 post on the O’Reilly digital media blog outlines the basics. It all still applies except that the hardware changes quickly, so you’ll want to look for latest, well-reviewed pieces of equipment. The good news is that not much is necessary—a mic for your laptop and podcasting software will get you started. Active podcasters like to record in the field and there are many cool new devices for capturing live events. You’ll also notice that there is more software available for Apple systems than for Microsoft—surprise, surprise!—but whichever operating system you use, you can find what you need, and free software will meet most needs. That said, it’s not the technology that’s the issue, it’s identifying the need for podcasts and integrating them into your overall marketing communications strategy. Christopher Penn, CIO of the Student Loan Network, is well known in the podcasting community for the success of his podcasts. He has a clear target audience—prospective and actual college students and their parents. They need the information in his podcasts and he makes it easy to subscribe, even pointing out that an iPod isn’t necessary. Most podcasters simply have a page on the main site. Chris has a well-organized microsite that is visually integrated with the main site while it focuses visitors on the podcasts. He podcasts on a consistent schedule and often brings in outside experts. His microsite is welcoming to parents who aren’t into the new media scene, while it encourages their children to do things like “add this to your Facebook page.” It also has other features that engage his high-school and college age audience such as links to “free stuff” and job search information. Brian Carroll offers good advice for beginning podcasters on Marketing Profs (free registration required).

Lesson #1: Producing your own podcasts must be an element of overall communications strategy. In developing a podcasting strategy, ask questions like “is this a customer acquisition or retention mechanism?” and “how am I going to attract listeners/subscribers to the podcasts?” Answering the second question will put the issue of integrated marketing communications squarely into focus.

It has been a couple of years since leading-edge marketers recognized that advertising on or sponsoring podcasts was a targeted advertising opportunity. One of the first to acquire sponsorship was MommyCast, still sponsored by Dixie. Since I was last on the site they have added a weekly Internet radio show to their product line. Young mothers rely heavily on the Internet for information and this successful podcast has turned out to be a great way to reach them. Ad networks Radio Tail and Wizzard Media help marketers reach the niche markets represented by podcasts and Podbridge/Volo Media offers metrics services. Here’s some advice for advertisers.

The conventional wisdom is that B2B marketers have been slow to adopt podcasts, which seem a natural for reaching customers with current developments. Current data about B2B podcasting is in short supply, but a directory called PodFeed lists over 600 podcasts that have the tag “business.”

Lesson #2: Marketers can sponsor individual podcasts or use ad networks to reach podcast audiences with their online advertising.

Podcasts probably aren’t for everyone (marketer or user!) but their ability to deliver information to customers and advertising to targeted audiences makes them worthy of consideration.
Sphere: Related Content

Adding Widgets to the Advertising Mix

In your list of the tools that make marketing diy these days a useful entry is the widget. You see them all the time; the “Subscribe by email” box beside this post is a widget supplied by services firm FeedBurner.

There are two types of widgets. There are widgets like “Subscribe” that can be embedded in any web page, including this blog. There are desktop widgets. You might have downloaded the weather widget from weather.com. It also gives you traffic reports, and while you’re there you can also download a widget linking you to your favorite major league baseball team. What a deal! Bringing all these visitors to Weather.com does great things for their advertising rate card. Having their downloadable widget works for Weather.com because it has a high awareness level, and everyone needs a weather service on their desktop, don’t they? They also need affordable air fares. Southwest Airlines booked over $100 million in fares in the first year of its Ding widget, which is customizable for the cities you favor.

Most marketers are especially interested in widgets that can be embedded in other sites and drive traffic back to the sponsoring site. These widgets can build awareness for newer businesses and generate sales, especially for well-known trusted sites.

Staying with weather for a minute a company called Nimbus has a cute weather cloud that publishers can embed on their site as a small bar, shown below expanded. It provides a lot of location-specific weather information and a link back to Nimbus to find out more. The publisher can also sell advertising on the weather widget. Enough to pay for this commercial widget, I wonder?

Sites like eBay can offer a toolbar made up of widgets. eBay also offers tools to help programmers write widgets that link to eBay. There are a number of sites that offer diy tools for widget development. One of those is MuseStorm. Personally, I’m a bit skeptical of their claim that it’s as easy as creating a PowerPoint presentation, but the tools do look easy to use.

I created one in Google Gadgets for another blog, and it was as easy as Google said it would be. It turns out, though, that you can only put Google Gadgets on Google pages, and I wanted it for a blog with another provider, so that effort didn’t work. But creating the widget was easy, and in the right situation I wouldn’t hesitate to try it again.

I added one to this blog yesterday; see Sphere below. It turned out as a link, not an icon, because I can’t figure out how to include their sphere symbol. But it works. I got the idea from the Wall Street Journal Online. Look at the bottom of their articles, for “Related Articles and Blogs.” They have a staff of expensive IT people to customize the look; I have only me—not very skilled and free. And I have the same functionality as the WSJ on my blog. Think about that!

Sphere: Related Content