Test Your Knowledge of Split Testing
Dear Business Builder,
Happy Friday and welcome back to In the ‘Net Trenches. Have you ever wondered why it is that seemingly similar situations can sometimes have such a tremendous difference in outcome?
Take for instance two men of similar background that attend the same university … both with the exact same major. Both graduate with the same GPA and go on to take the same position with the same Fortune 500 corporation. Over time, one of the men moves up the ranks eventually becoming the CEO. Yet the other man languishes in the same position year after year.
The man that becomes the CEO must have an edge over his counterpart. It’s likely you’ve immediately begun to think of the qualities that make one man succeed over another. Perhaps he is more confident, more driven, smarter, a better communicator, more relevant and more resonant.
On the surface these men appear to be nearly identical … but one ultimately pulls ahead.
What I’ve just described to you is the exact same scenario as a direct response A/B split test. Today’s issue is not about how to move up the corporate ladder. It’s about how to succeed in split testing your web campaigns so you’ll have accurate, measurable results every time.
Give me an A … A!
Give me a B … B!
What’s that spell? A/B!
To start in split testing your web campaigns, you must first understand the fundamentals of an A/B split test. An A/B split test simply takes one list and splits it in half into two test panels. But you don’t just split the list right down the middle.
Let’s take, for example, a list of 100 people. In order to do a proper A/B split, the first, third and fifth record (and so on) would go into list A. And then the second, fourth and sixth record (and so on) would go into list B. By splitting the list by every other record, you get a random sample of potential customers.
By doing that random sample, it cannot be argued that one segment of the list performed better than the other because of similarities in the list segments. So to give another example, in direct mail, lists are typically maintained in zip code order. By splitting the list into A/B segments, you’ll get two lists each with a random sample of people across a whole region. If you just split the list in half, you’d perhaps end up with a list of people living on the east coast of the U.S. and a second list of people living on the west coast.
In the above example, it could be argued that one list performed better than the other due to regional preferences or similarities. So, if you were ultimately testing two different headlines and you didn’t split your list into A/B segments, how could you possibly know whether it was the headline that improved response or a regional preference for your product? In this testing scenario, you’d have to throw the baby out with the bathwater … your test result would be invalid.
I’m about to lose control and I think I like it …
The winner of your split test is called the control. In all of your testing, you want to test against and hopefully beat the control. When you beat the control, the winner becomes your new control.
What’s important to remember when testing against your control is not to change too many elements at one time. If you are testing headline and deck copy … then only change headline and deck copy on your test panel. Don’t try to test price at the same time, because you won’t know whether the new price or the new headline ultimately made the difference.
You must do calculated and measurable apple to apple tests to accurately crown the winner.
Also when testing copy and creative, it’s important to make your tests swing for the fences. You want to make significant changes to the element you’re testing that can make a bigger impact on your results. Let’s say you are price testing and your control price is $49. Try testing a price that is quite a bit higher like $79 or quite a bit lower like $29 (if margins allow). You might be really surprised at the result.
In the higher priced scenario, you might find that you need fewer orders to break even yet bring in about the same amount of revenue. In the lower price scenario, you might find you bring in 50% more customers with a big increase in revenue. Both of these outcomes could be really significant depending on your goals. Remember, dramatic changes can make for dramatic winners.
Test!? I hate tests!
If you’ve been a student of direct response marketing for some time, you’ve heard over and over again the importance of continuous testing. And it can’t be stressed enough. You are only as good as your next control. You must have the attitude that every control can be made better and eventually beaten through testing.
Here are just a few elements of your campaigns you should consider testing to get a bump in conversion:
- Headlines
- Prices
- Layout
- Page Width
- Order Button Colors
- Offers
- Subject Lines
- Ads
This list is in no way exhaustive, but is a great place to start. Have fun with it.
Times change, sentiments change and preferences change, and the only way you learn what is working now is through ongoing testing. You can never really predict what will work until it’s been tested.
Heck, even after fifteen years in the direct response biz, I make predictions and am frequently wrong.
Hope that helped and have a great weekend!
Until next week,
Julie McManus
Editor, In the ‘Net Trenches
THE TOTAL PACKAGE
And Web Media Goddess
P.S. Are you in the ‘net trenches? Do you need help? Send
me an e-mail to AskJulie@MakepeaceTotalPackage.com and
I just might answer your question in an upcoming issue.
P.P.S. Have you checked out The Total Package affiliate program lately? We’ve added tons of new creative to help you earn cash on any new subscriber you refer and we’ve opened our archives up for the pilfering … Click Here to check it out NOW!
Looking for resources related to this article? Try some of these.
Looking for more of Julie’s articles? Check these out.
Looking for past issues of The Total Package? Click here for our archives.
 |
Want to share or reprint this article? Feel free. Just give us full attribution and a link to our Home Page when you do. |
Attribution Statement: This article was first published in The Total Package. To sign-up to receive your own FREE subscription to The Total Package and claim four FREE money making e-books go to www.makepeacetotalpackage.com.
Related posts
2 Comments »
Join the Discussion!
Let us know what you think. Or ask us anything. Or offer your own sage advice.
The only rule: RESPECT THIS HOUSE! Postings that contain abusive language and/or personal attacks will be cheerfully VAPORIZED. One cross word and – POOF! – your well-thought-out post will be gone in a puff of smoke.
– Clayton
Comment by Mike Humphreys — November 30, 2007 @ 9:37 am
Nice article Julie. By my estimate, over 90% of marketers don\’t test. It\’s an easy way for copywriters to position themselves as a conversion increasing expert simply by using testing with their copywriting skills.
Take care,
Mike
Comment by John Klein — November 30, 2007 @ 4:28 pm
Julie, While A/B split testing has been the workhorse testing method for years, newer \”multivariate\” testing methods, such as factorial arrays allow you to test several factors - headlines, illustration, price, color,ect. – all at one time. It\’s much faster that having to wait to do several A//B split testing on each element.
John
Comment by rcmdnerr — January 9, 2008 @ 8:43 pm
czmhbrxx foojaocp http://nkiqkaku.com pzfziyxz nbvxegzy [URL=http://odedoien.com]lmcjpeme[/URL]