The Bachelor opts for split testing 25 creatives then pruning them off over a 2 week period until his campaign of love is fully optimized.
Split testing is one of the core process in affiliate marketing. However, it seems every affiliate has their own unique definition as to what their ideal split test is. You could get a spread of answers as long as a piece of string.
Recently I have been asking myself two questions:
How many variations should you test in a campaign? If you split test too little, you may be missing out on profits. If you split test too long, you may be treading water.
How long should you be testing each of these variations for? If you split test too long, you will lose money. If you stop split testing early, you could delete the better performing ad (and lose money).
This is my theory:
How many variations should you test in a campaign?
Assuming that: You are using a landing page. You are targeting one country. You know your demographic. A campaign is only for one traffic source.
Before profit:
You should test 20 ad copies. Pause 15 of them. Then split test 5 different variations of your landing page. Make sure you create tests with SIGNIFICANT differences. If you don’t see a huge variance with results, you are doing it wrong. This should give you enough data to give an idea if your campaign could be successful without spending too much money. If the offer you are running is advertised all over the internet, as opposed to a fresh new offer that hasn’t been proven, I would spend a bit more money testing since you know it works.
If you see potential, then stick with the variations that performed the best. After make new tests tightly based around those variations. This phase should push you into the green, or give you a sign that you should ditch the campaign. Tightly based variations of profiting ads will also help with the longevity of your campaign preventing against the dreaded banner blinds.
After profit:
This is the interesting part. So you have a profiting campaign, yay monies! Now you can take your foot off the pedal and cruise. WRONG! This is when the crazy little test you never cared about before comes into play. Now look at the test created by my favorite marketing website (marketingexperiments.com). This test was for six landing page headline variations.
These variations were tested against the original headline. An increased conversion rate of 20.39% is not bad for a few words switched around on a landing page.
Changing things like background colors and “call to action” text from the default “Click Here” to something like “Download it Now” have a bigger effect then you would imagine.
Keeping all this in mind, don’t get carried away here. Yes you could start tweaking borders, drop shadows, fonts etc. but think about the time spent compared to possible profits gains. As soon as you start testing for fraction of a % conversion rate increases, move on to a new traffic source.
How long should you be testing each of those variations for?
This is a tough answer to crack. Obviously there is no universal answer for it.
If the payout is less than $8 I like to give 20 unique ads each a shot at spending the payout amount. You can easily find some accurate trends with those figures.
If the payout is higher than $8 then it starts getting tricky. What I usually do here is create 3 significantly different ad groups, which each carry 20 similar ads under them. If the payout is $30 then I would give each ad group a budget of $90. For most traffic sources CTR is king so once half the budget is spent, prune down half your ads with the lowest CTR.
- What is your definition of split testing? What is your formula? Are you a Bachelor or a Hugh Hefner?
P.S – For those of you who scrolled straight to the bottom of this post because you are too A.D.D to read it all, just remember to set some limits for split testing. Not too little. Not too much.
+rep
nice post man as always! besides google website optimizer u know any other tool that can help you to split test?
Ricardo
Hi Ricardo,
Sure there are tonnes of tools you could try. Some I know of are Verster, Offermatica or Optimost.
http://www.whichmvt.com is a site by the conversion rate experts folks & has a pretty useful comparison matrix of the tools available.
Wow! Writing a post about!
Thanks Lorenzo! Every single post of yours is packed full of
thoughts and advice, I really appreciate the depth that you go into too
Lorenzo, you are talking about PPC here, right? Or the same can be applied to some other methods as well?
BTW, what's you fav PPC network?
This can be applied to CPC and CPM of course there are some differences between them. My favorite PPC network would be Facebook…just.
It's amazing the difference between reading a blog written by a person who actually makes monies online and someone who just blogs about making monies online. This is a SOLID post bro, keep on keepin' it real!
amazing info on this site
Hope you will be at ASE we all owe you drinks.
Solid tips Lorenzo.
Sorry I forgot to add my input.
Basing that my traffic is coming from a media buy (tons of volume) here is what I like to do:
Since I'm paying CPM my CTR is my lifeline. Without clicks to my landing page my campaign doesn't have a chance. Over time as visitors see my creative CTR is going to drop. Therefore I want to start with a creative that has a lot higher CTR than my target goal.
For instance, let's say based on my calculations I set a CTR goal of 0.40%. Based on other tests I know my average landing page is going to get a 29.5% CTR to my offer and once traffic reaches my offer it's going to convert at 4% (before I take anything to a media buy I have some initial numbers in mind). So if I purchase 800,000 impressions at $1.75CPM that's $1400. If my CTR is 0.40% I'm going to receive 3200 clicks to my landing page which 944 will make it to my offer. Converting at 4% @ a payout of $35 I'm going to profit $1321.60. Still not breaking even.
Therefore I like to test 10 – 20 creatives per size (728×90, 300×250, 120×600, 160×600, 468×60). After I give each creative a fair amount of impressions, I like to eliminate all but about 5 per size. This is the time I like to begin split testing my landing pages. If I was to start from the beginning I have way to many variables and will more than likely spending money so recklessly I'll lose before I have an idea of what's going on.
My first landing page test I want all my pages to be different. Normally I start with about 5 landing pages and test them through my remaining 25 creatives (5 per size). I like to run things for about 24-48 hours to give everything a fair chance to run. I've noticed that some landing pages can perform better at certain hours since they may target a different demographic.
After 24-48 hours is up (all depending on how much money I've spent) I like to find what I call 'the winning pair'. This is taking my winning landing page and winning creative and 'plucking' them out. If two creatives are running really close CTR wise, I'll take them both. Once I have this winning pair it's time to get really picky. I make 5-10 different variations of my 'winning' landing page testing several different aspects: headlines, call to actions, colors, pictures, etc. While we know that this landing page is the winner out of the other 'formats' it's now time to see if we can't get an additional 1-2% out of the conversion rate. I do the same with my creatives. I take my winning one and create several more different variations of it. If I can get just a half-percentage higher of a CTR that's going to make a huge difference in my bottom line (especially with volume).
After another 24-48 hours I'm normally able to find one-two creatives per size and pick the winning landing page. At this point my campaign should be pretty profitable. If it's still 'treading water' its probably not a good fit for a media buy. After about a week or two I'm going to have to start the creative testing over again as it's likely my ad has gotten 'stale' and CTR is beginning to shrink.
Sorry for such a long reply, hope this adds some input!
Hi Ryan,
Thanks for your input! Your formula for testing seems solid. I see you like to use the Taguchi method, what do you use for tracking?
Hey Lorenzo,
As I told your reader below we actually became so frustrated we paid a freelancer to write us a custom PHP script to help with split testing. Additionally, we also use Prosper 202 (I love this script), the Zedo ad server, Google's Website Optimizer, and Google Analytics. I'm a pretty big 'tracking nerd' as I like to drill everything down as far as much as possible.
Thanks again for your solid tips, it's nice to see someone that knows their stuff;). We should def meet up @ ASE!
Custom scripts are the way to go for sure!
Another awesome post, keep em coming!
Mrgreen and Ryan Gray (poster above me) , which program do you use to split test landing page variables?
I am currently using a program called conversion chicken from the Warrior forums. Its cheap, simple and really easy to work with but I am looking for something more advanced.
Thanks
I actually became so frustrated to get what I wanted I paid a freelancer to write some custom software. In my opinion it was well worth the money spent.
Additionally, there is a lot of other resources out there free of charge:
1) Google Website Optimizer (http://google.com/websiteoptimizer)
2) Prosper 202's A/B plugin (http://prosper.tracking202.com/scripts/split-testing-landing-pages/)
Both are fine for A/B split testing but for multi-variance I'm yet to find a sufficient solution… I will be trying Verster though it looks decent from what I read about it.
hey, so with the 5 LP's you test you want sig differences, are you talking a review, an opt-in, basic LP, a quiz…what other options are there?
how do you track if certain ads work better with certain LP's?
great post green,
love your post scriptum lol
i'll be back to read the whole post tomorrow
For me, putting up the different variables is the easy part; the hard part comes when trying to remember what I was trying to split-test in the first place, and evaluating the results.
I REALLY should try using Conversion Optimizer.
Thanks for the great post, as usual!
Once again Lorenzo, an excellent post. Your blog has really become a great place for people to share and exchange ideas, as well as an invaluable source of education for affiliate marketers who are on the verge of taking it to the next level but need that extra lil oomph to break on through.
You definitely have my mental gears turning after I read your newest post!
[...] has written a few excellent posts on split testing and how to do it more [...]
This is the first time I saw such detailed and quantified discussion on split testing with solid numbers – thanks so much!