Split Testing Adsense

When discussion turns to which template works best or which ad format works best with adsense, you can usually rely on me to spout my default piece of wisdom; “test test test”. Problem is I am not sure everyone knows exactly what I mean. How do you test your adsense tactics?

One of the real benefits of online advertising for both advertisers and publishers is the detailed tracking that is possible. No need for focus groups and clipboards here, the data is all there in real time in black and white. You know what works and what doesn’t almost as soon as you have done it.

The problem is this immediacy can make you jump to conclusions. Clicks increase on the day when you post a picture of your cat dressed as Elvis and you think “hmmm … must post more rock and roll pets”. There are any number of factors that influence how many clicks your blog will get on any given day, not least being pure chance.

Reduce the effects of chance

To test any changes to your blog you need to compare results with all other factors being equal, as far as is humanly possible. This is where A/B split testing is useful. Split testing is where you compare one new change to the existing status quo. Your existing template or ad format is the “control”, your mission is to beat the results you get from that control with your change.

This requires two things;

1. Consistency – you need to know that the only thing being tested is the new change element. If other factors leak into the results you will not be able to make good decisions. It is easier to ensure consistency if you randomize what the visitor sees using programming.
2. Good Data – you need to be able to record the differences. Make sure you have good tracking in place. If testing ad formats give each their own channel name, if you are testing templates give all the ads the same channel name, one name for the new layout and another for the control layout, and implement a good tracking solution.

I mention using programming to ensure consistency. If you show randomly one version or another then you have effectively excluded time of day, referrer, content, etc from the equation. If you wait a day or an hour and make a change then wait another hour and change back you might be mistaking the effects of lunchtime traffic for the effects of your new design. If you do not feel comfortable programming you can still be systematic in your approach, you will not get as immediate results and will have to do more manual work.

Some simple code to illustrate how this might be achieved is below. It is a small piece of PHP that simulates a coin toss. The HTML output right now is just heads or tails but it could be anything from a banner to your adsense code.

heads

“;
}
else
{
echo “

tails

“;
};

?>

Many big sites do this sort of testing. Have you ever visited Google or Amazon and seen a difference in the design then gone back and it has changed back? This will be the companies split testing.

Significance and Confidence

Earlier I said it is easy to jump to conclusions. With split testing it is still easy to jump to conclusions but you are limiting the possibility somewhat. You need to wait for a significant amount of data. There is as much art as science with this kind of thing. If you are going to stick to truly scientific approach you will have to wait perhaps a few days or at least several thousand page views. Most people are not going to take that long and very often you will believe you see a difference before then.

There is a free online tool called Splittester that will give you an idea if the results you are seeing are significant or not. It’s there to sell an adwords ebook but you can ignore that aspect and still find it useful!

A lot of people in direct marketing go way further than simple A/B split tests and do multiple tests and use advanced formula to reduce down the amount of time it takes to get an answer. While you might want to try these sorts of things, and there are software packages that are designed to help you run Taguchi and equivalent tests, you must remember there can be an “annoyance” factor. Rapid and varied changes to your blog could be distracting to your visitors and just plain put them off your whole blog.

Summary

Seem like something you would do? Too much hassle? Is it something you already do?

I would be very interested to hear how you test your changes. Do you just post up the change and see what happens or do you have an approach?