How do you know, when you are testing two advert alternatives, which is performing better?

If you assume that the quality of traffic driven by the two different is the same, then you can simply look at the click through rate – the conversion rate won’t vary. This is a reasonable assumption if the key messages and tone are similar, and if the website doesn’t receive many conversions (or can’t track them), it’s necessary.

However, if there is a chance that the different adverts are likely to drive a different type of visitor to the website, and sufficient conversions pass through the adverts, then it’s important to look at conversion rates.

But does this really go far enough?

Consider a casino website. They can promote their site using a small, easy-to-claim bonus, which small-time players can probably claim, or they can promote a much larger bonus, that only high-stakes players are likely to qualify for. Which will work better?

It’s likely that promoting a higher bonus is likely to deliver a higher click through rate. Or is it? Are searchers savvy enough to realise that they can’t claim this bonus? If so, they may be put off by this, and would respond better to a lower, more realistic bonus.

What about the conversion rate? If users weren’t put off by the higher bonus, will they be put off once they read the terms and conditions? If the landing page also has more realistic bonuses as well, this may not be a serious problem. On the other hand, has the bigger bonus attracted newbies – potential players that aren’t aware of how these bonuses work? If so, they may click on multiple websites before converting.

Is this sufficient? Can you tell from the click through rate and the conversion rate which is the more effective advert? Well – no.

Consider the likely behaviour of the visitors that do convert. If they’ve been attracted by different bonuses, their behaviour could vary significantly. If the bigger bonus appeals to the high stakes gamblers, then their lifetime value could be substantially more. On the other hand, if the people attracted by large bonuses are just bonus-hunters, playing just enough to claim their bonuses before moving on, then their lifetime values may be lower.

It may be that this can’t be answered – it depends on the back end systems that the advertiser has. But if you add a variable to the end of the destination url, and the back end can tag up new accounts with this variable, then after a few months, you’ll be able to ascertain which advert generates the more valuable players. Combining this with the conversion rate and the size of the bonus that was paid out, you can establish the value of a click from each advert. And from this, you can make an informed decision.

If this seems like an unusual situation, it really isn’t. Consider another example – a retailer selling discounted designer labels.

The advertiser decides to lead with the size of the discounts in one advert, and promotes the designer labels in another. Whilst the former may generate a higher click through rate and conversion rate, it’s likely that average order values (and possibly margins) will be lower for these customers.

Of course, it takes a lot of customers/players to generate reliable results. One big order or degenerate gambler shouldn’t be sufficient to make a major impact on the outcome. But for advertisers that can perform this kind of analysis, the benefits could be substantial. After all, there are probably thousands of advertisers who have squeezed their average order values, chasing improvements in the click through rate.