I recently read a blog that suggested you should always begin testing with your shopping cart (with almost no caveats). This seemed a little unusual to me so I threw together a few different scenarios to see if this really was the case. Several other blogs over recent weeks have also suggested testing the checkout pages should be done first. Whilst these blogs were talking in general terms, they were written in such a way that implied checkout testing should always take priority.
Different (rather made up) scenarios
Consider the following: Now suppose we test the checkout first with the following results: Which would leave us with the following result once rolled out, So after 1 month we have a significant uplift in conversions. Now what if we had tested the homepage instead of the checkout page and achieved a 5% uplift in Category page views (and back to the original conversion rate)? Obviously in this scenario it has been much better to improve the homepage first then move onto testing the category pages. Consider the following scenario, (a site that has issue getting people to view products) In this case, the product page views (that are required in order to purchase a product) are not as high as they should be. If this is the case, it is more likely that a large uplift could be attained by testing these pages first! See below: Product page views have been improved from 20% of category page views up to 40% of category page views. For the checkout test figures, I have assumed an improvement from 60% to 80% of basket additions (quite an improvement I know). Now in this scenario we see that a large uplift in sales has been achieved by improving the problem area first.
Alternative Factors To Keep In Mind
There are many different research methods available ranging from Google Analytics to user testing, all which can point you in the right direction for test page selection. Other factors to bear in mind include: 1. Volume of traffic
- It is likely to take much longer to test a page with minimal traffic volume than one that receives a large quantity of visits. Because of this, testing a page earlier in the conversion cycle has two benefits. First the pages earlier in the cycle usually have a greater number of page views and, secondly, they will then drive more traffic through to the pages with fewer views, in turn speeding up the test time on these pages.
2. Positive research can help point out risky tests
- If you find the majority of visits are using a feature (such as a filter system, navigation), then you could assume testing this feature could prove to be of greater risk.
3. The quality of traffic going to the pages
- Spammy, irrelevant traffic all going to your site pages will skew results both when researching and testing. To get round this look at segments of traffic where the data is available, which will qualify the pages performance.
4. Don’t just judge a test based upon the end result, use analytics tools to see what effect the page changes have had.
- Suppose you had a major issue getting from the homepage to the correct category page AND category page to the product page. It is likely the figures will highlight the homepage to category page issue however if the number getting through is low enough they may not then highlight the product page issue as greatly. If after the first test, conversion rate has only slightly improved, investigate what else has happened, if the product pages are now being many more times then you know where to test next.
Moral of the story: Research your site FIRST then test areas that you know to be causing the issues!