I’m not here to settle this debate. Instead, this guide is a comparison of two of the tools that are applicable if you are trying to run CRO testing on a budget, especially if you believe the former.
In this case we are comparing VWO (Visual Website Optimiser) and Google Optimise (Beta).
Note: this review is using the free version of Google Optimise.
Cost
The version of Google Optimise that we are reviewing is the beta version of the site, and as such is free. VWO, on the other hand, costs a fee per month, depending on the traffic level that you are testing.
However, if you want to get the most out of Optimise with no limitations, then the flat rate Google charges is significantly higher than VWO.
Winner – If you have literally no budget, then obviously the free version of Optimise makes sense, however for similar capabilities, then the winner in terms of cost is VWO.
Implementation
Both implementation tools offer the option to hard code the JavaScript snippet onto the site that you are testing on, with their implementation information virtually the same on the interface. While it’s not recommended to fire VWO through GTM (Google Tag Manager), Google Optimise will work using this method, albeit with the caveat that there may be a slight flicker upon page load due to synchronous loading.
Winner – Google Optimise, due to the flexibility of having more than one option for implementation.
Script Insertion
Both VWO and Google Optimise work in a similar way when inputting script onto a site – you can either edit the elements on the variation by dragging them around and resizing manually within the editor pages, or you can create code and paste this onto the editor pages. For the former method, both are much of a muchness.
VWO offers a simple method for doing the latter:
Clicking on Edit Code opens two tabbed boxes that you individually drop the separate JavaScript and CSS into; saving this will input the changes onto the variation.
On the other hand, Optimise uses a rather less straightforward way for inputting code. You have two options:
The first one involves wrapping your JS and CSS into one long piece of code, and adding it all as JavaScript into the option highlighted above. If you have a good dev resource, this works well, until you hit the character limit of 12,840. We were required to use a code minifier to resolve this.
The second method is, in essence, the same as the VWO implementation. However, instead of both JS and CSS insertion being in the same place, you are required to search long and hard for the CSS insertion button.
Winner – VWO
Functionality and Features
It felt like the easiest and most digestible way of reviewing this part of the two tools was with a good old-fashioned list of features that both tools offer.
VWO: · In built editor · Previews on all device types · Heatmaps · Click tracking · Form analytics · Surveys · Website reviews
Google Optimise: · In built editor · Previews on all device types
Winner – VWO
Integration with GA
Integrating Google Optimise with GA couldn’t be simpler – while the implementation involves following a step by step process on the site, once this is done, the ‘Experiments’ section within will track all of your results automatically, based on what you have chosen your conversion metrics as on Optimise.
Integration with VWO is less straightforward, and either involves linking via a Universal Analytics account or via GTM. Also, once this has been done, custom dimensions must be set up on GTM, followed by Custom Report/Segment creation in GA itself to keep track of your experiments.
Winner – Google Optimise
Experiment Types
Both platforms offer A/B testing, multivariate testing (MVT), and split testing (redirect experiments). However, only VWO offers multi page experimentation.
Winner – VWO
Restrictions
Optimise (the free version, remember), has a few limitations. The main one of these is the inability to pause a test once it has started running – you can merely finish the test and restart a new one, which is not ideal if you encounter a problem mid-test. As well as this,
Optimise only allows you to run three tests simultaneously, as opposed to a potentially unlimited amount of tests on VWO (depending on your traffic allowance that you have paid for monthly).
The only real limitation on VWO, on the other hand, would be the slightly clunky user experience on the interface, which we will be exploring at a later date.
Winner – VWO
So, which is better?
The real answer to this question is: it depends.
The scores above would suggest VWO is the winner, coming out on top in five out of the seven categories that we reviewed, but it is very subjective.
My overall advice would be: if you have more resource (particularly in analytics and web development), then VWO is the tool to use.