23 . 4 . 14

Astounding Results After Responsive vs Non-responsive A/B Testing

We tested responsive vs non-responsive versions of the same page. The results we saw afterwards were astounding.

The mobile-first concept is not a new thing or a novelty. This discussion has been going on for a number of years now in the industry – but the real truth is there are still brands out there that are late to the party.

We have seen the number of visits from mobile device-types increase dramatically in the last year or so, and know that the mobile-first concept addresses usability issues for these device types. This is especially so for ecommerce websites.

However, in the world of hard ecommerce metrics like transactional targets, customer growth and product inventory churn – usability is usually at the bottom of marketing priorities, meaning the process to transition or adapt to a mobile-first world becomes slow at best and painful at worst.

But what if I say we can use these very same hard metrics to prove that mobile-first web design is key to improving conversion rates, thereby equating to increased revenue figures?

The Proof

Here’s a case study we’ve done for one of our clients in the travel sector.

We know that the travel sector is generally slow to adopt to responsive design, but with a website that is seeing a sharp increase in mobile use, we know this is where we can make the biggest positive impact with this client.

mobile device type usage

Figure 1: Mobile device type usage taken from Google Analytics

 We analysed their data in Google Analytics, selected one of their key destination pages which we identified as a Highest Revenue Potential focus due to the size of the traffic to the page, the amount of other online marketing activity on the page and the price of their tickets for that destination. We understood that customer retention on mobile on this page is very poor with significant bounce rates.

We conducted an A/B split test using Visual Website Optimizer and utilised CSS and jQuery to generate a responsive, mobile-friendly version of the page with key call to actions and content optimised for conversions. This was tested against the control – the original version of the page. We ran the test for a minimum of 14 days until we arrived at statistical significance. What we saw in the results was astounding:

Results of winning variation

Figure 2: Results of Winning Variation. Data taken from Visual Website Optimizer

 Right from the word go, the responsive variation produced more than 50% uplift versus the control’s conversion rate at 100% statistical significance! What this means is:

  • The responsive mobile version is ~50% better at getting users to convert (ie. to buy tickets)
  • This makes the responsive mobile version ~50% more cost-efficient in paid search
  • With PPC mobile ads being cheap already, this means the responsive mobile version will be converting more users at an even cheaper Cost per Click!

The case is clear: a responsive, mobile-friendly web page increases your client’s bottom line significantly.
Want to find out more about what Analytics & CRO can do for your business? Contact us here.

Trent Y. is Head of Analytics and Conversion at twentysix. Analytics, usability and conversion strategies. Loves coffee, interested in how data and technology can positively impact our lives. “Learn fast, learn often.”

Share This: