Split Testing

Split Testing Definition

Split testing is the activity of randomly distributing the incoming traffic to the website between two different URLs of the same page with the aim of improving the overall performance. According to the main goal of the page, the performance can be calculated using different KPIs: conversion rate, bounce rate, number of downloads, number of leads, etc.

The traffic involved in a split test experiment can either be all the visitors that access that specific page, or a portion of them, usually called a segment. There are different traffic segmentation options such as: by location, by referring page, by source, by technology, etc. 

So split testing uses traffic segmentation to randomly distribute visitors between two different URLs of the same page with the aim of determining which one performs better. This is why split testing is also known as Split URL Testing. These two terms are interchangeable. The URL split usually occurs between only two URLs, but tests can be run on multiple URLs at once as well. 

Split Testing vs A/B Testing

A/B testing and split testing are often interchangeably used. Although there no universally accepted definition for each of them, for many marketers, the main difference between the two is that for the first one, testing takes place on the same URL while for the other on multiple URLs.

So during an A/B testing experiment, the URL is the same for all visitors, but what is changed are the various HTML elements on that page. Because of the way, it’s set up A/B testing is usually used to test the impact of individual elements while split testing is better used for testing different page layouts altogether. 

So if you want to test the page headline, the hero image, a different call to action, etc, then A/B testing is the way to go. But if you have to different designs for the same page such as a long-form sales letter vs a short-form video sales letter, then split testing should be the preferred methodology. 

Another way to put it is that A/B testing is a more general term and it also incorporates the meaning of split testing, while split testing refers to a specific part of A/B testing, which is about running two or more different URLs head to head in order to establish a clear winner. 

Split Testing vs Multivariate Testing (MVT)

There’s a lot of confusion between split testing and multivariate testing, especially when several URLs are involved. Split testing, even when it involves more than two URLs is still split testing and not MVT.

Multivariate testing is a type of experiment that, similarly to A/B testing, takes place on the same URL, only that different elements of the page are automatically tested in order to find out which combination works best.

So for example, when testing the best combination between a different page headline, the hero image and a different call to action you need to create a single MVT test. If you want to do the same using A/B testing, you’d need to manually create 7 different tests. 

Split Testing Examples

One of the most famous examples of split testing is the Moz landing page case study done by Conversion Rate Experts. In this case, they tested between a short form (original) and a long-form sales page (improved version). As it turns out their improved version ended up generating more than $1 mil/year in new subscribers for Moz!

Here is another inspiring example from WWF, who by implementing a simple split test managed to generate an extra $100k to their annual income!

Methodology

For example, a high cart abandonment rate may reveal the need for changing either the design of the cart page or its content. If you decide to modify a page element without testing, you the risk of hurting the page key metrics even worse. Therefore, using split-testing software guarantees that any change made on a page is improving the metrics when the winner version is implemented.

With split-testing software, anyone can run experiments on the website’s most important pages. Declaring a test success requires the validation of the following conditions:

• the impact on the key metric of the variation compared to the original version has to be significant for the variation page to be declared the winner
• a statistical confidence rate of 95%, which is, usually, revealed by the split testing software – if not sure how to interpret statistics, read statistical significance topic
• both the original page and its variation have to gather enough traffic
• the test should run for at least 14 days
• the winner version(variation B, C, D, etc.) gets implemented on the website once it has achieved all the mandatory conditions mentioned above

Benefits for Implementing Split Testing

Some of the key benefits of using this marketing technique are:

• it allows measuring the significant impact of the solution over the website’s metrics that reveal the sales evolution, average order value or other business-related metrics
• it guarantees that once implemented, the change won’t affect the website’s performance
• it provides information when the tested solution doesn’t have a positive impact on the key metrics
• it helps with adapting the website’s design according to the volatile customer behavior
• it is risk-free: even if the variation doesn’t win, the website’s performance is not affected
• it allows testing on small cohorts (5% of the page traffic)
• it takes 5 minutes to set up a test once you’ve finished creating the A/B testing plan

What to Testmarketing Campaigns Can Perform Better With Split Testing

A split testing tool can be used to optimize the key metrics which reveal how you do against your marketing objectives. For instance, people who want to try a demo of a software product can be either interested in improving conversion rates for an e-commerce website or a subscription-based one. Therefore, if the e-commerce guy is landing on a page that doesn’t show him how to solve his specific problems, he will leave the website.

Website optimization goes hand in hand with traffic segmentation. As long as you identify the most profitable segments for your business, you can target them with conversion optimization efforts.

Analyzing behavior based on the traffic source, you’ll observe that people who come to your website from social media expect different offers or content than the ones coming from an organic search, for example. Therefore, based on the performance of the most profitable traffic sources, you have to create landing pages optimized for each segment.

“Most websites in the world don’t have a massive traffic problem, however, every website in the world has a conversion problem.” is the conclusion of Bryan Eisenberg, an expert in split testing best practices and conversion optimization.

What to Test

• Texts: descriptions, USP, headlines, call to actions
• Visual elements: images, colors, video
• Layout: buttons and forms placement
• Examples of pages that need optimization
• Register to webinar pages
• Download pages (ebooks, guides)
• Try our product page
• Adwords Landing Pages

Split Testing Tools & Software

There are multiple ways to run split tests, some more complex other more simple, some requiring more resources (time, money, know-how) while others not so much. Initially, split testing was done by large companies using software build in-house. Although viable, it was quite expensive and small to medium businesses could not afford it.

This is how SaaS (software as a service) platforms that offer these services were born. With plans that start with a few dollars/month or even completely free in some cases, now SMBs could implement split testing experiments with ease. One such SaaS platform that among other features, offers the possibility to run split tests, is Omniconvert.com. Signup here for a free trial and create your first split test within a few minutes from now!

FAQs