Skip to Main Content

Optimization glossary

Bucket testing

What is bucket testing?

Bucket testing (sometimes referred to as A/B testing or split testing) is a term used to describe the method testing two versions of a website against one another to see which one performs better on specified key metrics (such as clicks, downloads or purchases).

There are at least two variations in each test, a Variation A and a Variation B. Metrics from each page variation are measured and visitors are randomly placed into respective ‘buckets’ where the data can be recorded and analyzed to determine which performs best.

Companies that market and sell products or services online rely on bucket testing to help them maximize revenue by optimizing their websites and landing pages for conversions.

How it works: an example

Let’s look at a hypothetical example. Each bucket test begins with a hypothesis that a certain variation on a landing page will perform better than the control. Say you have an existing landing page for a free nutrition eBook, Eat Raw Foods and Live Longer.

The button on the bottom of your landing page’s sign-up form says ‘Submit,’ but your hypothesis is that changing the text to ‘Get Your Free Copy’ will result in more form conversions. The existing page with the ‘Submit’ button is the control, or Variation A. The page with ‘Get Your Free Copy’ on the button is Variation B. The key metric you will measure is the percentage of visitors who fill out the form, or a form completion.

Because you have an ad campaign driving several thousand visitors a day to your landing page, it only takes a few days to get the results from your bucket test. It turns out that ‘Get Your Free Copy’ has a significantly higher click rate than ‘Submit,’ but the form completion rate is basically the same. Since the form completion rate is the key metric, you decide to try something different.

Bucket tests & conversion optimization

Bucket testing plays a big role in conversion rate optimization. Running a bucket test allows you to test any hypothesis that can improve a page’s conversions. You can continue to try higher-converting button text for Eat Raw Foods and Live Longer or you can go on to test other hypotheses, such as bolder headline copy, more colorful imagery or arrows pointing to the sign-up button that will get more people to convert.

Companies spend millions of dollars to drive traffic to landing pages and websites that promote their product or service. With simple variations to page copy, imagery, and layouts, you can conduct a series of bucket tests to gather data and to iterate toward your highest-performing version of the page. You simply create variations of the page, changing one element at a time and measuring key metrics, then collect the results until reaching statistically significant results for each experiment.

Bucket testing can make a significant impact on conversions per page, resulting in revenue increases on your highest-trafficked pages.

Bucket testing can also help to eliminate subjective opinions as deciding factors in a page’s design or layout. The author of Eat Raw Foods and Live Longer may think that her photo will drive more customer demand—or she may insist on a rainbow palette of colors.

With bucket testing, there is no need for debate on what design or page elements will work best to convert a customer. The quantitative data will speak for itself, and drive the decision for you.

Tests should be prioritized to run on your most highly trafficked pages, since you may need hundreds or thousands of visitors to each variation to gather statistically significant data. The more traffic a page receives, the quicker you will be able to declare a winner.

Common page elements to test:

  • Headlines and sub-headlines: varying the length, size, font and specific word combinations
  • Images: varying the number of images, placement, type of imagery (photography vs. illustration) and subject matter of imagery
  • Text: varying the number of words, style, font, size and placement
  • Call-to-action (CTA) buttons: varying common ones such as ‘Buy Now,’ ‘Sign Up,’ ‘Submit,’ ‘Get Started,’ or ‘Subscribe’ and varying sizes, colors and page placement
  • Logos of customers or third party sites: build credibility and convey trustworthiness (could include Better Business Bureau, TRUSTe or VeriSign logos as well as customer logos)

Bucket testing & Intelligence Cloud

Optimizely’s A/B testing software allows you to run bucket tests on all of your webpages and landing pages without relying on IT for support. You can set up tests within minutes, optimize all your pages and start making more informed decisions about your business.

 

Start testing with Web Experimentation today!