Bucket testing (sometimes referred to as A/B testing or split testing) is a term used to describe the method testing two versions of a website against one another to see which one performs better on specified key metrics (such as clicks, downloads or purchases).
There are at least two variations in each test, a Variation A and a Variation B. Metrics from each page variation are measured and visitors are randomly placed into respective ‘buckets’ where the data can be recorded and analyzed to determine which performs best.
Companies that market and sell products or services online rely on bucket testing to help them maximize revenue by optimizing their websites and landing pages for conversions.
Let’s look at a hypothetical example. Each bucket test begins with a hypothesis that a certain variation on a landing page will perform better than the control. Say you have an existing landing page for a free nutrition eBook, Eat Raw Foods and Live Longer.
The button on the bottom of your landing page’s sign-up form says ‘Submit,’ but your hypothesis is that changing the text to ‘Get Your Free Copy’ will result in more form conversions. The existing page with the ‘Submit’ button is the control, or Variation A. The page with ‘Get Your Free Copy’ on the button is Variation B. The key metric you will measure is the percentage of visitors who fill out the form, or a form completion.
Because you have an ad campaign driving several thousand visitors a day to your landing page, it only takes a few days to get the results from your bucket test. It turns out that ‘Get Your Free Copy’ has a significantly higher click rate than ‘Submit,’ but the form completion rate is basically the same. Since the form completion rate is the key metric, you decide to try something different.
Bucket testing plays a big role in conversion rate optimization. Running a bucket test allows you to test any hypothesis that can improve a page’s conversions. You can continue to try higher-converting button text for Eat Raw Foods and Live Longer or you can go on to test other hypotheses, such as bolder headline copy, more colorful imagery or arrows pointing to the sign-up button that will get more people to convert.
Companies spend millions of dollars to drive traffic to landing pages and websites that promote their product or service. With simple variations to page copy, imagery, and layouts, you can conduct a series of bucket tests to gather data and to iterate towards your highest-performing version of the page. You simply create variations of the page, changing one element at a time and measuring key metrics, then collect the results until reaching statistically significant results for each experiment.
Bucket testing can make a significant impact on conversions per page, resulting in revenue increases on your highest-trafficked pages.
Bucket testing can also help to eliminate subjective opinions as deciding factors in a page’s design or layout. The author of Eat Raw Foods and Live Longer may think that her photo will drive more customer demand – or she may insist on a rainbow palette of colors.
With bucket testing, there is no need for debate on what design or page elements will work best to convert a customer. The quantitative data will speak for itself, and drive the decision for you.
Tests should be prioritized to run on your most highly trafficked pages, since you may need hundreds or thousands of visitors to each variation to gather statistically significant data. The more traffic a page receives, the quicker you will be able to declare a winner.
Optimizely’s A/B testing software allows you to run bucket tests on all of your webpages and landing pages without relying on IT for support. You can set up tests within minutes, optimize all your pages and start making more informed decisions about your business.
Start testing with Optimizely today!
Be inspired by 40+ experiment ideas that have generated millions in revenue.
Learn the benefits of experimenting at scale from this original research report from the Harvard Business Review
This assessment is the starting point to understanding your organization’s capabilities and will set you on the path to building a high-performing program.
An error has occurred
You can get the very best of Optimizely without spending a dime. Try it out for 30 days, on us.
Almost ready! Processing form...
Please correct form errors
Sign up to receive resources, product updates and other communications from Optimizely.
Get a free account with full access to Optimizely's APIs and SDKs.
Start releasing products smarter with feature flags and rollouts. Sign up today.