What is A/B Testing?
Optimizely: A/B Testing Simplified
A/B testing is a simple way to test changes to your page against the current design and determine which ones produce positive results. It is a method to validate that any new design or change to an element on your webpage is improving your conversion rate before you make that change to your site code.
Testing takes the guesswork out of website optimization and enables data-backed decisions that shift business conversations from “we think” to “we know.” By measuring the impact that changes have on your metrics such as sign-ups, downloads, purchases, or whatever else your goals may be, you can ensure that every change produces positive results. Quantitative data speaks for itself. You and your coworkers may have hunches about how site visitors will respond to certain design elements; A/B testing allows you to show visitors two versions of the same page and let them determine the winner. Constantly testing and optimizing your page can increase revenue, donations, leads, registrations, downloads, and user generated content, while providing teams with valuable insight about their visitors.
Here's how it works:
An A/B test involves testing two versions of a web page — an A version (the control) and a B version (the variation) — with live traffic and measuring the effect each version has on your conversion rate. Start an A/B test by identifying a goal for your company then determine which pages on your site contribute to the successful completion of that goal.
Imagine a company, Acme Widgets, that operates a web store selling widgets. The company’s ultimate goal is to sell more widgets and increase their yearly revenue, thus the checkout funnel is the first place Acme’s head of marketing will focus the optimization efforts.
The “buy” button on each product page is the first element visitors interact with at the start of the checkout process. The team hypothesizes that making the button more prominent on the page would lead to more clicks and therefore more purchases. Using Optimizely, the team simply makes the button red in variation 1 and leaves the button grey in the original. They quickly set up an AB test using an AB testing tool that pits the two variations against each other.
As the test runs, all visitors to the Acme Widgets site are bucketed into a variation. They are equally divided between the red button page and the original page. Optimizely measures the number of visitors who saw each version of the button and then clicked it. It also measures the number of visitors who completed the purchase funnel and landed on the “Thank You” confirmation page.
Once enough visitors have run through the test and Optimizely indicates that the results are statistically significant, the Acme team ends the test and is able to declare a winner. The results show that 4.5% of visitors clicked on the red buy button and 1% clicked on the original version. The red “Buy” button led to a significant uplift in conversion rate, so Acme then redesigns their product pages accordingly. In subsequent A/B tests, Acme will apply the insight that red buttons convert better on their site than grey buttons.
What assumptions are you making about your site right now? Load your site into the Optimizely editor, click “Test it Out” and see how simple it is to set up an AB test or multivariate test with Optimizely. Your improved site is the only outcome.