What is A/B Testing?

Optimizely: A/B Testing Simplified

A/B testing is a simple way to test changes to your page against the current design and determine which ones produce positive results. It is a method to validate that any new design or change to an element on your webpage is improving your conversion rate before you make that change to your site code.

Testing takes the guesswork out of website optimization and enables data-backed decisions that shift business conversations from “we think” to “we know.” By measuring the impact that changes have on your metrics such as sign-ups, downloads, purchases, or whatever else your goals may be, you can ensure that every change produces positive results. Quantitative data speaks for itself. You and your coworkers may have hunches about how site visitors will respond to certain design elements; A/B testing allows you to show visitors two versions of the same page and let them determine the winner. Constantly testing and optimizing your page can increase revenue, donations, leads, registrations, downloads, and user generated content, while providing teams with valuable insight about their visitors.

Here's how it works:

How it works

An A/B test involves testing two versions of a web page — an A version (the control) and a B version (the variation) — with live traffic and measuring the effect each version has on your conversion rate. Start an A/B test by identifying a goal for your company then determine which pages on your site contribute to the successful completion of that goal.

Imagine a company, Acme Widgets, that operates a web store selling widgets. The company’s ultimate goal is to sell more widgets and increase their yearly revenue, thus the checkout funnel is the first place Acme’s head of marketing will focus the optimization efforts.

The “buy” button on each product page is the first element visitors interact with at the start of the checkout process. The team hypothesizes that making the button more prominent on the page would lead to more clicks and therefore more purchases. Using Optimizely, the team simply makes the button red in variation 1 and leaves the button grey in the original. They quickly set up an AB test using an AB testing tool that pits the two variations against each other.

With Optimizely

As the test runs, all visitors to the Acme Widgets site are bucketed into a variation. They are equally divided between the red button page and the original page. Optimizely measures the number of visitors who saw each version of the button and then clicked it. It also measures the number of visitors who completed the purchase funnel and landed on the “Thank You” confirmation page.

Once enough visitors have run through the test and Optimizely indicates that the results are statistically significant, the Acme team ends the test and is able to declare a winner. The results show that 4.5% of visitors clicked on the red buy button and 1% clicked on the original version. The red “Buy” button led to a significant uplift in conversion rate, so Acme then redesigns their product pages accordingly. In subsequent A/B tests, Acme will apply the insight that red buttons convert better on their site than grey buttons.

What assumptions are you making about your site right now? Load your site into the Optimizely editor, click “Test it Out” and see how simple it is to set up an AB test or multivariate test with Optimizely. Your improved site is the only outcome.

Start using Optimizely to test your website today! View Plans & Pricing

Something went wrong.

If this error persists, please let us know at our support page.

Our engineering team has been notified.

Browser Compatibility

You are currently using a browser that is not supported by Optimizely's editor. Please try using one of the following instead:

  • Chrome 20+
  • Firefox 4+
  • Internet Explorer 10+
  • Safari 5+

Create a free accountCreate a free Optimizely account

You're seconds away from running A/B tests!By creating an account you can post in Optiverse and test drive Optimizely

Already have an Optimizely account?

Create Experiment

Your new experiment will appear under "DRAFTS" until you start it. Then it will appear under "ACTIVE".

Enter a URL below to get started on a new experiment!

Create iOS Experiment

Your new experiment will appear under "DRAFTS" until you start it. Then it will appear under "ACTIVE".

Forgot Password

We'll send you an email with a link to reset your password.

Implementing Optimizely

Copy this code and paste it immediately after your opening <head> tag:

Because you are using the Shopify App, the Optimizely embed code is already added to your shop automatically!

If you want to run Optimizely on a page NOT hosted by Shopify, copy & paste this code snippet to the top of the <head> tag:

Copy to Clipboard
Send to Developer

Include this snippet on every page you want to run experiments on and track as a goal. This snippet will not change. Note the //cdn prefix is NOT a typo.

To add new projects, go to the Dashboard.

Each project has a unique Project Code snippet that must be included on every page running experiments or being tracked as a goal.

Your account has more than one project, so be sure to add the correct snippet. To view your projects and project codes, go to the Dashboard.

Click on Project Code to get your snippet:

For advanced implementations, see our API Documentation.

Send Code to Developer

Does someone else on your team implement the coding of your site? Put their email address below to send them instructions on how to implement Optimizely on your site.

Sign in to Optimizely