How do I measure lift?

The difference interval shows you a range of values where your conversion rate is likely to lie, depending on the statistical significance threshold. It helps you interpret the trend of your data. This video walks you through an example of how to read the difference interval.

 

When you notice one variation has a better conversion rate than another, you might be tempted to immediately implement that variation. But if you do so, can you expect this variation will actually lead to the better conversion rate once implemented?

First, you should take a look at Improvement, which is the relative difference between the conversion rate in your original and your variation. In other words, if your original converted 5% of the time, and your variation converted 10% of the time, then Optimizely would calculate a 100% improvement. This improvement is also commonly referred to as "Lift."

Then, you can look at the Difference Interval to determine what this improvement is likely to range between if you actually implement the variation. For example, in the above example where your conversion rate was 10%, you may see an interval that covers 9% to 11%, which means that if you implemented this variation, the actual conversion rate would lie between 9% and 11%.

In the image above, you can expect this variation to beat the original if you implement it on your site, because the difference interval is entirely "in the green."

Here, however, you can expect this variation to underperform the original if you implement it on your site, because the difference interval is entirely "in the red."