Faster decisions you can trust for digital experimentation

At the core of Optimizely’s experimentation platform is our Stats Engine: a unique approach that was developed in conjunction with leaders in statistical research at Stanford University. Stats Engine embeds several innovations to deliver what every digital experience optimization effort needs—accelerated speed and greater scale of experimentation while also improving the accuracy of results and outcomes.

With Stats Engine

setting a 95% significance level results in a 95% chance you'll make the right decision

With traditional statistics

setting a 95% significance level results in a <70% chance you'll make the right decision if you don't avoid pitfalls*

* We simulated millions of A/A tests under both frameworks and calculated error rate by determining the percentage of experiments that declared a winning or losing variation given a continuous monitoring policy. (1000+ visitors)

Data has changed, statistics should too

Traditional statistics were developed over 100 years ago, for an offline world. These methods are no longer up to the task of helping businesses make decisions with real-time data. If you look at results in real time, as most optimization platforms allow you to do, you'll see winning results where no winner actually exists. In fact, your error rate can skyrocket to over 30% if you don't take precautions.

Without taking precautions, like setting a sample size and committing to a minimum detectable effect in advance, you could make changes to your website or product that at best, have no impact, and at worst, could be hurting your bottom line.

Download the whitepaper

Introducing always-valid results

With traditional statistics, peeking at your results any time other than your set sample size increases the chance you’ll find a winner when there isn’t one. With Stats Engine, check whenever you want. With a method called sequential testing, you’ll see your statistical significance increase with more time and data—no more waiting.