Download the full Evolution of Experimentation research report (60 pages) and start learning from 127,000 experiments today!
The standard experiment run around the world is an incremental A versus B test. While these tests are easy to run, they are rarely associated with performance breakthroughs. Our data shows that the largest breakthroughs come from the test that follows a very different model. Tests designed to test complex, interdependent changes- but within a single variant and across multiple variants- are more likely to be among the top 5% performing experiments in our sample. Rather than shying away from complexity, firms can potentially harness it to deliver high performance in testing. The key is to pair complex tests with a theory for how the multiple elements work together to deliver returns. Theory and testing together can help unlock breakthrough performance.Dr. Sourobh GhoshEconomist at Amazon/Audible