Socialize your optimization wins with multi-armed bandit reports
You asked, we answered! Optimizely’s Multi-Armed Bandit now offers results that easily quantify the impact of optimization to your business.
Brendan McSherry
Optimizely Multi-Armed Bandit uses machine learning optimization to maximize conversions for campaigns like conversion-focused landing pages, promotions, webinars and headline testing. In the past, we’ve captured an “improvement over equal allocation” metric on the Optimizely results page to demonstrate the advantages of running Multi-Armed Bandits optimizations over A/B tests with equally allocated traffic. What have we heard from customers? This is useful information contextually, but, not as effective for results sharing or ROI-related conversations.
Based on this feedback, we’ve iterated on the results page to include a new metric: “improvement over original variation”. This metric represents the absolute impact of leveraging our Multi-Armed Bandit compared to keeping existing code, or your control, deployed to all traffic.
We want to empower you to report out the impact of the experiences you’re delivering through Optimizely more effectively. A few months back we launched the ROI Model, which offers the ability to quantify the monetary value your experiments have generated at an individual experiment and program level. This type of analysis is now possible for Multi-Armed Bandit as well. With a manual input for $ value per conversion, this “improvement over original variation” metric makes it easy to compute revenue impact.
In a recent Multi-Armed Bandit we ran at Optimizely, for example — captured in the results page above — we saw an additional 45 lead conversions. If we assume that a lead is worth $250 to Optimizely, this would equate to an added $11,250 in revenue.
Our ROI Model and new Multi-Armed Bandit Results calculations are valuable tools for socializing the impact of your experimentation and optimization program. For access and guidance, please reach out to your Customer Success Manager or CustomerSuccess@optimizely.com.
What else can we do to help you demonstrate the impact of experimentation and optimization internally? Tweet us @Optimizely.
About the author
