Has the outcome of an experiment ever gone so strongly against your intuition that you doubt its correct implementation? This is certainly a possibility, as correctly implementing an online experiment is fraught with data quality challenges. There is a considerable amount of engineering and processing of data before it reaches the analysis, leaving plenty of room for bugs and biases to creep in. One of the strongest signals of an incorrect implementation is a sample ratio mismatch (SRM) - when the number of users assigned to each variation differs significantly from what is expected under the intended random allocation.
This talk will:
Sorry, there was an error. Please try again.
An error has occurred
You can get the very best of Optimizely without spending a dime.Try it out for 30 days, on us.
Hang tight! We're creating your account and password instructions are headed to your inbox.
Please correct form errors
Start releasing products smarter with feature flags and rollouts. Prove value with A/B testing. Built on our Full Stack platform.
Welcome, we're creating your account...