Quiz: How Much do You Know?

Before we start, let’s see what you already know with a short quiz. Select the best response to each question. When you've answered all the questions, select the submit button to check your answers.

What is one thing a losing variation teaches before you even analyze your results further?

  • That your variation was possibly set up improperly
  • What changes visitors liked
  • That you changed something your visitors care about

How does a losing experiment help you in the long run?

  1. It saves you money by keeping you from investing in revisions that will fail to increase lift
  2. It allows you to keep customers who may have navigated away from your site or app if you'd implemented the changes
  3. It gives you the opportunity to pursue new ideas
  4. It can be used, in combination with your winning experiments, to create a complete profile of visitors’ preferences
  • 1, 3, 4
  • 1, 2, 3
  • 3, 4

How is the hypothesis most useful in a losing variation?

  • You can return to the hypothesis and analyze each part to see where your assumptions may have been incorrect.
  • You have a clearly documented case study that you can use to make sure you do not create a similar experiment in the future.
  • You now know that something that you thought was a problem is definitely not really a problem for your visitors.