Quiz: What Did You Learn?

Now that you’ve had a chance to learn more about this topic, let’s review what you should have learned with a short quiz. Select the best response to each question. When you've answered all the questions, select the submit button to check your answers.

What questions can you ask yourself as you analyze your problem statement?
1- Was this the right problem?
2- Should I have not been so bold?
3- Did the solution address the problem adequately?

  • 1, 2, 3
  • 1, 3
  • 2, 3
  • 1 only

Why is it important to experiment for learning instead of exclusively for lift?

  • You are less disappointed by a losing variation if you’re not trying to make money off every single experiment.
  • If you’re experimenting for lift only, you’ll be more inclined to abandon losing variations rather than analyzing them to discover what you can learn.

Before calling an experiment a complete loss, what should you make sure you do?

  • Create a new hypothesis that will test the exact opposite of your losing experiment to ensure a guaranteed winner.
  • Segment the results to see if there are winners in any segments that can teach you more (or that you can implement!).
  • Review your results to see if all your metrics reached statistical significance. Maybe by running the experiment longer, you can find a metric that comes back as a winner.