Quiz: What Did You Learn?

Now that you’ve had a chance to learn more about this topic, let’s review what you should have learned with a short quiz. Select the best response to each question. When you've answered all the questions, select the submit button to check your answers.

If you need results from an experiment before it reaches statistical significance, you may choose to do so if:

  • The timeframe is limited for the experiment and you need the results sooner
  • You have a clear winner or loser early on
  • Your early results indicate that the experiment won’t reach statistical significance
  • The impact of incorrectly calling a winner is low

Which is NOT true about the Roadmap?

  • It can be used to coordinate resources for experiments
  • It can be used to calculate the timeframe for experiments
  • It can help ensure experiments don’t conflict with other planned campaigns
  • It can keep experiments prioritized

What information is irrelevant when determining how many visitors it will take to reach statistical significance?

  • Results from previous experiments
  • Baseline conversion rate
  • Minimum detectable effect (MDE)
  • Statistical significance threshold

If you don’t know your baseline conversion rate, how can you determine it?

  • Look at your year-over-year sales figures
  • Run an A/A experiment
  • Check a heatmap
  • Ask your manager

Which of these questions will your roadmap answer?

  • What are we experimenting on?
  • When are we experimenting?
  • Who is involved in experimentation?
  • All of the above