Quiz: How Much do you Know?

Before we start, let’s see what you already know with a short quiz. Select the best response to each question. When you've answered all the questions, select the submit button to check your answers.

What is statistical significance?

  • A way to have assurance that experiment results are entirely accurate and reliable
  • A method for mathematically demonstrating the reliability of a given metric
  • A process for calculating whether an experiment is valid as an integration on your website or app
  • The percentage of statistical uplift you can expect as the result of an experiment

Which of these statements is true?

  • An experiment can’t end unless statistical significance is reached
  • Your chosen level of statistical significance reflects how comfortable you are with risk.

Miranda has to run an experiment, but due to a tight deadline, she decides to set the significance to end it at 80% instead of the 95% statistical significance they usually aim for. What risk is she running in doing this?

  • She’s taking a 15% higher risk than usual that the experiment results aren’t accurate.
  • There’s only an 80% possibility that the information gathered so far reflects the activity of users of the site in the experiment period.
  • She has a 15% chance that her results won’t be as accurate as they usually would be.
  • At 80% significance, there’s really no significant risk that her results are inaccurate. She can be just as confident in her results as she usually is.