In this challenge, you'll read through a couple scenarios and be asked to analyze the facts you're given and make decisions about how to handle each situation. Once you've aswered all the questions for all the scenarios, select "Submit" at the bottom of the page to check your answers.
For their first experiment, Majors and Brooks, a high-end makeup retailer, decided to run a more conservative experiment to test out the optimization process. They changed the color and size of all their “add to cart” buttons to make them more visible to people looking to buy.
After two weeks, Cyrus, the head of the optimization team, noticed that there had been enough traffic to the experiment to reach statistical significance, but the results were inconclusive.
What can Cyrus infer right away from these results?
Based on the results, should Cyrus continue the experiment to get a conclusive response?
Asha is a part of an optimization team for a timeshare website that is offering spa vacation packages at a steep discount to entice visitors to come try their properties and potentially buy into the timeshare. She ran an ad on Google advertising the spa package and ran an experiment on the main page of her website, where the ad directs visitors, to see if the ad enticed more conversions for the vacation, thinking it would be an easy win--of course more people are going to buy the vacation after seeing the ad! However, she’s nearly hit statistical significance and the results are still inconclusive! Visitors seem to be hitting the homepage and dropping off before progressing to the page about the spa vacation.
What could be the primary problem with the way she set up the experiment?
How could Asha fix her primary issue?