Challenge: Call it a Day?

In this challenge, you'll read through a couple scenarios and be asked to analyze the facts you're given and make decisions about how to handle each situation. Once you've aswered all the questions for all the scenarios, select "Submit" at the bottom of the page to check your answers.

For their first experiment, Majors and Brooks, a high-end makeup retailer, decided to run a more conservative experiment to test out the optimization process. They changed the color and size of all their “add to cart” buttons to make them more visible to people looking to buy.

After two weeks, Cyrus, the head of the optimization team, noticed that there had been enough traffic to the experiment to reach statistical significance, but the results were inconclusive.

What can Cyrus infer right away from these results?

  • No one cared about the change
  • The change was confusing
  • The change kept people from converting
  • No one liked the change

Based on the results, should Cyrus continue the experiment to get a conclusive response?

  • Yes, he may find that interest will change
  • Yes, it’s possible that people will get used to the change and start converting
  • No, it’s obvious that the experiment isn’t going to work and it should be abandoned
  • No, more changes should probably be made to the experiment to get better results

Asha is a part of an optimization team for a timeshare website that is offering spa vacation packages at a steep discount to entice visitors to come try their properties and potentially buy into the timeshare. She ran an ad on Google advertising the spa package and ran an experiment on the main page of her website, where the ad directs visitors, to see if the ad enticed more conversions for the vacation, thinking it would be an easy win--of course more people are going to buy the vacation after seeing the ad! However, she’s nearly hit statistical significance and the results are still inconclusive! Visitors seem to be hitting the homepage and dropping off before progressing to the page about the spa vacation.

What could be the primary problem with the way she set up the experiment?

  • The spa package wasn’t a good deal
  • The location of the ad wasn’t enticing to potential customers
  • The ad directed to the homepage rather than to a page created for the vacation
  • The ad didn’t give enough details about the low price and benefits of the spa vacation

How could Asha fix her primary issue?

  • Direct visitors directly to the spa vacation page rather than to the homepage
  • Attempt to run an ad through a different vendor and see if the response is better
  • Change the content of the ad to highlight the value and quality of the vacation
  • Consider changing the parameters of the spa vacation to make it a more enticing deal for visitors