Challenge: Losing Variations
Lin leads an optimization team for TravelAsia, a website and app that helps visitors create and coordinate travel across multiple Asian countries in one trip. Both the website and app currently feature a search function at the top of the homepage. However, Lin’s team found that most of the information people searched for was accessible in some way from the main page of the website. This information was clearly visible as long as people actually looked, and by accessing this information directly from the main page, it’d take fewer clicks to get to what visitors needed. Because of this, they considered the search bar a distraction that didn’t encourage people to look at the whole page to get what they needed. In order to encourage visitors to review more of the page to come across what they needed organically, Lin’s team created an experiment that moved search bar from the top of the page to the bottom to encourage people to scroll entirely down the page to find the information they need before trying to search for it.
After a few weeks, the experiment reached statistical significance and Lin checked the results. The experiment was a huge loss. In the experiment group, they had a nearly 50% increase in dropoffs compared to the control group.
Before reading how Lin and his team handled the loss, take a minute to think about how you would handle the situation.
What should Lin and his group do next?
What analysis tool would most effectively help Lin’s team understand what caused the failure?
What else could Lin’s team do to learn more about their results?