Before scrapping the experiment, Lin and his team got together and tried to assess what could have caused the experiment to fail. In analyzing their scroll depth tools, they learned that most people weren’t getting far below the fold on the page before abandoning the website. This made them wonder if the performance was different on their app, where most of the information was more concise and easier to access. When they segmented their results, they found that moving the search bar in the mobile version of the website actually reduced their dropoff by 12%.
So then, Lin wondered, were there distractors on the main website that could be eliminated? Should they move the vital information so it’s more visible? The losing experiment launched a great discussion for Lin’s team about what their visitors needed most and where it should be located, resulting in a new hypothesis and experiment that ultimately improved their website and resulted in a greater percentage of conversions overall.