Improving Customer Experiences Through Experimentation

Conor Coughlan is Metromile’s enterprising Senior Marketing Manager. The Metromile team already had a robust channel experimentation program in place, and Coughlan wanted to take this approach to optimizing the website. The ad hoc testing his team was conducting at the time wasn’t achieving the results they needed, so he set out to clarify their vision and add new resources.

Initially, he reached out to Optimizely’s James Clifford, but together they determined that Metromile's team was already fully dedicated to many important projects, and didn't have the time required to run the impactful, strategic experimentation program Coughlan had in mind. That’s when Clifford introduced Coughlan to one of Optimizely’s top services partners, CROmetrics.

Coughlan and CROmetrics began work on a plan to launch and grow Metromile’s experimentation program. At first, the team experienced both failure and success while experimenting on its website redesign. But by analyzing insights gleaned from the initial experiments, the team was able to zero in on its most impactful experiments as well as potential new tests. This all led up to a pivotal moment in the progression of Metromile’s experimentation program — the Metromile Conversational Test Series.

Coughlan sensed that their website forms were daunting and outdated. He hypothesized that by simplifying the forms, Metromile’s conversion rates would increase. The team also believed that embedding the conversational questions directly on the homepage would improve conversions.

Using the Optimizely X Web experimentation platform, the Conversational Test Series was launched. Metromile and CROmetrics tested new forms on both the homepage and the insurance quote page. They broke down the long, intimidating forms into a succession of conversational, easy-to-answer questions. As the results started to flow in, they iterated on the tests and refined the forms as they learned what customers responded to most.

Just as hypothesized, the simpler forms and conversational tone worked. After streamlining the format on the insurance quote page, Metromile experienced a 12% increase in customer conversions. On a second test on the homepage form, the team saw an 8% increase in conversions by asking a few questions at a time instead of all at once.

But numbers don’t tell the entire story. By analyzing site visitor behavior, Metromile’s team was able to to develop a testing roadmap that influenced future results in positive, impactful ways.

Metromile’s inbound marketing now works harder than ever. Experimentation has allowed the company to immediately lower its customer acquisition costs. By driving a lower cost per acquisition, the company can allocate more resources to exploring new acquisition channels and testing bigger bets.

The best results, they’ve learned, come from conducting more experiments and iterating quickly based on the results, rather than attempting to perfect every experiment before it launches. That’s the kind of insight that can only come from trial and error.

“I think this paints a great story. An important part of our journey was learning from our negative tests, which helped us understand what things do and don't work.” — Conor Coughlan

The Metromile team is incredibly pleased with the improvements that were made possible by CROmetrics and the Optimizely X Web experimentation platform. The team now plans to take experimentation even deeper into their conversion funnel, test new hypotheses, and continue revealing new ways they can lower costs and improve customer satisfaction.