Customer Story

DriveTime Uses Optimizely Full Stack to Develop Experiments and Implement Winners, Faster

The car buying and financing revolution

DriveTime, headquartered in Tempe, Arizona, is the largest privately owned used car sales and financing company in the US. With more than 120 dealerships, 3 operations centers, and 20 inspection centers., DriveTime is focused on getting the right customer in the right car, with the right terms.

The culture of experimentation is in our company’s DNA

DriveTime revolutionized the used car market. They are the largest privately owned used car sales and financing company in the US. DriveTime is integral in all steps of the sales process from advertising and selling to financing, and servicing car loans.

DriveTime delivers the best customer experience possible whether in person at their dealerships or online. They do it with trusted data.

“We have visibility into the entire funnel, which allows us to develop a holistic picture of the end user,” says Kyle Tassinari, DriveTime’s Director of Market Analytics. “That also lets us leverage data and optimization tools to maximize profitability of the consumers that are on our website, with levers and filters through every step of the customer journey. ”

One key way that they get actionable insights is by A/B testing. Those objective inputs shape their business strategy and mitigate risks.“We've proven that you don't have to make an operations decision without knowing how that's going to impact your customer,” says Joanna Golina, a DriveTime Director of IT. “The culture of experimentation is in our company’s DNA. We experiment, we pilot, we run tests across everything that we do, and that gives us a leg up.”

They needed an easier way to test, learn, and win

For years, DriveTime used an experimentation platform that limited their access to quick and trusted insights. “It didn’t provide the rich results analysis we needed,” says Nate Warner, a DriveTime Director of Application Development.

Speed to implement was another critical factor. “We wanted a platform that would allow us to experiment quickly, get our test out, figure out if it worked, and move on to the next one,” says Golina. DriveTime’s increasingly complex experiments were taking too long; Golina points to one example—an experiment with 13 variations—that took four weeks to configure.

Finally, the team no longer wanted to rely on developers to implement every experiment. “Where do we go from here? How do we get something a little more feature rich? We understood what we wanted to get out of our platform, and Google Experiments wasn’t giving it to us,” Warner reflects.

“We had to build our own solutions, and had to do the calculations ourselves. The reports would take a while to pull together—we needed something better.”

Nate Warner
Director of Application Development

They run more experiments, faster, with Optimizely

DriveTime evaluated several leading experimentation solutions. There was a clear winner. In Warner’s words “It was obvious: [Optimizely] is a great solution, this works for us.”

In a matter of weeks, DriveTime ran its first experiment using Optimizely Web. Golina attests, “It was a very short amount of time to get a brand new platform up and running and something that's adding so much value.”

Their early homepage experiments optimized DriveTime car search and the loan approval process. At the time, the homepage featured so much information that it was negatively impacting customer acquisition and conversion.

“We thought we were doing the right thing by telling customers about all the products they could buy with their car, how long DriveTime has been around, our philosophy as a company,” says Golina. “One of our early experiments was to remove all of that information from the mobile view. You couldn’t even scroll on the homepage, it was just: here's three things you can do. There was a lot of apprehension around that. The entire team was like, this can't win.”

To the team’s surprise, the new variation of the homepage drove a 20% jump in CTA click conversions. The shorter home page simplified the user flow through the entire DriveTime site. Tassinari explains that “By using the data from the Optimizely experiments in conjunction with internal analytics tools, we were able to follow customers from conversion all the way through the sale.That’s what ultimately got us comfortable to deploy the shorter homepage to 100% traffic—being able to measure [the winning variation] all the way through.”

A few months later, DriveTime noticed that high-tier inventory was sitting on their car lots for twice as long as the average vehicle. Through a series of experiments, DriveTime identified how many of these vehicles they needed to showcase on the website to optimize in-store sales.

Tassinari shares, “The big lesson for us was, yes, we need to be in tune with operations, and take into account the requirements of their supply chains to run end-user tests and let the data help us decide what we do.”

The inventory test was one example of the increasingly complex experiments the team was implementing with Optimizely Web. “It took a lot of effort to get that up and running,” says DriveTime developer, Maddie Reichman. “We asked ourselves: how complex are we planning on getting with these experiments? And the answer was: this is just the beginning.” They needed a better way for engineering to test deep in their products and tech stack.

Testing deeper with Optimizely Full Stack

The growing development team at DriveTime, needed a flexible, code-based solution to run powerful experiments in their digital products. With Optimizely Full Stack they can experiment in any application, anywhere in the customer journey.

“Our developers were building out very complex experiments using Optimizely Web, using it in a very Full Stack way, which was taking us a lot longer than it needed to,” says Warner. “Full Stack was a clear way for us to move forward—speed of implementation was the motivator.”

The DriveTime product development team also appreciated the power and flexibility of Full Stack’s feature variables, which allow them to dynamically update variables in the application outside of a release cycle. “Full Stack variables bring endless opportunities, being able to configure things without having to do a full new round of an implementation,” says Reichman.

Optimizely Full Stack and Web use the same Stats Engine, giving the team the rich and efficient analysis they were already using. “Now we can set up complex experiments relatively quickly, but we get fast results that we can trust,” says Warner.

“What's so awesome about Full Stack is that you can iterate off of the feature that you’ve already set up, rather than having to re-evaluate and start over and get your hands all dirty again.”

Maddie Reichman
Application Developer II at DriveTime

Adopting Full Stack

DriveTime implemented Full Stack and started running their first A/A tests to validate their setup and used the process as an opportunity to educate stakeholders. “We involved the whole team in the implementation process,” says Reichman. “We rounded up all our IT teams and explained and evangelized the differences between Web and Full Stack, and said, here are the cool things that we're doing with Optimizely in our customer-facing sites—start thinking about how you can be using it on your apps.”

“We got everyone really pumped about it.”

Reichman points to Optimizely’s support for controlled and targeted feature rollouts as a major Full Stack benefit. “It was exciting to see it working, and now we know we can test all of our experiments as we're developing them, to make sure that they are rollout friendly.”

Scaling experiments, faster

Although DriveTime has only been using Full Stack for a few months, the platform is living up to expectations. “We’re able to implement our winners quickly using Full Stack, because we're creating experiments in the best possible dev way,” Reichman says, attributing the speed to Optimizely’s feature rollouts, as well as clean, efficient code-side implementation.

Of course, COVID-19 hit the US in early 2020 and threw businesses everywhere into a tailspin. DriveTime took quick action to let potential customers know how its business would operate during the pandemic. DriveTime set up a banner to show to specific customers.

“Because it's a feature flag, we have the ability to turn it off whenever we feel comfortable doing so,” says Reichman. “Another thing that we are absolutely loving on Full Stack is attributes for audiences. We created a new audience for the COVID-19 info banner. We only wanted the bucket of users that came in on the homepage; now we can make 30 different audiences based off of whatever your landing page was. We love that flexibility.”

What’s next

As DriveTime emerges from the effects of the COVID-19 shutdown, the company will refocus on sales, and the development team looks forward to fully exploring the possibilities of Full Stack. “Our next step is what we're calling vehicle impressions and understanding what customers are seeing and the “why” behind what they are actually doing,” says Warner.

“We’ll definitely be looking more at inventory,” Golina adds. “We need to figure out okay, which makes and models are the ones driving the incremental web conversions and making sure people are still coming into the store.”

Full Stack, opens multiple opportunities to work more closely with the DriveTime operations team. “On the business strategy side, we've proven that you don't have to make an operations decision without knowing how it’s going to impact your customers, even though you seem like you're pretty far away from the end user,” Golina says. “There's still so much that we want to do.”

Find out how teams are using Optimizely to deliver new experiences quickly with more confidence and higher quality, and sign up for Optimizely Rollouts to get started today with free feature flags and A/B tests!