Electronic Arts Inc. (EA) is a leading global interactive entertainment software company that develops and distributes interactive software worldwide. Maxis is a division of EA and the creator of The Sims and SimCity video game franchise.
In March 2013, EA was preparing to release its latest, highly-anticipated version of SimCity. As one of EA’s most popular games – the new SimCity launch was expected to gain a significant attention. Maxis, the division of EA in charge of SimCity, wanted to maximize online revenue from the launch.
The team opted to extend a promotional offer in order to drive more online pre-orders. The offer was displayed as a banner across the top of the pre-order page and as part of the ad creative in across paid media campaigns. This prominent placement put the promotion in front of many potential buyers, but was not driving the increase in SimCity pre-orders they had expected.
Optimizely has helped us learn a lot about our users – what’s working and what’s not. In the end, this translates into higher revenue for us.— Mike Burk, Senior Online Product Manager, Maxis
While many site visitors did see the promotional banner, its large size moved the primary call-to-action button further down on the page. The team hypothesized that moving the primary CTA higher on the page (and thus, changing the way the promotional offer was displayed) could drive more total game purchases.
The team set up an A/B test pitting the original SimCity pre-order page against several variations that displayed the promotional offer in different ways. Each variation corresponded with a hypothesis the team had set beforehand regarding placement, color, and display of the promotional offer. One variation removed the promotional offer from the page altogether.
After building each variation in Optimizely’s visual editor, the team set their experiment goal — clicks on the “Pre-Order” button — to track purchases generated from the test. They allocated 100% of site traffic to the experiment in order to find an optimized solution as quickly as possible during the high-traffic SimCity launch period.
The team quickly discovered very surprising results – the variation with no offer messaging whatsoever drove 43.4% more purchases.
We found that the variations with the promotional offer actually performed worse than just a clean version of the page. This was really surprising to us, but it had a pretty significant impact.— Mike Burk, Senior Online Product Manager, Maxis
Armed with data-driven knowledge and significant results from their initial test, the team decided to go even bigger. A follow-up test on EA’s paid media creatives that mirrored the initial test — removing promotional text from each ad) also increased conversions from highly-qualified ad traffic.
A/B testing provides real, actionable data to help you better understand how your web properties are performing. Even if think you’ve optimized your site, test it out. In EA’s case, the common belief was that promotional offers will drive more purchases — this turned out to be false. Testing gave them the information needed to maximize revenue in a way that would not have been otherwise possible.
Testing arms you with information that can increase conversions beyond just your website. EA applied the lessons from the experiment on the SimCity pre-order page to their paid advertising campaigns, yielding exponential results.