Leading global electronics company Sony uses optimization to increase conversions across its web platforms, and they use the testing cycle to help keep their experimentation process constantly moving forward. For Evelien Greens and her team at Sony, the experimentation cycle is integrated into every aspect of their optimization process. Evelien, a web merchandiser for the company, starts with ideation. She recently audited their website analytics and found that banner ads promoting a customizable Sony Vaio laptop were underperforming. She built a hypothesis based on her data that the dual Calls to Action (CTA) on the ad were diluting the message, confusing and overwhelming visitors.
To test her hypothesis, Evelien planned and developed an A/B/C test to compare two versions of the banner against the original. One banner focused on the customizable laptop, and the other focused on the sitewide offer. She hypothesized that a part of the issue was the complication of offering a custom laptop, which many customers seemed to believe would take a lot of time and not be worth the time investment, Evelien decided to target her experiment to her two largest custom markets--the UK and the Netherlands. She designed the experiment to follow visitors from the clickthrough on the ad through the checkout funnel, setting a primary click goal on the banner itself and a secondary pageview goal measuring when site visitors reached shopping cart landing pages after clicking through the banners.
When the experiment finished, she found that the variation focusing only on customization resulted in a more than 21% uplift of customers putting the product in their cart, with the variation only focusing on the sale actually performing worse than the original version. After the results came in, she analyzed them using segmentation to compare both markets, the Netherlands and the UK, individually, and comparing performance in various device types. Her analysis launched her back into the ideation and planning phase, where Evelien and her team began to use what they’d learned from the first experiment to iterate into planning and prioritizing future experiments.
By integrating the testing cycle into their experimentation, Sony is able to make the most out of every experiment, learning and expanding their process with each result. Each experiment results in new ideas and opportunities for Sony’s optimization team.