Our hypothesis was that if we bundled products together, it would be more convenient for users. We hypothesized that the more products we bundled together, the more our users could try our products and adopt [them]... What we found was that if we bundled more than two products at a time, in fact, if we bundled three products together, it actually confused users and made it harder for them to be engaged because they were spread so thin between three products. They couldn’t dive into one or two and really engage with the product and be activated. So, we started dialing that back. [Now] we don’t bundle more than two products at a time. We want to keep it simple for the user.”
Tom TsaoGlobal Head of Ecommerce, Atlassian
The Big Book of Experimentation
Discover 33 Proven Ideas: How Real Businesses Mastered Experimentation to Enhance Customer Engagement, Personalize Digital Channels, and Boost Customer Satisfaction.
In this comprehensive collection, explore how successful businesses harnessed the power of experimentation to gain invaluable insights and exert greater control over their customer interactions. Uncover strategies they employed to tailor their digital channels effectively, creating personalized experiences that left customers more satisfied than ever before.
Atlassian leverages experimentation to optimize its customer experience
Atlassian’s suite of collaboration software includes Jira, Confluence, Stride and Trello. Optimizely helped Atlassian experiment across all aspects of the buyer journey to optimize the digital experiences of all their customers. Experimentation and digital experience optimization are critical to the company’s success and are incorporated into the following areas:
- Product discovery
- Pricing and packaging
- Onboarding
- Cross-selling and upselling
Experiment:
Moving from bundling
three products to two
Key results:
Higher retention rates
Mailchimp expands revenue with successful experiments
Mailchimp, the marketing automation platform and email marketing service, embraced experimentation to boost their bottom line. By utilizing Optimizely, their growth squad conducted an experiment that introduced an upgrade option in the sign-up flow. This simple addition resulted in millions of dollars in revenue for the company.
According to Lauren Schuman, Senior Director of Product Insights, the experiment validated the idea that users are willing to pay when asked. The success of this first experiment led the growth team and senior leadership to fully embrace the power of experimentation as a crucial strategy moving forward.
Experimentation removes the bias from your decisions.
Lauren SchumanSr. Director Product Insights, mailchimp
Experiment:
Making it easier to buy paid plans
Key results:
Millions of dollars in incremental revenue
Discovery reveals how to optimize digital ad viewability and CTR
As a global mass media and entertainment company, Discovery Communications reaches an impressive two in five people worldwide through its diverse portfolio of television networks. To deepen their engagement with fans, the Digital Media product team turned to Optimizely for solutions.
With limited resources to work with—only images and headlines—the team devised the 'Ken Burns Test,' inspired by the renowned documentary filmmaker's panning effect on photos and still images. The team quickly set up an experiment, pitting the panning effect against the original video stills, and tested it on around 20,000 web visitors. The result was remarkable: a 6 percent increase in click-throughs to the video content.
Now, this effective technique has been integrated across the site, leading to higher overall engagement and significantly impacting Discovery Communication's business.
Our product team can fail and make mistakes that are inexpensive versus very expensive mistakes that end up using lots of resources. Everything from QA to deploying code and so on. We can prevent developers from making changes that won’t ultimately matter… that’s really the kind of culture that we’re building here.
Jeffrey DouglasDirector of Product , Discovery
Experiment :
Optimize digital ad viewability
Key results:
6% increase in clickthroughs for Discovery’s video content
Discover the full ebook
Explore all 33 inspiring ideas on how Optimizely customers leverage experimentation to generate valuable insights and consistently optimize experiences across the board.