These A/B Tests Increased Revenue Per Visitor for VeggieTales by 38%
VeggieTales wanted to systematically optimize the design of their entire site. They worked with BlueAcorn, an optimization agency that focuses mainly on e-commerce businesses, to run tests on their homepage, checkout page, product page, shopping cart, and mobile website. These tests produced sitewide increases in revenue per visitor and conversion rate. Read more to see exactly what they tested.
VeggieTales is an animated children’s TV series, hugely popular with pre-schoolers in the United States. Since 1993, they have sold over 70 million videos, 15 million books, 7 million CDs, and all sorts of accessories touting the iconic vegetable characters.
They wanted to systematically optimize the design of their entire e-commerce site, store.veggietales.com, so they got in touch with my optimization strategy team at Blue Acorn for direction and results.
Our initial research drove us to start with some subtle design changes that, after reaching statistical significance through A/B testing, now receive 100% of the traffic as permanent design changes. All together, these subtle changes led to significant changes in revenue and growth:
- 38% increase year-over-year in sitewide revenue per visitor
- 42% increase year-over-year in sitewide conversion rate
By first examining their analytics, we were able to determine where to start testing and what to test.
At a glance, here are the results we produced for the VeggieTales store:
- 17.4% increase in revenue per category page visitor
- 13.9% increase in revenue per product page visitor
- 14.3% increase in revenue per checkout page visitor
- 36.8% increase in revenue per homepage visitor with social media changes
- 28.1% increase in revenue per mobile visitor
In this video, Kevin Eichelberger, Founder of Blue Acorn and John Pfaender, Director of eCommerce at VeggieTales, speak about their approach to optimization.
Let’s look at the hypotheses and variations that produced these results.
Primary conversion metric: RPV
We feel the truest metric for measuring the success of optimization testing for an e-commerce site is Revenue Per Visitor (RPV). Increasing our clients’ conversion rates is easy if they agree to start to sell everything at 90% off, but that would kill their revenue. Constantly attempting to improving your conversion rate is still a smart idea. However, e-commerce merchants should focus on RPV their primary Key Performance Indicator (KPI).
VeggieTales’ category pages had featured very large, bright, distracting banners. Customers landing on the page didn’t have general guidance on where to go, and they were getting distracted from finding products they wanted and continuing to the product page.
Hypothesis: Dropping the banners would result in a higher RPV.
Results: Without the banner, VeggieTales’ category pages produced a 17.4% increase in RPV.
Statistical Confidence Level: 95.3%
Using custom heat mapping tools, we confirmed that VeggieTales’ current product pages had very poor spacing, distracting attributes, and lacked organization. They weren’t awful by any means, but they weren’t optimal either.
Hypothesis: Improving the spacing and increasing the prominence of the primary call-to-action will result in a higher RPV.
Results: The product page variation produced a 13.9% increase in RPV.
Statistical Confidence Level: 96.1%
This is where eCommerce conversions happen. However, VeggieTales’ checkout page included all of the same header and footer elements that were found on every other page of the site. They’re good elements, but they serve as unnecessary distractions to a visitor trying to convert to a customer, and can be eliminated using a website optimization tool like Optimizely.
Hypothesis: Simplifying the checkout by removing all of the distractions in the header and footer – except those designed to act as confidence builders—will result in a higher RPV.
Results: The more streamlined checkout page variation produced a 14.3% increase in RPV.
Statistical Confidence Level: 98.1%
The original homepage had a strong callout for free shipping, which is never a bad idea for converting visitors, but it overshadowed the site’s social icons.
However, we didn’t just want to move the social icons to the header, we wanted visitors to be aware of the immense size of the VeggieTales community, which serves as a confidence builder.
Hypothesis: By adding social icons to the header and making visitors aware of the size of the VeggieTales community, visitors will be more likely to have social interactions within the community and consumer confidence will increase.
Results: Not only has VeggieTales gained more fans and followers, but the variation homepage produced an astounding 36.8% increase in RPV.
Statistical Confidence Level: 95.7%
Original Mobile UI
There’s actually nothing mobile about this UI. It is simply the desktop UI viewed on a mobile device. It’s nice in that it doesn’t rely on Flash, which is still not supported on iOS, but it requires a tremendous amount of effort to navigate the site on a smartphone.
Hypothesis: Sending mobile traffic to a mobile-themed UI will increase the mobile conversion rate and revenue per visitor.
Results: Thanks to testing, VeggieTales now knows they can be generating a lot more revenue from their mobile visitors, as this quick, non-customized, mobile-themed UI saw a 28.1% increase in revenue per visitor.
Statistical Confidence Level: 99.1%
Just the beginning
The improvements VeggieTales has experienced thus far, which have resulted in 38% sitewide increase YOY in RPV, are just the beginning. Once the changes to the homepage variation are applied to the rest of the site, we expect that number to grow even higher. We focused on the lowest of the low-hanging fruit—or should we say vegetable—in order to produce the quickest gains for our client. Now it’s time to pick the rest.
Download the PDF version of the VeggieTales case study from Blue Acorn.