Upworthy’s Jaw-Dropping A/B Test (That Will Restore Your Faith In Recommended Content)

“We’re maybe as good as a coin-flip at guessing what’s going to work best for our users. We rely on testing to just make better decisions. People are really fascinating and interesting… and weird! It’s really hard to guess their behaviors accurately.” – Peter Koechley, Co-founder, Upworthy

It’s not often a founder is so heavily involved in website optimization – but Peter Koechley likes to do things differently. He’s has been evangelizing A/B testing at Upworthy since just after its website launched back in 2012.

As an online media company focused on elevating issues that matter on social media, Upworthy aims to combine compelling videos with a seamless sharing experience. And from day one, Peter has focused on A/B testing to achieve this. He’s built a company-wide testing culture that relies on data, not opinions, to make decisions – testing headlines, content, and other modules across Upworthy’s site to see what performs best.

One recent test yielded surprising results (and a huge win) for Upworthy.

Recommended content meets social sharing

“In the earliest days of Upworthy, our goal was to find people on social media and grab their attention,” explains Peter, “and then get them to share back out to social media as well. We wanted to optimize that loop.”

But as Upworthy’s audience grew, the team realized that the site’s design was not keeping up with the needs of frequent visitors. Engagement was growing and people were spending more time on the site – but it was difficult to find additional content after landing on a particular video or graphic.

“Our users wanted to dig deeper, but there was no obvious way to get to a second piece of content,” says Peter. Peter’s goal was to increase sitewide engagement, while maintaining the share-optimized user experience.

Peter believed that adding a recommended content module would decrease the number of social shares for each article on average.

“We had already done a lot of testing and found that when we added distractions, user sharing went down,” says Peter, citing the “Paradox of Choice” concept. “We were actually hesitant about adding the module at all.”

The team decided to test it out, hoping to find an option that would provide users with more content without getting in the way of those who wanted to share on social media.

Testing it out

The Upworthy team built and tested 7 different placements and layouts for the recommended content module to see which performed best. They used Optimizely as an A/B testing tool to experiment with content nuggets above and below the featured content, as well as left and right sidebars, and combinations that included a footer.

One variation displayed recommended content nuggets about the feature.

Another variation tested a ride sidebar module.

Caption: One variation included a sidebar and a footer.

After running the split test for just a few days, the team uncovered some surprising results.

View the full story to find out which variation performed best and learn more about Peter’s testing strategy.