September 17, 2015

Headline A/B Testing to Boost Ad Impressions: A Case Study from Ringier

Blick, the leading online and print tabloid in Switzerland, has boldly ventured into unchartered territory for most in the media industry: editorial A/B testing. By experimenting with headlines and teaser images, Blick’s editorial team is able to better understand what readers are most interested in, in real time.

Blick, the leading online and print tabloid in Switzerland, has boldly ventured into unchartered territory for most in the media industry: headline testing.

“It has become evident that you can no longer drive journalism by gut feeling alone.” – Simone Haberli, Project Manager, Blick

By experimenting with the headlines and teaser images on their homepage, Blick’s editorial team is able to better understand and measure what readers are most interested in and boost ad impressions across their site.

“It’s fascinating to follow live what visitors are actually interested in.” – Leo Wehrli, Editor, Blick

Blick is a publication owned by Swiss Ringier Group – a global media company that oversees more than 120 news­papers and mag­a­zines, sev­eral ra­dio and TV stations, and 80+ web and mo­bile plat­forms. The Ringier team recently adopted editorial A/B testing as a strategy to help better understand users, get direct feedback about products, and position their publications more favorably against their competitors.  

Editorial A/B testing – and headline testing in particular – is a hot topic in the publishing scene.

But only a few pioneers (Upworthy is one example) currently use this lever actively to increase visit duration, page views, and advertising revenues. Ringier recently launched its editorial A/B testing program and in just one month has seen remarkable results.

The Editorial Testing Process

Being relatively new to testing and optimization, the Ringier team opted to start on a small scale – the goal being to first prove the ROI of testing before investing in a full-scale CMS integration. Using Optimizely’s REST API, Wehrli, and his team built a simple tool, with the same basic functionality of a CMS integration.

Using their headline testing tool, Wehrli and his team build and launch experiments, and optimize three articles each day. For each test, they measure the average number of page impressions per session and click-through rate to the article. Using Optimizely’s Auto Allocation feature (currently in private beta, with a broader launch coming soon), they are able to optimize the experience more quickly by sending more traffic into the variation that Optimizely deems statistically most likely to win, even before each test shows a clear winner. 

The following are two examples of the headline and teaser image experiments the Ringier team ran to optimize their content.

Experiment 1: Teaser Image

The Ringier team ran an experiment on a recent article on how babysitting responsibilities have changed with the emergence of new technologies.

They tested two variations:

They tested teaser image A (left) a babysitter’s “to do” list against image B (right) a remote control theme.

Results: Image B won the experiment, generating 38.46% more clicks to the article and 8.58% more page impressions for visitors who saw the remote control.

Experiment 2: Headline A/B Test

For a recent photo gallery posted on their website, the team tested two different headlines and teaser image combinations.

Combination A: an image of a hand and a the headline “These Pictures were Taken with an iPhone!”

Combination B: an image of a cute child and the headline “World-Class Apple Photos”

Results: They hypothesized that the variation with the photo of the child would attract more attention and engagement. After running the experiment, however, the opposite was true.

Of the visitors who saw the variation with the cute child image, 25.56% fewer clicked through, resulting in 7.19% fewer page impressions in the sessions of those visitors.

After just one month of testing, the team has seen a tremendous amount of success and ROI from their editorial optimization program.

Overall, editorial testing has had an extremely high ROI for Ringier’s business – accounting for a significant increase in page impressions that has resulted in a major lift in ad revenue.

More than half of all editorial A/B tests had a significant winner only a short time after the start of the test. This means the team is able to quickly optimize and iterate, maximizing the number of clicks and impressions generated by each piece of content.

On average, page impressions per visitor increased by 4.9%, and the click-through rate on the homepage to the article increased by 19.7%. With such a large amount of traffic flowing through the site on a daily basis, these improvements have had a tremendous impact in driving some of Ringier’s key business objectives–content views and engagement.

“The aim with A/B testing is really to completely integrate it in our process.” – Martin Fessler, Head of Digital Media, Blick

The team will continue to amplify the number of tests they run and will gradually equip even more of their global offices with similar tools and strategies for successful editorial content optimization.

Ready to learn more about how best-in-class publishers use optimization to their advantage? Check out our free resource: The Ultimate Guide to Website Optimization for Media Companies.