How IGN Moves Fast and Challenges Assumptions
IGN’s product team created a culture of experimentation to speed up product development and increase views of their video gaming content
IGN is the web’s largest video game content publisher on the web, catering to more than 68 million monthly users on their website and in their apps. For their fans, IGN’s premium content keeps them engaged with their digital presence and their brand. For advertisers, IGN offers a vital, premium audience.
Since even the smallest variations in their website design, copy, and images can have a big impact on their key audience metrics, such as video and page views, no change to IGN’s properties is published without A/B testing it first.
Experimentation runs deep in IGN’s culture. It helps the product team run development at a speedy pace, pushes them to think of alternative creative treatments, and reminds them to challenge their assumptions before publishing a change based on intuition alone.
A company-wide culture of experimentation
Experimentation using A/B testing taught the IGN Product team to experiment and learn how their audience will react to features, content, and design in the real world before making a permanent mistake.
“You should absolutely be thinking about testing alternatives, because your first guess is often the wrong one,” said Todd Northcutt, VP of Product at IGN. “The faster you can test, the less likely you are to waste time in development or push something out to the audience that’ll have negative impact.”
The product team repeatedly shares successes to spread this philosophy to other departments at IGN. “At an all-hands company meeting, I put up six variations of a video player we tested and had everyone vote with a show of hands for the one they thought would perform the best. And they were wrong. The prettiest one did not have the best performance,” said Todd.
They even bring the results of Optimizely tests into their sales meetings with clients to encourage them to test different ad creative.
Design decisions that balance data and brand
When multiple teams are involved in an important design decision, A/B testing helps IGN resolve disagreements, saving valuable resources and time.
“Data is the great equalizer. It levels the playing field. Everybody can get hot in a brainstorming meeting, and data takes the emotion out of it, as well as the idea that ‘This guy’s opinion is more valid than that guy’s opinion.’ You can have a discussion based on results,” said Todd.
For example, when IGN redesigned their video player page, the team was divided about whether to pursue a more familiar video player format, or if the UX should be a departure from other online video players. To determine whether familiarity versus reinvention was the best optimization path to pursue, the IGN team tested several patterns of the video player against one another.
Ultimately, the familiar video player performed best with IGN’s audience, so the team chose to optimize and continue iterating upon those familiar patterns instead of a reinvented player.
But IGN is careful to balance test results with brand considerations, and sometimes they override data to stay true to their values.
For example, IGN increased clicks on their video “poster states” (static images of embedded videos) by 18% after testing 12 design variations. One of the successful variations darkened the video image so it was more difficult to see. Users had to click the image to figure out the video’s true content.
The team considered carrying that learning over to the video player page and obscuring other, smaller video thumbnails to encourage more follow-up video views. But this felt like clickbait, and they decided against it.
“It would've been easy to say, ‘Let's just make the thumbnails dark so people have to click on them to see what the video is about.’ But we don't want to trick people into clicking, even though the data said this may perform a little bit better. That’s not what we’re after as a brand,” said Todd.
Moving fast in product development
A/B testing allows IGN’s product team to skip some of the time-consuming activities that drag down development, such as building consensus among teammates and stakeholders from cross-functional teams.
“We’ve got pretty smart people, but I don’t think consensus is where we find the optimized solution to a problem,” says Mark Ryan Sallee, a Product Manager at IGN. “That’s where A/B testing is really valuable. You can get ideas from a bunch of people, put them to the test, and empirically find out which one is best.”
Access to an experimentation platform like Optimizely keeps IGN’s product managers productive even when they don’t need to reach consensus.
For a company that is constantly publishing new gaming content, not to mention their expansion to film, television, and comic books, there’s no shortage of features to build and tests to run. IGN stays nimble and their users love them for it.
We can test and iterate at great speed. Some days we have conversations over lunch about variations. When lunch is over, we can fire up an Optimizely test and see what the audience responds to, instead of just guessing or wondering.— Todd Northcutt, VP of Product, IGN
Increase in Video Views