Testing at the Romney 2012 Campaign Drives Voter Engagement and Online Fundraising

Website optimization drove six-figure donation total
$182 Million
Online Campaign Donations
10%
Increase in CTA Clicks
6x
Increase in Sign Ups
19%
Increase in State Specific Sign Ups
  • Industry
    Political Campaign
  • Location
    Boston, MA

From day one, Mitt Romney’s digital campaign team understood a common truth: the campaign is not a creativity contest – what looks best and what works best for the website is not always the same.

“We tried to be very conscious that this team doesn’t have creative opinions, this team has data,” says Ryan Meerstein, a senior political analyst from Targeted Victory, the agency who ran testing and optimization for the Romney campaign. “It’s hard for the team to argue with a graph that proves what works and what doesn’t.”

From the start, the team looked to increase email sign-ups on mittromney.com a primary goal.

Email is still the golden goose of fundraising when you’re making direct solicitations. We’re seeing each email valued at anywhere between $7-8 in future revenue.

— Ryan Meerstein, Senior Political Analyst, Targeted Victory

Knowing how beneficial email was to raising money for the campaign, they tested heavily on the homepage and splash pages of mittromney.com always optimizing for email sign-ups. Among AB testing tools, Optimizely was their platform of choice.

Between May 2011 and November 2012, the Romney Campaign’s 140-person digital team along with Targeted Victory ran hundreds of tests.

“Once we saw the ease of using Optimizely, the ideas started flying. We wanted to start testing just about everything,” Meerstein says. “We started on the splash page and when we saw success, we continued to build from there.”

Call-to-action button test

The team started optimizing for donations with a test on the main call-to-action in the right upper corner of the homepage. They wanted to see whether button color – blue, green, yellow or red – and word choice – “Contribute,” “Support”, or “Donate” – impacted the likelihood of a visitor to click.

Overall they found that color did not have a definitive impact, but the word “Contribute” did show a statistically significant improvement of 10%

Knowing that “Contribute” converted visitors to click more often than “Donate,” the team changed verbiage all over the site – and in all email messaging – to reflect the test results.

Still armed with the goal of maximizing email sign-ups, the team focused the next iteration of testing on the carousel images on the homepage. A carousel is a rotating slideshow of images that designers frequently use to showcase featured content. They tested using a carousel versus a static image offering visitors the chance to win a trip to the Republican National Convention with the headline, “Be There in Tampa.” The main metric they measured was the percentage of visitors who reached the email sign-up confirmation page.

They tested four variations:

  1. The control – A full page moving carousel.

  2. A half-height moving carousel.

  3. A static image with an “enter to win” form.

  4. A static image with a “learn more” button.

Adding the form to the homepage image increased the percentage of visitors who signed up by 632%.

In this case, visitors seldom reacted to the “Learn More” more button. They reacted extremely well to the immediacy of the sign-up form giving them the chance to win with filling out just two form fields.

State Specific Splash Pages

Next, the team used geographic location as a pull to encourage visitors to sign up for email updates. The team wanted to gauge whether visitors signed up more with a message specific to their state or a generic one.

The data clearly showed success in personalizing the message. With this test as testament, the team decided to make the splash page specific for each state. They used geotargeting in Optimizely to send visitors from each state to a page with a message specific to that state. So no two visitors to mittromney.com from different states saw the same message. They tested personalized call-to-actions based on absentee states and early vote states. Visitors from Ohio saw messages directing them to early voting locations and the hours they were open. Visitors from Colorado saw targeted messages for how to get an absentee ballot.

We could go in [Optimizely] and set up the personalized experiences in 30 minutes. In the final weeks of the campaign, there’s a huge difference between something being live on Tuesday morning and Thursday night.

— Ryan Meerstein, Senior Political Analyst, Targeted Victory

Testing Takeaways

Focus on Time-to-Test

When the stakes are incredibly high, waiting for results or for bottlenecks incurred by the software is not an option for presidential campaign teams. In a matter of days, the Romneyteam had conclusive results about which variations won. Without relying on the creative or engineering team, the analysts themselves used the tools within Optimizely to create huge gains in email sign-ups.

Deliver a Personalized Experience

Using Optimizely, the team delivered a unique one-to-one experience for every visitor to the site from September 2012 to election day. Personalization proved to be a powerful tool for the Romney campaign. They saw greater signups on the splash page and more interaction with local events advertised on the site, especially as voting started.