6 Best Practices from 251,391 Website Experiments (and How to Apply Them)

The beautiful thing about learning is that no one can take it away from you. –B.B. King

Simple is good.

In this forum, we try to keep things short & sweet because we know you are frantically busy. After all, this is a blog—not API documentation.

With that in mind, here are Dan’s 6 “Best Practices” from 251,391* A/B and Multivariate tests:

1. Define quantifiable success metrics.

2. Less is more. Reduce choices.

3. Words matter. Focus on your call to action.

4. Seek the global maximum. (explore before you refine)

5. Fail fast.

6. Start today.

But as an old sailor once told me: “Just because you have a compass doesn’t mean you know how to sail.”

Touché, old sailor. Touché.

Likewise, just because you read the 6 best practices above doesn’t make you an optimization expert. So let’s take a look at how some of Optimizely’s customers put these 6 best practices into action:

1. Define quantifiable success metrics.

Defining success is your north star. Your guiding light. Your destination. Attempting to optimize your website without knowing your Key Performance Indicator (KPI) is akin to setting sail without charting your course (something our old sailor friend certainly wouldn’t condone). Here’s how Demand Media charts their course:

• KPI #1: Revenue (overall)

• KPI #2: Revenue per visitor (RPV)

Boom. You’d be surprised how many companies set sail on the high seas of optimization without first defining quantifiable success metrics for their businesses. By defining the above KPIs, Demand Media is able to stay on course amid the many optimization decisions they make every day.

To learn how to identify KPIs at your business, download our eBook,Building Your Company’s Data DNA.

2. Less is more. Reduce choices.

Profile: Online food ordering service; 4th fastest growing private company in US.

Seamless applied best practice #2 to their homepage:

Original

Variation

By removing the optional “Search” field, Seamless was able to reduce choices and create a more… er, seamless experience for their visitors.

Results:

• 20% increase in click-through-rate

• 5% increase in completed orders

3. Words matter. Focus on your call to action.

Profile: One of the fastest-growing online legal services company, based in SF.

Rocket Lawyer applied best practice #3 to their “incorporate” landing page:

Original: “Incorporate Now”

Variation: “Get started”

Result: “Get started” had a +1.5% increase in conversion.

4. Seek the global maximum (explore before you refine).

Profile: 170-store private retailer of housewares, furniture, and home accessories

Crate & Barrel employs best practice #4 by doing in-depth usability testing as a way to infer things they should test. This approach empowers them to cast a wide net and explore big ideas based on user feedback (as opposed to the HIPPO method).

Read more about crafting excellent experiment hypotheses with user feedback in this case study from Realtor.com.

5. Fail fast.

Profile: 400-person video game entertainment company.

IGN decided to test a seemingly innocuous idea: moving the “Videos” link (top navigation) from the right to the left. Simple right? Surprisingly, this small tweak catalyzed a -92.3% drop in clicks. Bad news bears.

Original: “Videos” link on the right

Variation: “Videos” link on the left

But IGN was on top of it: they paused the losing variation and diverted traffic back to the original. IGN was able to navigate the squall and correct their errant course quickly. They failed fast.

Interesting side note: Based on segmentation data, IGN learned they had alienated their returning visitors—who constituted the vast majority of the decline—by moving the Videos link to an unfamiliar location. As Bill Gates reminds us, It’s fine to celebrate success, but more important to heed the lessons of failure.

6. Start today

Profile: $315MM provider of social, local and mobile marketing solutions for local businesses.

Start today. Sounds easy, right? Not always. DexMedia (formerly DexOne) was going through a large corporate restructuring. Given this fundamental shift, there was a general lack of interest, guidance and initiative regarding website optimization within the org.

But then a Senior Product Analyst named Ed Gallagher took it upon himself to change course:

1. Ed contacted Optimizely to coordinate additional training for his team.
2. Ed got folks from Marketing, Website, Product and Engineering in the same room to brainstorm high-impact testing ideas.
3. Ed worked with Optimizely to design a testing game plan for 1, 2, 3 months out.
4. Ed scheduled a recurring weekly team meeting to discuss their progress and instill accountability.

Ed started today. He took initiative. He lit a fire.

Where do you want to sail? What do you want to test?

Create an experiment now and start cruising.

*Total number of A/B & multivariate experiments created through Optimizely’s split testing tool as of 6/4/2013.