5 Issues 97% of Testing Teams Want to Fix

“Optimization is a journey, not a destination.”

We’ll often point to this statement as a guiding light for our own testing, and as a recommendation to our thousands of customers—testing and optimization are iterative, learning-driven processes that are ongoing.

Chances are, if you’re an optimizer at heart, you’re experimenting with just about everything.

In our recent benchmark report survey, we asked optimizers at companies of all sizes and industries – what would you change about your optimization efforts, if anything?

Only 3% of respondents said they wouldn’t change a thing about their optimization efforts! They reported that there are always opportunities to improve, in true optimizer fashion. The changes that they mentioned fell into two main categories:

• Changes to the people working on optimization
• Changes to their optimization processes

These are difficult, but approachable challenges that we’ve researched and written articles about before. Here is a roundup of articles and resources to help you:

• Get more engineering and design resources for your testing program
• Add new members to your optimization team
• Establish more consistent processes for optimization
• Decide on your overarching goal/success metric for optimization
• Secure buy-in for optimization from your executive team

1. What to Say When: You Need Engineering and Design Help to Run More Complex A/B Tests

You’re not alone: 53% of the optimizers we surveyed for our Benchmark Report plan to increase their budget for optimization in the next year. Hiring additional technical help was high on their wish list when we asked about why they were planning a budget increase.

There’s no question; if you had technical know-how on your side, your optimization program could scale faster, experiment with bigger, more complex tests, or move from testing on the marketing site to testing in your product.

Educate and align your design and technical teams by addressing their concerns and speaking their language:

• Share how Suma Warrier of Move, Inc. aligned her optimization team with their internal QA and Engineering teams to ensure she follows their existing workflows when running experiments. She now gets part-time help coding more complex experiments from her engineering peers.

• Show how optimizing through a redesign process can help ensure that a new design is both performing well and visually delightful. We love these redesign examples from Smartwool and Soccerloco.
• They might not know that A/B testing is for engineers too, according to Kyle Rush, Director of Frontend Engineering and Optimization at Hillary for America. When he led optimization at the Obama 2012 campaign, his team A/B tested a new architecture for processing donations, and discovered a huge bug in the process.

If you’re in need of resources, bringing in an agency partner to help can be a flexible short or long-term solution. Take a look at Optimizely’s directory of approved Solutions Partners for more information.

2. What to Say When: You Want to Hire Dedicated Team Members for Optimization

41% of optimizers would like to add more people to their teams in order to scale their efforts.

Maybe you’re only just getting started with optimization, or you’ve been championing optimization while doing all of the work yourself. You want to scale your program, but you need help. Here are some of the roles like you are looking to add to their teams:

Already have a job req ready to go? Post open positions to our Job Opportunities board in Optiverse.

3. What to Say When: You Need More Consistent Processes For Optimization

Almost half – 48% – of optimizers wish they had more consistent processes.

Excellent optimization is only possible when you pair excellent team members with a consistent group of processes. Sometimes, getting process right takes time and iteration, and some self-reflection.

When you want to streamline you brainstorming, hypothesis generation, and communication processes, bring these tips to your team:

• Learn how Pauline Marol and the Hotwire team generate hundreds of high-quality experiment ideas by structuring their brainstorming around conversion “themes”: Organized ideation is how Hotwire generates 120+ experiments per year
• Show why a strong hypothesis is an essential component of every test you run: An experiment without a hypothesis is dead on arrival.
• Pick the right cadence for your testing program and run more, high-quality experiments with this example: How The Next Web scaled their A/B testing output 100x.

4. What to Say When: Each Experiment is Measured Against Different Metrics

35% of benchmark survey respondents said they were still in search of an overarching metric for optimization. The key to optimization alignment with the broader business is choosing the right metric to measure. You’ll always be tracking micro-goals, like conversions and engagement, but it’s important to track the downstream effects of your experiments so you can articulate the business impact of your program.

Great optimization programs have nested goals, like this example: your conversion metrics should ladder up to larger company goals.

• To make optimization stick, focus on a quantifiable goal. Many companies will measure revenue as a straightforward, iron-clad method of measuring and communicating the value of their testing.
• Finally, for a more in-depth read, learn how to leverage the right metrics to delight your customers, engage your audience, and grow your business with our complete guide: Building your Company’s Data DNA.

5. What to Say When: You Need Executive Support for Testing to Get Started

About one in five optimizers (22%) are looking to establish executive buy-in for optimization. This can be a daunting challenge, but we’re here to help.

• Keep it simple with 6 killer stats to sell your boss on optimization.
• Here’s a strategy lesson on building your testing organization that can help you define your vision for your team and program before presenting it to your higher-ups.
• For a soup-to-nuts guide that will educate any stakeholder on the benefits of optimization, send them the Experience Optimization Playbook, complete with case studies, optimization benefits, and tips for getting started.

Looking to dig further into metrics like these? Join tomorrow’s webinar: Sharing Findings from the Optimization Benchmark.