While the examples I give are geared towards e-commerce sites, any type of site can find ideas here.

Before you can come up with meaningful A/B tests for your e-commerce store, you need to understand what your customers value when making purchases and what turns them away. Surveys directly on your site or through email are a popular method of collecting qualitative customer feedback. Surveys have some drawbacks though: It takes a lot of time to collect enough data from them, many people will completely ignore them and the data can be biased if you ask leading questions.

Other, more unconventional, sources of information exist that can be extremely helpful in generating ideas for A/B tests.  These sources are:

1. Customer service records
2. Public forums
3. Competitor data
4. Internal search keywords

In this post we’re going to look at how you can use the data from these sources to seed meaningful A/B tests.

1. Customer service records and tickets

You customer service calls and chat transcripts provide a treasure trove of data. You can choose to look at them as complaints about your store and products, or advice about where you can improve. These questions, complaints, and requests customers submit are important enough to warrant a phone call or an email — do not ignore them as a source of ideas for optimization.

Waterfilters.net received numerous complaints from customers having trouble finding products on their site. Even the support team had a tough time finding certain products when customers asked.

Instead of ignoring the complaints, the team at Waterfilters.net realized something wrong with the store search. They decided to test out a new search function on their store, which lead to an increase in conversion rates by 11%.  Check out the full case study from Google here.

Tip: Categorize tickets for easier discovery

Scanning through all of your past customer service records can reveal great test ideas, but scanning your tickets is tough if you have hundreds of records. Make this process easier by grouping them into categories

For example, issues related to products are one group; issues related to checkout are another. Make sure there’s no ambiguity regarding which group an issue should go into. By grouping tickets like this, you make it easy for your team to sort through them and identify common complaints.

When daFlores.com started working with Conversion Rate Experts to optimize their online flower store, they started with lots of customer research. Apart from regular methods like Qualaroo surveys, they roped in their customer support team to make lists of common inquiries from customers.

They found that customers were often concerned about how long their flowers would take to arrive. This data led to a hypothesis for a new test: communicating urgency around delivery would increase checkouts. They tested a banner that urged customers to buy quickly to be eligible for same day shipping.

daflores.com

Et voila, it increased conversion rates by 27%. A huge bump in conversion rates from addressing a common customer inquiry.

It’s important that the marketing or optimization team have a tight feedback loop with the customer service team. If a certain issue is happening regularly, there’s clearly a need to be proactive and nip it in the bud before it becomes a big issue. Also, after countless hours on the phone with customers, they definitely have some ideas for tests they’d like to try out themselves.

2. Public feedback

These days, consumers are quick to complain about an experience on any public site. While it can be frustrating for many brands, public feedback also offers a wealth of ideas for your A/B testing pipeline. Take to online forums, social media, and even your own product pages are all fair game to see what ideas you can find.

For example, these Zappos customers commented on a Nike shoe product page. Both customers were misled by the sizing and found that the shoes were too thin.

zappos.com

The 3-star ratings they gave  makes the product look bad and will probably hurt its conversion rate. Data like this could lead Zappos to test out a different sizing chart.

Tip: Get public feedback delivered to your inbox

It’s easy to keep track of comments on your site and social media accounts, but what about all those other websites, blogs, and forums out there? Just like with customer service records, if you sort through public comments you can find trends that will lead to meaningful A/B tests. There are a number of online tools you can use to get:

• Google Alerts – Enter your company name or other keywords you care about and you’ll get an email alert any time someone mentions you online.
• Social Mention – Track mentions on blogs, social media and bookmarking sites.
• Mention – A paid tool with real-time tracking and an analytics dashboard.

Dairy Queen has developed a monitoring system for brand mentions across various channels, from social media to their blog pages. They track anyone who has complained about them online and redirect them to a feedback form on their site.

By consolidating mentions across all channels, including their core customer support, DQ makes it easy for their team to handle issues and identify trends. Creating a similar process for your store will save you a lot of time and help you identify major issues before it’s too late. Read more about DQ’s approach to customer service on MarketingSherpa.

From cumulative data via feedback forms, service records and other research, the team at TopCashback.co.uk learned that customers lacked trust for the product. However, they had a lot of good press going for them so they thought that testing out press logos would help.

topcashback.co.ukSure enough, with social proof from the addition of logos, conversions increased. Here’s the full TopCashback case study from Conversion Rate Experts.

Another example of ideas from public feedback: Mobile retailer, Wanelo wanted to test a new product feed for their mobile app. The initial data suggested that the new changes increased engagement but at the same time, also received a lot of negative feedback about the new feed.

Instead of relying only on the data for their engagement metrics, Wanelo listened to customers and experimented with it. They hypothesized that adding a second feed would help them solve customers’ complaints while maintaining the engagement their existing feed created.

The new test proved to be a success and the positive reviews were off the charts.

3. Competitor data

Newsflash: customers are also talking about your competition, and you have access to that data too. Complaints and compliments about your competitors site is another fertile ground of ideas for new tests.

There’s no reason why you shouldn’t find out what’s going wrong with your competitors and fix it on your site. When customers have a problem with a competitor’s site, they’ll come to you because you have a solution.

Just like you should set up Google alerts for your own site, set them up for competitors’ sites. Track all the public feedback, good and bad, about your competitors. This will help you analyze what they are doing so that you can do it better.

When travel company, Sunshine.co.uk ran usability tests on their site, they also thought to run them on their competitors’ sites. This gave them a direct comparison and showed them exactly what their competitors were doing right.

By comparing themselves with travel competitors, Sunshine discovered that people prefer to book with Sunshine because they offer specific key benefits. So, they tested different ways to play up these benefits on their site. These tests, generated from usability tests on other websites, doubled their conversion rate, bringing in an additional 14 million pounds per year!

sunshine.co.uk

Voices.com also analyzed competitors’ websites to determine their strategies. This helped them find opportunities to position themselves against competitors. The A/B tests that came out of this analysis contributed to an increase in conversion rates by 400%!

4. Internal search keywords

If  your website has a search bar, it’s likely shoppers are using it to find products quickly. The keywords shoppers use are great sources for A/B test ideation.  Is one term appearing all the time? If so, try highlighting that product on the homepage. Moral is, if you’re not tracking the terms consumers enter into your search bar, you should start doing that now.

To track this, you need to have the right tool. Check out Elicit, Celebros, and internal site search from e-commerce platforms like Shopify, Drupal, and Magento.

4 Wheel Parts offers a good example of using search queries for A/B tests. They started using a new tool, Celebros site search, to track search queries across every page on their site, giving them valuable data they never had access to before. After a couple of months, they had enough data to show them trends like which products were searched for most often and from which page.

They found that consumers searched for a particular brand name on one of their product pages. Testing out brand logos and links on that page was the next logical step for them. The result was a 59% increase in revenue from that page.

They also tested different ways products showed up in search results. By emphasizing certain promotions, they were able to increase conversion rates across their site by 4.65%. Read the full case study about 4 Wheel Parts on Marketing Sherpa.

Stock photo website, Bigstock experimented with their search algorithm, comparing the likelihood photo searchers would select a photo with a “fuzzy” or “exact match” would return better results.

bigstock.com

Start (and don’t stop) digging

Hopefully, you’ve already been keeping records of your customer service calls and internal site search queries. If you haven’t, it’s time to start now. Public feedback and competitor data already exists, but it needs to be collected and sorted. The only thing left for you to do is to dig through this data and find insights that will lead to new A/B testing hypotheses.

This isn’t a one-time task though. Creating a continuous process that tracks, gathers, and sorts all this data as it comes in will make it easier for you in the future when you run out of new ideas to test. Add this data to insights you’re pulling from other sources like surveys, brainstorm sessions, usability tests, your site analytics and you have an endless source of A/B testing ideas!

What are some other unconventional sources of inspiration you’ve found that have sparked interesting A/B tests on your site?