‘How does an A/B test affect my page load time?’ and other important questions

George Stephanis calls himself a code monkey. He is a developer at Speck Products and a Core Contributor at WordPress. We asked him a couple questions about the technical side of A/B testing.

Optimizely: How does A/B testing affect my page load time?

George: Split testing or A/B testing as a concept doesn’t need to affect page load times at all. You could, for instance, split the versions on the server-side in your Content Management System (Magento, WordPress, Drupal, whatever you’re using), so that two versions of a page are being delivered to users. Javascript tools such as Optimizely are completely client-side, and these may add some additional load time onto your pages. And even then, the size of the Optimizely script is largely dependent upon whether it has to bundle jQuery in with itself (which it does by default) or if you are already using jQuery in your website — in which case it could reduce the size of the base Optimizely script from 35kb to 17kb excluding jQuery. This is of course neglecting the fact that the script will be cached for subsequent page loads, not needing to redownload itself — but even so, the load time of the first page on your site is typically understood as the most important.

What is actually happening on my site while running an A/B test?

What actually happens when you’re running a split test is that some of your visitors will see something different from the rest.  An element could be moved around from the left to the right side of the page, some verbiage may have changed, a color may be a bright green, instead of a mellow blue.  How this is accomplished varies from one method to another — some folks may add a new body class to your ‘challenger’ variant or add a new stylesheet to the page head, and then style things via CSS differently like that.  Some may simply change out the text server-side.  Others, like Optimizely, use JavaScript libraries such as jQuery to grab elements on a page and modify their attributes.  This makes it much easier for non-coders to devise and implement tests, however it may not be compatible with every website, for example, sites that are based off of a “Responsive” design using CSS @media queries to rearrange layouts based on the browser dimensions.  When you change the layout of items via jQuery in one browser width, they may suddenly begin breaking when the layout should be adapting in another browser width.

How does A/B testing affect my SEO ranking?  

A/B testing itself doesn’t impact SEO, but making your site more relevant/interesting to visitors will improve SEO rank in the long term. Too many people view search engine ranking as being wholly dependent upon what Google sees when it looks at your site directly.  The best way to increase your search engine ranking is to aim your content at your visitors — not Google.  After all, it’s the inbound links that your visitors post elsewhere that will most positively affect your ranking, and split testing and other data-driven and data-informed design methods are aimed directly at making sure you have the most positive impact on your visitors.

What happens if the testing platform crashes?

It will depend on the method used to implement the split testing. Some companies that run multiple servers behind a load balancer may designate one of their application servers as the ‘dark horse’ challenger, and use its analytics to compare the results it yields versus the others, and if it happens to go down, the traffic will get spread evenly between the other servers behind the load balancer.  For a client-side service such as Optimizely, the included JavaScript is pulled in from a CDN.  If that goes down (for example, Amazon’s cloud going down for Christmas and taking Netflix down with it) the changes won’t happen to the page — long story short, if your testing system breaks, your original page – the ‘status quo’ –  will display as normal.

Do you need to know how to code to do A/B testing? 

Performing split or A/B testing may require some knowledge of code depending on the method that you’re using.  Optimizely, as an example, doesn’t require an advanced knowledge of web development methods — but if you do know what you’re doing, you can certainly leverage that for better results.  Some other server-side methods may require you to know CSS, HTML, PHP, ASP.net, or others.