Optimizely’s experiment results page provides a high-level summary plus modules for each metric attached to your experiment. In the next few sections, we’ll walk through the summary and information provided in the metric modules. To be clear, we focus here on the results of Optimizely Web experimentation. A slightly different Results page is provided for Optimizely Web Personalization and Multi-Armed Bandit campaigns and will be discussed in separate lessons.
Before we go into the modules available on the Results page, let’s focus a moment on the left-hand navigation.
Here you have:
Actions available on the experiment - including pause/start, preview, or archive (depending on your collaborator rights, these icons may be inactive)
Information regarding - date when changes were last published; number of days running which looks at the exact times an experiment was started/stopped and represents the number of full days an experiment is/was running, only whole numbers are used, so there is no rounding up (i.e., 33.9 displays as simply 33); audience(s) targeted and pages included in the experiment; and, number of visitors to-date
Note: All results in Optimizely are in local time, according to the time zone set on your machine. Results are typically available within 5 to 10 minutes of Optimizely's receipt of the data.
The Experiment Results page begins with a Summary which provides a high-level overview of data from the experiment. It allows you to quickly compare how each variation (listed vertically) is performing for each metric (listed horizontally), compared to the original.
Optimizely shows the number of unique visitors who’ve been bucketed into each variation. In our example above, 13,937 visitors (or 25.11% of visitors in this experiment) have seen the original variation, 13,886 (or 25.02%) have seen variation 1, and so on.
Note that we refer to unique visitors. Whenever a new visitor is bucketed into any Optimizely experiment for the first time, they are counted immediately as a new unique visitor in the experiment and the Visitors count on the Results page is incremented. All subsequent visits by this visitor to other pages or returning visits to the site at other times (later in the day, tomorrow, next week, next month, etc…) will not increment the Visitors count again. Users who delete their optimizely EndUserId cookie or use a different browser/device are treated again as a brand new visitor.
Note: The visitor count here speaks solely to visitors on your page(s). This does not affect billing or impression counts in any way.