By default, Optimizely shows results for all visitors who enter your experiment. However, not all visitors behave like your average visitors. You can use various attributes to filter your entire Results page or the results for an individual metric; this is called segmenting. Segmenting your results is a powerful way to gain deeper insights about your customers to design data-driven experiments and personalization campaigns.
For example, imagine that you run a subscription media site and you're experimenting with value messaging across the site. Your results show a "win," but you'd like to see how visitors on different plan types (basic, plus, and premium) responded. For a more granular view, the experiment designer would have created a custom attribute to pass in visitors' plan types. You can use this attribute to dive deeper into experiment results. Maybe you'd find that your change performs well for basic and plus plans, but premium users reacted negatively. This insight could influence your next round of experiments and decision-making.
Using the Segment dropdown you can drill down into a segment of your visitors and filter by a single attribute value, or by a combination of attribute values (learn more about using combinations).
For Web, we support segmentation by Browser, Device, Source (Direct, Campaign, Search, Referral), Campaign (UTM parameter contents). If the experiment designer built custom attributes, these are also sent to us and are available to use in filtering.
For Full Stack, or SDK experiments, segmentation is available by custom attributes you send us.
Concentrate on Variations
Another important feature on the Results page is that you can adjust your view to concentrate on variations other than the original. So if you see variations 1 and 4 appear to be closely tracking each other, change the Baseline pull-down to set the baseline to one of these variations and compare your results to something other than the original.
Important things to note about segmentation:
- You can get results based on any singular condition, any 'OR' conditions, and any 'AND' conditions.
- You can also segment by date range.
Cautions when Filtering
Keep in mind that you cannot draw statistical conclusions from segmented results. Use this as an exploratory data analysis tool only. While you will see the segmented Results page plot a statistical significance graph, this graph should be disregarded. Optimizely does not do any multiplicity correction when viewing segmented data. The data, however, can be used to inform future experiments.