For experiments that are not mutually exclusive, Optimizely uses a seed, or unique value, for each experiment to bucket the user. The unique value determines whether a user enters a particular experiment. Because the seeds are random, unique, and not mutually exclusive across experiments, some users enter multiple experiments. For example, imagine two experiments: A and B, each receives 20% traffic allocation (the percentage of total traffic that is eligible for the experiment). Here's the expected traffic allocation:
16% of traffic falls in experiment A only
16% of traffic falls in experiment B only
4% of traffic falls in both experiment A and experiment B
64% of traffic is not in any experiment
In the example above, results from experiment A and experiment B may be skewed. If users who see both A and B behave differently from users who see just A or just B, then the results for A and B are skewed by the overlap. This is called an interaction effect.
If experiments A and B are mutually exclusive, Optimizely chooses the same random seed (which is unique to the exclusion group) to bucket users in experiments A and B -- and distribute traffic to one or the other. This method ensures that experiments can't overlap for the same users. If experiments A and B are mutually exclusive, the traffic allocation looks something like this:
20% of traffic falls in experiment A only
20% of traffic falls in experiment B only
60% of traffic is not in any experiment
Optimizely also ensures mutual exclusivity between experiments in an exclusion group that run at different times. How? By assigning bucket ranges to experiments using a stratified sample of available buckets, with strata that consist of all current and previously allocated bucket ranges.