Sample Size Genie

Optimize Your Experiments: Introducing Optimizely Opal Sample Size Genie

Unlock faster, more accurate experiment planning directly within Optimizely.

Calculating sample sizes and baselines for A/B tests is a critical, high-volume task often performed outside Optimizely, leading to manual effort, delays, and reliance on specialist analysts. The Sample Size Genie eliminates these bottlenecks, making experiment planning accessible and efficient for everyone.

Why the Sample Size Genie?

• End Manual Data Work: No more pulling traffic and conversion data from GA4 and manually typing it into calculators.
• Integrate Planning with Optimizely: Position Optimizely as your end-to-end platform, from planning to execution.
• Empower Every Team: Make experiment planning faster and more accessible, even for teams without dedicated analysts.
• Say Goodbye to Spreadsheets: Use Opal for precise calculations instead of cumbersome Excel sheets.

How the Sample Size Genie Works:

The Sample Size Genie simplifies complex calculations, providing instant insights for your A/B tests.

1. Calculate Sample Size for a Hypothesis

   Input your hypothesis, desired Minimum Detectable Effect (MDE), and statistical significance, and the Genie will calculate the required sample size and estimated duration.

2. Optimize for Limited Traffic Segments

   Tailor your calculations for specific user segments or pages, ensuring accurate results even with smaller traffic volumes.

3. Leverage Industry Benchmarks

   Don't have baseline data? The Genie can use industry benchmarks to help you estimate sample sizes for new functions or products.

4. Manual Input Fallback

   If GA4 connection issues arise, you can easily provide baseline conversion rates and weekly traffic manually to continue your calculations without interruption.

Interact with the Genie:

Here are some examples of how you can use the Sample Size Genie:

• Calculate sample size for a general hypothesis:
  I'm planning to test this hypothesis: "If we enable image swiping function on Quick View, we will increase ATC on Quick View by 5% because users have more resources to quickly make the buying decision." Can you help me calculate the sample size needed based on current site data?

• Specify targeting for your experiment:
  I'm targeting all visitors from the US on desktop and mobile devices. Yes, I want to detect a 5% lift with 95% Statistical Significance.

• Calculate for a limited traffic segment:
  I'm planning an experiment to rename the 'Complete This Look' block on the PDPs, but targeting mobile users only in the United States region. Can you help calculate sample size?

• Use industry benchmarks:
  I'm about to launch an experiment to allow Add To Cart directly from PLPs on mobile to increase the Add to cart rate side wide. However, this is a new function so I don't have the stats for conversion rate yet. Can you refer to the industry benchmark? (Opal will then ask for your industry, e.g., Please locate to Fashion industry)

• Provide manual input if GA4 connection fails: I want to run an experiment on the image swipe feature and Add to Cart button on product listing pages. Can you calculate the sample size needed based on our mobile users' data? (Opal will then prompt for manual data, e.g., I'll enter the data manually: 1. Baseline Add to Cart conversion rate: 4.5% 2. Average weekly traffic: 10,000)

Technical Foundation:

The Sample Size Genie is built as a composable multi-layer agent, leveraging the Opal Tool SDK, REST Tool wrappers, and external agent adapters. This robust architecture ensures simple tool calls for common tasks and reliable workflows for complex calculations.

---

Stop guessing. Start optimizing. Get precise experiment insights with the Optimizely Opal Sample Size Genie.