Program Managers Level 1

As the experimentation lead, who we call The Maestro, you will learn strategies involved in setting up an experimentation program, how to design data driven hypotheses, the basics of creating experiments, how to interpret and share results across your company.

Concepts addressed in this path:

  • Foundations of experimentation cycle and program structure
  • Align experimentation to your business goals
  • Build an optimization team
  • Understand the purpose of a testing charter
  • Decide what is important to include in your charter
  • Build your own testing plan using our starter templates
  • Connecting revenue to experimentations
  • Primary, secondary, and monitoring metrics
  • Using direct and indirect data to develop experiment ideas
  • Identifying and defining problem statements
  • Crafting hypotheses using the Problem > Solution > Result framework
  • Account structure
  • Setting up projects
  • Manage Collaborators
  • Project and account settings
  • Calculating sample sizes
  • Prioritizing ideas with a defined framework
  • Best practices for:
  • Defining experiment components: Pages, Events, Extensions (to add), Audiences
  • Building experiments with those components
  • Creating variations using Visual and Code Editors
  • Basic experiment QA using the Previewer
  • Includes Full Stack and Web
  • Managing hypotheses
  • Statistics for experimentation
  • How Stats Engine is different from other statistical models
  • Reading the Optimizely results page
  • Taking action on winning, losing, and inconclusive variations

For those who prefer to follow a study-guide instead of reviewing eLearning, we also offer this study guide which provides you articles and other content designed to prepare you for the certification exam.

The certification exam can be accessed at The Maestro Level 1 Exam.

30 mins
Course

Setting Up Your Experimentation Program Structure

This course will teach you the various ways an experimentation program structure can be set up, as well as which individuals should be involved and what various teams might look like.

30 mins
Course

The Experimentation Cycle

Experimentation is an iterative cycle. Learn how to develop processes for your experimentation program and improve your optimization methodology.

30 mins
Course

Align Experimentation to Your Business

The best experimentation programs align closely with the goals of the business. Learn how in this course.

30 mins
Course

Where Should You Experiment?

Figuring out where to experiment is hard. In this course, you will learn how to use existing site analytics to create high impact tests relevant to your industry.

30 mins
Course

Leverage Data to Drive Experiments

Learn to explore data from your analytics, heat maps, and other tools to take your experimentation program to the next level.

30 mins
Course

Solving Problems That Matter

Use your data to identify the problem that matter to your business, and then generate effective solutions to address these problems.

30 mins
Course

Build Your Optimization Team

You can't be a lone wolf forever. In this course, we'll help you build your experimentation team and nurture a culture of optimization.

30 mins
Course

Create a Testing Roadmap

In this course, you'll learn how to define your primary and secondary metrics and create roadmap of future experiments.

30 mins
Course

Prioritize Your Experiments

In this course, you'll develop a prioritization framework to ensure you are focusing on the most impactful experiments.

30 mins
Course

Write an Effective Hypothesis

At the heart of the scientific method is the hypothesis. Using the framework presented here, you'll learn how to write effective hypotheses for your experiments.

45 mins
Course

Manage Projects & Collaborators

This course will teach you how to manage projects in Optimizely X, and add collaborators with the appropriate permissions.

1 hr
Course

Prepare Your Site for Optimizely X

This course will teach you how to prepare your website for Optimizely X Web by creating Pages, Events, and Audiences.

45 mins
Course

Build an Experiment in Optimizely X

This course will take you step-by-step through building an experiment in Optimizely X.

45 mins
Course

Create a Variation in Optimizely X

This course will teach you how to use the Optimizely X Editor to create variations for your experiments.

30 mins
Course

NEW! Strategies for Using Optimizely Rollouts

Optimizely Rollouts is a free feature flagging solution development teams can use to roll out features confidently, target key audiences, and easily roll back poor performing features. This course offers insights to the benefits of feature flags and where they can be used in development workflows.

30 mins
Course

NEW! Optimizely Full Stack: Strategy - Build a Feature

Let's jump into Full Stack and learn a little bit about what SDKs are for, what the datafile is, and how to build our first feature!

30 mins
Course

QA Your Optimizely X Experiment

This course will teach you the various ways in which you can QA your Optimizely X experiment before launching it.

30 mins
Course

Optimization Statistics

In this course, you'll learn how the Optimizely Stats Engine calculates results to enable business decisions.

30 mins
Course

Understand the Results Page

In this course, we'll give you an overview of the Results page so that you can understand and interpret the data.

30 mins
Course

Share your Results with Others

Once you've interpreted your results, why not share them with others to invest in long-term success for your program?

30 mins
Course

Take Action on Winning Variations

A winning variations is almost always a reason for celebration. But what you do next matters more!

30 mins
Course

Take Action on Losing Variations

A losing variation can sometimes feel disheartening, but take comfort in knowing you've tapped into something your visitors care about!

30 mins
Course

Take Action on Inconclusive Variations

How can you make more impactful changes to ensure that your next experiment will be statistically significant?

Certification Exam

Once you've completed the above courses, you are ready to test for the certification. Please visit the The Maestro Level 1 exam.

New Courses

web personalization

Personalization Overview and Strategy

In this course, you will learn various ways of personalizing a website, as well as review best practices for utilizing personalization strategies.

technical

Building Adaptive Audiences in Optimizely Web

In this course, learn about Adaptive Audiences, how to use it, and how it allows you to deliver relevant experiences based on visitor interest.

full stack

NEW! Optimizely Full Stack: Technical - Build a Feature

In the first part of a series of how-to courses, you will learn the first steps to implementing full stack, including installing the SDK and becoming familiar with the datafile, and learn to build your first feature.

full stack

NEW! Optimizely Full Stack: Strategy - Build a Feature

Let's jump into Full Stack and learn a little bit about what SDKs are for, what the datafile is, and how to build our first feature!