How to host a virtual experiment ideation session

“The best ideas emerge when very different perspectives are met.” - Frans Johansson 

Is your experiment program stuck in a rut? Are the test ideas on your roadmap sparse or very one-sided? Are you in need of a little more creativity and inspiration to boost the value of the program? If yes, these are all key indicators that you may need to facilitate an experiment ideation session.   

So how can virtual experiment ideation sessions help? They dedicate a specific time and virtual space to:  

Altogether, they allow you to generate innovative new ideas that wouldn’t have otherwise been possible if done individually in silos. In this article, I will show you in the two parts how I’ve run productive virtual ideation sessions with Optimizely’s customers by addressing (a) the tools and setup needed and (b) a step-by-step facilitator guide.

Setting it Up 

There are plenty of software options available to virtually re-create the in-person whiteboard to which your team may have been accustomed. Options range from simple (using a Google Slide) to more robust and purpose-built platforms like Mural and Miro.   

To get started, you’ll need to prepare your ideation session by selecting and setting up some basic tools: 

A highly usable and expansive virtual whiteboard in Mural.

MURAL has features to vote on ideas and display the results directly on the cards.

In lieu of a robust collaboration platform, you can have dots or shapes represent votes in a Google Slide.

(Tip: Many whiteboard platforms can create share-links to send to external stakeholders without requiring individual logins. This is helpful for generating excitement around the workshop and leads to follow-up sessions, which is a benefit to a virtual session!) 

Running the Virtual Session: Step-by-Step Facilitator Guide

Now that we have all the virtual tooling in place, let’s work through how to facilitate the virtual workshop. At Optimizely, we title our ideation workshops as hypothesis workshops because the end goal is to have a collection of hypotheses that can feed into a testing roadmap. So, now that we understand the output, let’s get to the step-by-step process to ensure you can facilitate a productive hypothesis workshop of your own starting with the agenda.   

A good starting point for an agenda is: 

Let’s break down how we prepare for and facilitate each one of these activities: 

Prework and Problem Definition 

There may be some pre-work needed to have individuals added to whichever software platform you’re using to host the virtual whiteboard. Sometimes, a light introduction is needed, and many software vendors have self-guided user onboarding, so I recommend that participants log on beforehand and familiarize themselves with the experience. 

In the meeting invite, be sure to define the identified problem statement for which we’re developing hypotheses/solutions. This should be a user problem that’s specific enough, tied to a business KPI, and not a solution in itself. See these examples on formulating the perfect user problem.  

It may take a few prep meetings with key stakeholders to align the top user problems to focus on. I’ve even seen entire workshops dedicated to just problem definition (yes, it’s that important to know what you’re solving for!). 

If using a Creative Canvas link, it’s helpful to have categories for each of the ideas/solutions (Design Clarity, Content Clarity, Urgency, etc.). And it’s also another column if there are multiple user groups or stakeholders that are impacted by the problem.   

Introduction 

Start with introducing the purpose of this session. I like the format of answering the “why” (why is this important?, why now?), then the “how” (how will this be facilitated?, how will we elicit ideas?) and then move to the “what” (what is the problem statement?). 

Also, use this time to address any FUDs (fear, uncertainty, doubt) that participants have. This is the point to allay any concerns with the process. Common doubts I hear from teams are “how will I know that my idea will be taken seriously and not ignored?” or “I have ideas that require a lot of change and effort, should I voice these, too?” 

Generally, I like to employ the “creative matrix” framework to guide the brainstorming process. 

Individual Brainstorming 

Set a timer and time-box the brainstorming activity (5 or 10 mins). This part of the activity is individual and gives participants a chance to add their own ideas to the “board” without the influence of the group to deter what one thinks is a good idea. 

Remind participants to not worry if their ideas overlap with someone else’s and don’t worry about the granularity of the idea yet - we’ll refine the specificity of the idea later. The idea should be framed as having two components: the proposed solution and a rationale (either qualitative or quantitative) that indicates why this solution would solve the problem. 

Share and discuss the ideas 

One by one, read out the ideas and encourage discussion. Have the idea owner field clarify questions and support evidence why this idea would work. Decide if the idea needs to be elaborated in greater detail or broken into more granular ideas. My rule-of-thumb: if the solution can be implemented by a “pizza-sized team” (a team that can be fed by a single pizza), then it's the right size. 

Group Voting  

Give the participants a set number of votes that they can use to indicate which idea would have the most potential at solving the stated problem. This is by no means a replacement for a comprehensive scoring rubric, which can come later when these ideas get added to the larger roadmap (our partners at WiderFunnel use PIE). You can also predetermine how many votes a participant has available to cast their vote and if they’re allowed to vote multiple times. 

After tabulating the votes, the distribution of votes may require the facilitator to be the tiebreaker.

Effort vs. Impact Prioritization  

This step encourages participants to consider the effort of implementing the idea versus the impact that it may have at solving the problem. A way to frame this is to plot the cards on a matrix with low to high effort going up a vertical axis and low to high impact going left to right on the horizontal axis.   

As a group, take the top ideas (e.g., top 5) and ask how they would order the ideas on the bottom axis in ranking of Impact only (which one has the highest likelihood to solve the problem?). Then in another round, do the same for relative Effort. Asking the group to rank the ideas relative to one another is beneficial in that it avoids having to define a specific scoring rubric that can often lead to nitpicky discussions on what qualifies as a “5” versus a “4”. It’s easy for people to make relative comparisons rather than assigning absolute scores. 

As a group, it’s helpful to plot your ideas on this Effort vs. Impact matrix to see relative importance.

Framing the Hypothesis   

After the first attempt at idea prioritization, take the ideas that are in the top right quadrant for high-impact and low-effort ideas. These are the ideas that as a group, we can begin building a hypothesis around. Just like any good hypothesis, it should be framed in a simple statement that includes the solution, the result(s), and the rationale. For example: If we do “X”, then we expect “Y”, because of “Z”.  Stated this way, “X” naturally becomes your stated solution, “Y” is your metrics, and “Z” is your rationale. 

Now that we have a fully formed hypotheses, we’ve reached the goal of the ideation session. Depending on your existing process, these new ideas can either be added to a larger roadmap where they’re evaluated holistically (perhaps with a more detailed scoring rubric) or green-lighted as they stand.  

The next step would be to follow your existing experimentation process, which generally would be detailing the idea in more granularity into a test plan to understand the experiment configuration (e.g., on which pages would this test run? How will we configure the stated metrics? Will they be measured as click events, pageview events or custom events?, etc.). 

Now that you’ve accomplished one successful workshop, let’s not stop there. You’re now armed to repeat this with additional functional teams to drive high-impact testing in your company.   

If you are interested in getting started with progressive delivery, you can sign up for a free Optimizely Rollouts account.