Challenge: QA Your Experiment with Arijit

Arijit is one of the stakeholders in your project who understands the need to QA your project, but he’s not sure about the specific logistics yet and has a few questions for you. Can you help him understand how QA will work? See if you can correctly field his questions.

We’ve got some stakeholders who should have their eyes on the QA process. Is there a way we can allow them to see this test and give input?

  • We’ll have to launch the experiment first, but we can send them links once the experiment is live so they can review it.
  • We can create a mock-up of each variation so they can review our proposed changes and make suggestions.
  • We can share specific variations with stakeholders by sharing a link to the test without exposing that link to our visitors.
  • We’ll have to create an account in Optimizely for each of them so they can review the QA like we are. I’ll set up a webinar to walk them through it.

Is there a way for us to see how our variations will look in different browsers to ensure that they’ll look and behave the way we want them to everywhere?

  • Sure! Optimizely has a feature built-in that allows us to run a cross-browser test as part of our QA process.
  • Optimizely’s visual editor has a warning that will allow us to know when something we enter won’t format properly in a specific browser type. We can use that warning to identify areas we need to adjust as we build.
  • We’ll want to run the QA on a variety of browsers and hardware so we can see what our experiment will look like in each format. That will help us to know if we need to make any changes before launch.

Can we QA paused or draft experiments, too?

  • Yeah, that’s no problem. There’s a setting that we can use to activate paused and draft experiments specifically for QA.
  • Paused and draft experiments are automatically available to QA, just like any other experiment we are in the process of launching.
  • Sure, we can do that. That data is excluded by default, but we can use a forced parameter, the share link, or the preview tool to QA those experiments.
  • We can QA paused or draft experiments only if we make them active. We can return them to draft or paused status after running the QA.