Learn Something New: QA and test

Short video
To manually ensure your feature is working as intended before rolling it out to more users, we recommend setting up a system using audiences to QA your feature before rolling it out to more users.

In the following steps, you will learn how to configure Optimizely so that when you set a test cookie, you will force enable or disable the feature you want to QA. Although this guide is targeted towards a web application and use of cookies, you can use the same principle in any application without the use of cookies.

1. Create a QA Audience

Create an attribute in the Optimizely UI and name it testAudience.

Then create an audience and name it Test Audience. For the audience definition, drag and drop the testAudience attribute you defined earlier. Set the condition for the audience to be when a visitor matches the condition for when testAudience Boolean is true.

Add this audience to your rollout so that the rollout is targeted only to users in the Test Audience and roll the feature out to 100%. This audience will be your way of targeting the feature to yourself when you want to manually verify the feature is working as expected.

Note: If you already have audience targeting conditions for the rollout, then you can combine the test audience with other audiences with an OR condition to ensure you will pass the targeting conditions with just the test audience condition.

2. Set a test cookie for yourself

We recommend using a cookie to control the QA experience. Although this recommendation is primarily for Web based applications, the overall pattern can be applied to other platforms without the use of cookies.

If working from a browser, set a cookie in your browser with the name optimizelytestcookie and value to be the string true. Set the expiration date of the cookie to be when you expect to be done manually verifying your feature. We recommend a few hours later than the current time.

3. Implement the attribute

In order for the cookie to take affect, we need to implement the attribute defined above by passing it to the Is Feature Enabled API as one of the attributes.

From your application code, get the value of the optimizelytestcookie, convert the string 'true'/'false' value stored in the cookie to a boolean, and pass that value as a testAudience attribute to the isFeatureEnabled API as outlined in the code sample below:

rollouts testAudience

Once the value of testAudience can be controlled by the value of the test cookie, you can now add and remove the test cookie to manually force the experience and verify the feature is working as expected.

4. Verify your Feature

You should now be able to verify your application feature while the cookie is set for yourself.
Manually verify your feature is working as expected.

5. Remove the QA Audience

Once you are done manually verifying the experience. You will want to first rollback the experience to 0% and then remove the QA audience from the Optimizely rollout. You have now verified that the feature is ready to rollout for users without the test cookie and can perform your rollout confidently and safely!