When to Feature Flag, Rollout, or A/B test?
Feature flags, phased rollouts, and A/B tests all help you ship more confidently. Feature flags allow seamless feature releases and rollbacks. Phased rollouts catch bugs early and often. A/B tests make sure you’re building the right thing for your business.
So, when do you use one over the other?
This is a question I’ve heard from different developers, so I put together this high-level decision tree which is useful for determining when to set up a feature flag, plan a phased rollout, or run an experiment. If you find this helpful, this diagram is an example from my free e-book Ship Confidently with Progressive Delivery and Experimentation, which offers end-to-end guidance for deploying feature flags at scale. Now, let’s dive into the decision tree!
In general, it’s good to recognize that a/b tests are best when you have specific measurable metrics and a hypothesis for how your changes will affect those metrics.
Feature flags give you complete control over a release or launch date.
And rollouts mitigate risky changes by releasing code to small portions of traffic at once.
However, it’s important to note that certain feature flag and A/B testing platforms, like Optimizely, allow for all three capabilities to run off the same feature flag implementation. This means that you can decouple how you implement your feature from how you plan to release, experiment, and learn from it. See how this breaks down in the API call `isFeatureEnabled`:
You can also see how they all work together to support a strong Progressive Delivery practice across your entire product development team.
Let me know what you think!
This is part of a series of best practices to help your company successfully implement progressive delivery and experimentation to ship faster with confidence.
If you like this content, check out my free e-book: Ship Confidently with Progressive Delivery and Experimentation which offers more best practices from just getting started to scaling these technologies organization-wide.
And if you are looking for a platform to get started, check out Optimizely’s free offering.