Incorporate feature flags and experiments into your sprint cycles

Not only are teams organizing differently, the way in which the software development community releases software has also dramatically changed. Innovation in technical software has made developing software easier; from improvements in tracking workload and workflow, to managing version control, to providing continuous integration and continuous deployment capabilities. In the past, software was developed and released with a mentality of Build, Launch, Pray, with very little insight into how the feature resonated with users, or if they even wanted the feature to begin with.

The Old Way:

With all of these advancements in technology, process and a healthy dose of cultural mindset change, there is often this question of “How do I get started and how do I incorporate this into our day-to-day process to get to the “New Way”?

The New Way:

So, how could this be incorporated into your process? 

Just as you may be thinking about new feature ideas for your users, you’re likely to be stepping through some experimentation to see if it’s something you should build. Whether it is prototypes in front of a few users, conducting qualitative research, or running a painted door experiment you’re exploring the possibility of the feature resonating well with users (i.e. delights them!) and is valuable to your business. Especially if you’re limited on time and budget, the painted door experiment will give you confidence in tandem with Optimizely’s Stats Engine, to decide whether you should build the feature or not.

Once you’ve determined a feature is going on your roadmap and you begin to define and break up the work, you will also be thinking about how you want to release this feature (on/off) and to whom. These tasks are something you want to capture as part of the breakdown of work and Jira story creation, all of which can easily be linked to Optimizely. A series of Jira stories may look like this: 

Create a feature:

Jira #1: “As an end user, I want to see Feature A, the redesigned look/feel of the page.” Note to engineering: The flag will help us roll out this new page to a subset of users. As a developer, I need to open the Optimizely UI and create a feature in the UI. 

Implement a feature: 

Jira #2: “As an end user, I want to see Feature A, the redesigned look/feel of the page.” Note to engineering: For Feature A, I need to implement the feature flag in my code.

Create feature variables:

Jira #3: “As an end user, I want to see the optimal experience for Feature A.” Note to engineering: Create variables that allow you to configure the feature (without deploying new code) and to discover the optimal experience. 

Toggle and rollout:

Jira #4: “As the system, I want to turn Feature A on/off and rollout to a subset of users.”

In a future sprint, we suggest you run Feature tests to further optimize key metrics. Testing with different configurations and combinations helps you find the most optimal experience. 

Steps to a Minimum Viable Product

In my days as a Product Manager at Orbitz Worldwide (think Orbitz.com, now owned by Expedia), my development team and I had to figure out how to sequence the same types of concepts above but with tooling that wasn’t as robust and flexible as Optimizely. Each software development team is different and each has their own way of working so you’ll have to experiment on what works for you and your team(s). After all, you’re applying these concepts to building a minimal viable product (MVP) with hopes of getting to a mature product.

In my experience, these principles went a long way to making feature flagging and experimentation work as part of your sprint cycle:

Only plan for what you can do in a sprint cycle

• Be sure to start small and break up work that can be delivered in a sprint. Identify as part of your sprint work that a feature can be turned on/off via a feature flag. 

Have shared goals and understanding of each other’s goals

• Be transparent and clear about what user and business metrics your product hopes to drive – everyone on the team should understand these metrics and embrace them. It’s also important to understand engineering’s point of view on what’s important to them as well (e.g. releasing on-time, increasing throughput and high quality code). Agreed upon and aligned product & engineering teams results in happier users and employees. 

Optimizely Rollouts are an easy path to releasing features

• Identify a feature and use rollouts to validate and drive key performance and business metrics. 

Collaboration and empathy go a long way

• Be as collaborative as possible in finding the right solutions to customer problems — really dig down deep and have lots of empathy for user problems
  • Full stack variables can help create solutions that offer options and configurations that allow you to answer unknowns
  • For example, if you are building a new mobile app sticky search feature, you may want to know, what should the search button CTA be? Where should we place it? How long should the bar be?

You can define variables as part of this feature and experiment with the most optimal combination that drives usage and engagement. See below for an example.

Variables within a Feature Configuration

Define who does what

• Define a RASCI for your team that aligns to the activities of your sprint cycle and the experimentation process. 

Leadership creates a safe environment

• Product and Technology leadership: encourage a safe environment to make it safe to: “Deploy at least 50% of the team’s code behind a feature flag or experiment”. 

Below is a RASCI of how my team incorporated experimentation into our software development lifecycle. Depending on the feature and the feature’s maturity, we’d prioritize, define and plan the work based on the principals above as part of our sprint. As with any process, you refine it until it’s working well for you (i.e. continuously experiment on your process!).

Aligning Software Delivery and Testing