The Future of Software Development: Progressive Delivery & Experimentation

Hi, I’m Lawrence Bruhmuller, Optimizely’s new CTO. This is the first of many blog posts on best in class engineering practices. To kick things off, I wanted to share why I joined Optimizely, and how I see software development evolving.

I’ve been leading and scaling engineering teams for over a decade for high growth startups and large tech companies. From my experience, I’ve seen the same problems crop up repeatedly for technical teams:

• Long release cycles, especially with older, monolithic codebases
• Attempts to move to Continuous Delivery that result in customer found bugs and urgent rollbacks or patches
• Expensive efforts to catch issues in pre-production environments
• Frustrated teams operating in “reactive mode”, spending too much time fixing bugs and reworking features instead of burning down the backlog.
• Engineering cycles spent on changes that don’t have a positive impact on the business … or might even be making things worse!

You may think of Optimizely as an A/B testing platform for marketers. That is definitely the company’s heritage. However, for the last 4+ years, Optimizely has been bringing the power of experimentation to developers, via Optimizely Feature Experimentation. This mission has now broadened into building the leading Progressive Delivery & Experimentation platform to solve the problems described above for product & engineering teams.

That is why I’m here. I want to solve the problems that I’ve faced for the past 10 years, and that many of you are facing today.

Why Agile, CI/CD, and Microservices are not enough

There are many best practices out there as to how to ship faster and more reliably. Today, most teams operate with an Agile mindset, looking to deliver value to customers quickly and then iterate towards a better solution. Continuous Integration and Continuous Delivery automation tools help teams overcome the friction of shipping to production. And breaking up monolithic software into a constellation of services means teams can deploy services separately, without shared schedules or lots of process.

But alone that is still not enough. It does not guarantee that you’ve built the right thing or built it in the right way from a quality or performance standpoint. It does not guarantee that you will delight customers in ways that drive better business results.

There is a new way of delivering software that makes it possible to move fast and get it right. That new way is Progressive Delivery & Experimentation, and it’s what brought me to Optimizely.

What is Progressive Delivery & Experimentation?

In the simplest terms, it is the ability to release software quickly to a subset of your users, and then test and learn in production before rolling out more broadly.

It means validating quality and performance in your actual production environment, with easy ways to roll back if issues are found. Internal users only? No problem. Beta customers only? No problem. Targeted rollouts to certain regions or segments of your user base, or only percentages of each? Easy to do. This is Progressive Delivery.

It means iterating on functionality and UX until you know you are building what users love, and what drives business impact. You can try out multiple variations with enough real user traffic to *know* which is the best choice, for whatever key metric you are optimizing for … user conversions or engagement, or even “under the hood” metrics like latency or error rates. The data will tell you the truth. This is Experimentation.

It means a unified platform for feature flagging, progressive rollouts, 1-click rollbacks, remote configuration, as well as A/B/n and multivariate testing that is both easy to use and based on leading analytical techniques.

Why every product team needs both Progressive Delivery & Experimentation

What starts as the most basic feature flag in your codebase can ultimately turn into a key product decision that requires a full blown A/B or multivariate test. The trouble is that you don’t always know which one it’s going to be. If you have a feature management solution and an experimentation solution that are separate, you need to decide up front if you are planning to perform an experiment, vs. simply a phased rollout or even just a “don’t release it yet” flag. And if you’re wrong, you end up going back and having to re-instrument your code to work with the other solution. The best approach is one that offers a comprehensive approach, with a single set of tools and metrics, allowing more and more sophisticated techniques for the features that need them.

When Progressive Delivery & Experimentation are used together, you have an efficient system for validating both quality and customer engagement across your development lifecycle. Optimizely, as the pioneer of bringing the scientific method to the web, and now with 100s of enterprise customers using our Feature Experimentation SDKs, is uniquely positioned to deliver on this vision. Which brings me back to how excited I am to have joined this team!

Optimizely on Optimizely

Over the coming weeks, I’ll be publishing more on how our engineering organization is putting Progressive Delivery & Experimentation into practice. In the meantime, take a deeper dive into our best practices with this Progressive Delivery & Experimentation Technical Guide and check out our Developer Hub full of use cases and documentation. And to continue the conversation, you can find me (@Lawrence) in our Developer Slack Community.