The Progressive Delivery and Experimentation Virtual Summit

Thanks for attending our free virtual event to serve product development teams who are pushing the limits of modern software delivery. You can relive the magic by registering below to receive the recordings!

Deliver resilient software

Learn how engineering teams can build more resilient products by embracing progressive delivery techniques like feature flags, progressive rollouts and experimentation

Ignite product growth

See how the most successful product teams in the world design, develop, launch, and grow digital products that drive growth and engagement

Go from insight to action

Hear from leaders in experimentation, data and analytics to learn how to create a feedback loop to deliver deeper insights and ultimately better products

Nicole Forsgren

Keynote Speaker

Nicole Forsgren is a DevOps research and strategy expert who shows leaders how to make their technology count. Best known as lead on the Shingo Publication Award-winning book Accelerate and on the State of DevOps Reports and currently the VP of Research and Strategy at GitHub.


  • Test & Learn

No results found

  • Now more than ever, technology matters. Long before COVID-19 made remote work the norm, digital products were driving value and disruption for organizations. But delivering software safely remained a debate: For years, we thought slow meant more stable code and releases… even though we kept seeing fire drills and suffering through weeks-long releases.

    The best, most innovative organizations have realized that it's possible to release code -- in ways that are safer, more stable, and more reliable while providing insights into what users value and need. Nicole Forsgren, author of "Accelerate" and leading software development and DevOps researcher will share insights on what teams can do to improve their software delivery as well as stories from organizations that have seen the benefits of these practices.

    • Ship quality code more efficiently
    • Create an adaptable engineering culture that delivers products that users need
    • Establish best practices for tightening feedback loops with reliable data
    Learn More
  • In this session, Ellen Chisa, the founder of Dark, will discuss what's desirable (and risky) about programming in production. This workshop will explain the tradeoffs between delivering value to users quickly, and being able to provide the safety and reduced risk that modern software development tools allow. She'll address solutions ranging from no code to the full software development lifecycle, and what it looks like to just use code.

    Join the session to learn:

    • About modern infrastructure patterns, including how to use "deployless" infrastructure as a step beyond "serverless."
    • How working with real data while developing software can reduce errors and decrease time to value.
    • To determine when a comprehensive toolchain, no code, or "just code" is the right solution to the challenge your team is facing.
    Learn More
  • HelloFresh is the global leader in meal kits. As a subscription-based business, with both a digital and in-person user experience, they are not the typical e-commerce platform. Solving product challenges through experimentation is embedded in their culture. Join Abdelrahman Hussein, Software Engineering Lead, and Aleksandra Vangelova, Product Owner from HelloFresh to learn how they built an experimentation architecture that allows them to run and evaluate tests for increasing customer retention.

    In this session,

    • Understand the challanges of a subscription based product
    • Learn about the approach to evaluating retention experiments for such product
    • How engineering enables product experimentation in such setup
    Learn More
  • Selecting the right metrics is key to proving sustainable business impact through experimentation. When it comes to product optimization, only focussing on simple metrics like conversion rates is a short-sighted product strategy. Choosing the wrong indicators of success leads to misguided decisions that hamper growth, innovation, and velocity. In this session, Luís Trindade, Principal Product Manager from the global fashion platform Farfetch, will discuss best practices for identifying metrics to understand and address the constantly evolving needs of your users. Join the session for best practices using data to invest in features that drive sustainable growth and deliver customer value.

    Learn how to:

    • Instrument product experiments with better and faster results
    • Use data to get customer insights and problem solve
    • Define a north star metric and a single goal tree to align priorities for success
    • Drive efficiency by increasing accuracy for measurement and estimation
    Learn More
  • Your product is the most important influence on your user experience and lifetime value. The ability to continually learn from your product changes and maximize your outcomes is a critical indicator of driving revenue for your business. Don't be complacent with incremental changes, learn how to find insights that will drive a 10x impact for your business.

    This session will cover:

    • Using your product as the influencer for your user’s experience and lifetime value.
    • Learning from your product changes and maximize your outcomes to drive revenue for your business
    • Finding insights that will drive a 10x impact for your business
    Learn More
  • Has the outcome of an experiment ever gone so strongly against your intuition that you doubt its correct implementation? This is certainly a possibility, as correctly implementing an online experiment is fraught with data quality challenges. There is a considerable amount of engineering and processing of data before it reaches the analysis, leaving plenty of room for bugs and biases to creep in. One of the strongest signals of an incorrect implementation is a sample ratio mismatch (SRM) - when the number of users assigned to each variation differs significantly from what is expected under the intended random allocation.

    This talk will:

    • Demo a novel SRM test which allows experimentation teams to rapidly detect if there is a bug in the implementation, even while the experiment is running, allowing developers to quickly fix the underlying issue.
    • Provide an introduction to both the mathematics of the SRM test as well as the newly open sourced library which developers can immediately integrate into their platform.
    Learn More
  • Developing a robust A/B/n testing program is critical for every product organization. But there is not a one size fits all approach. Your program must be tailored to your business and audience to be successful. Join Oji Udezue, VP of Product at Calendly, who will share how to craft high-impact customer insights and product experiments, tailored for your company. Oji will draw insights from 20+ years of Product Management at companies like Microsoft and Atlassian and discuss the importance of using qualitative methods to help develop smarter hypotheses with the right tools and practices for your organization.

    In this session you will learn:

    • Why a strong experimentation culture is important for your product organization
    • Proven ways to execute high-impact A/B/n tests with high-quality customer discovery
    • The importance of conducting interviews with successful and unsuccessful customers and how to see through your customer’s eyes with live watching techniques
    Learn More
  • Segment supports a huge ecosystem of partners and platforms, but there are always requests for more customizable options. What if, instead of answering each custom request, we empower customers and partners to build exactly what they’re looking for? Join PM Andy Yeo as he discusses the ideas, conversations, testing, and validation that brought their latest product release—Functions—to life.

    When you attend this webinar, you will:

    • Learn how product teams go from customer feedback to shipping features
    • Walk away with tactics for testing and iterating to launch features that solve customer needs
    • Hear an exclusive preview of an upcoming product launch at Segment
    Learn More


Nicole Forsgren is a DevOps research and strategy expert who shows leaders how to make their technology count. Best known as lead on the Shingo Publication Award-winning book Accelerate and on the State of DevOps Reports, Nicole has helped some of the biggest companies in the world improve their technology work — including culture, process, and architectural aspects. She is a technical founder/CEO of DORA (DevOps Research and Assessment) with a successful exit to Google and a member of ACM Queue's Editorial Board. In a previous life, Nicole was a software engineer, sysadmin, professor, and hardware performance engineer.

Nicole holds a PhD in Management Information Systems and a Masters in Accounting. She has published several peer-reviewed journal papers and has been awarded public and private research grants (funders include NASA and the NSF), and her work has been featured in various media outlets, including the Wall Street Journal, Forbes, ComputerWorld, and InfoWeek. She has been named a Top 10 Thought Leader and expert in DevOps, and advises companies, execs, and investors regularly. You can read Gartner's take on Nicole’s move from academia to bring science to the tech industry here.

Nicole Forsgren

VP of Strategy and ResearchGitHub


Oji Udezue is the VP of Product for Calendly where he oversees and leads all of product and user experience. He was formerly head of product for Atlassian’s communications products, and launched its first new product since their IPO: Atlassian Stride (sold to Slack). A product veteran with past roles at Microsoft, Bridgewater Associates and his own startup, InterMingl; Oji is passionate about bringing new unique and scalable products and ideas into the world. He mentors startups with a focus on companies with diverse founders, and drives startup investments in Africa as a passion project.

As a 20+ year veteran of product management, he is very visionary and thoughtful about product craft and practices that lead to building great products. His ultimate goal is productizing functional telepathy, but for now will settle for transforming how teams and the people in them collaborate, and optimize their workflows at work. Most days you can follow him on twitter @ojiudezue, instagram @ojiudezue or find his scribblings on

Oji Udezue

VP, ProductCalendly


Ellen Chisa is the CEO & Co-Founder of Dark, a programming language for building backend webservices. Prior to Dark she was VP Product at Lola and a Product Manager at Kickstarter and Microsoft. She holds a B.S. Electrical & Computer Engineering from Franklin W. Olin College of Engineering.

Ellen Chisa



Luís is currently the Principal Product Manager for the Experimentation area at Farfetch, promoting a Test & Learn Center of Excellence across the company. And before was leading the Data Products area (recommendations, conversational commerce and a big sponsor of the computer vision initiatives). Luís started a few years back as a software engineer but quickly had the necessity to understand how users interacted with products and that lead to being responsible for all the interaction and user experience. Then in a shift to the startup ecosystem accepted the challenge as Technology Lead at a company builder, developing new global distributable products and services in the digital area, helping the portfolio to get the necessary boost on tech, user experience and product expertise.

Luís Trindade

Principal Product ManagerFarfetch


Aleksandra Vangelova

Product OwnerHelloFresh

Abdelrahman Hussein

Software Engineering LeadHelloFresh

Tanner McGrath, Head of Product Analytics at Amplitude, brings with him many years of experience advising corporations in product, analytics, and growth. With a diverse background ranging from biotech and pharmaceuticals to the on-demand economy, he was most recently at Postmates where he started the Growth Engineering team. Applying product, engineering and data leverage to grow Postmates as fast as possible. He created an impact centric culture through product analytics, growth automation and experimentation.

Tanner McGrath

Head of Product AnalyticsAmplitude


Andy Yeo is a Product Manager focused on platform extensibility at Segment. He has also held various go-to-market roles at Hootsuite and LinkedIn while living and working in Australia, Asia, and North America. Andy is an avid volleyball player, amateur photographer, and food fanatic. He loves connecting with fellow PMs to discuss enterprise tech, remote work and the future of education - feel free to connect via Twitter (@yeoa) or LinkedIn.

Andy Yeo

Product ManagerSegment


Justina is the Developer Evangelist Lead at Optimizely. Prior to joining Optimizely, Justina worked at Dashbot as a developer evangelist, where she educated people on analytics for conversational interfaces. She studied Business at U.C. Berkeley, Haas School of Business and is passionate about data visualization and pizza, occasionally both at the same time.

Justina Nguyen

Lead Developer EvangelistOptimizely


Michael Lindon has a PhD in Statistics from Duke University and joined Optimizely in 2019 from his previous role as a senior data scientist at Tesla. His research interests are primarily in Bayesian hypothesis testing methods and Bayesian decision theory. At Optimizely he works with other statisticians to develop sequential tests for novel applications in the online experimentation space. He's a big fan of functional programming and likes to use clojure in particular.

Michael Lindon

Staff StatisticianOptimizely


Asa is the Developer Advocate Lead for Optimizely. Prior to joining Optimizely, Asa worked at Microsoft as a software developer where he built the Internet Explorer site recommendation algorithm. He studied Mathematics and Computer Science at Harvard University and is an avid breakdancer.

Asa Schachar

Principal Developer AdvocateOptimizely


Bilal Mahmood is the Head of Product of Machine Learning at Amplitude. Previously he was the co-founder of ClearBrain, a predictive analytics startup acquired by Amplitude, and in a prior life led data science at Optimizely. He studied biology and economics at Stanford University and is a Gates Cambridge Scholar.

Bilal Mahmood

Head of ProductAmplitude


Lawrence Bruhmuller joined Optimizely as CTO to lead the technology strategy and overall engineering organization. He previously held VP of Engineering roles at WeWork, ClearSlide, and Symantec (Vontu), taking on entire software engineering organizations and leading them into the next phase of growth. Lawrence holds a Bachelors in Mathematics and Engineering from Harvey Mudd College, and a Masters in Applied Mathematics from Claremont Graduate University. He lives in Oakland, California, with his wife Amy and their three daughters.

Lawrence Bruhmuller

Chief Technology OfficerOptimizely


Kris is a Senior Product Manager at Optimizely, leading product development for Optimizely’s Program Management product and managing overall enterprise experience. In her spare time, you'll likely find Kris day dreaming in a local art gallery or playing with the nearest dog.

Kris McKee

Senior Product ManagerOptimizely


Mike Davis is a senior member of the Optimizely engineering team with a passion for the developer experience. Before developing on the Full Stack product, Mike was a founding member of the Optimizely Data Infrastructure team, building enterprise scale analytics solutions.

Mike Davis

Senior Staff Software EngineerOptimizely


Claire is the Chief Product Officer at Optimizely, the world’s leader in progressive delivery and experimentation. Previously, she was the CEO and co-founder of Experiment Engine which was acquired by Optimizely in 2017. Prior to launching Experiment Engine, Claire led product development for several startups and technology companies including Electronic Arts and Claire is an expert in high-velocity experimentation programs, product strategy and user experience, and a passionate advocate for women in technology. Claire graduated from the University of Texas with a B.A in American Studies.

Claire Vo

Chief Product OfficerOptimizely


Jessica is a Senior Software Engineer at Optimizely whose life was changed by the I/Own It program. She has a BA in Geography from Vassar College. In her spare time, she loves to sew, read, bike, talk, and spot critters in the wild.

Jessica Chong

Senior Software EngineerOptimizely


Tom Zurkan is a Senior Software Engineer at Optimizely working on Optimizely Full Stack, Mobile, and OTT. He has also been working with AWS tools for several years. His interests lie in modern languages and how best to develop elegant crash proof code.

Tom Zurkan

Senior Software EngineerOptimizely


Nathan is a Senior Software Engineer at Dosh, focused on growth and experimentation of their consumer-facing apps. He loves building products that positively impact people's lives and chose to work at Dosh to help in the mission to democratize the advertising industry for the benefit of people everywhere. Before joining Dosh, he was on the mobile platform team at HomeAway. He has a B.S. in Computer Science from the University of Texas at Austin, with concentrations in Mobile App Development and Information Security.

Nathan Chapman

Senior Software EngineerDosh


Jon Noronha is responsible for leading a team of Product Managers to discover new ways for companies to experiment more across websites, apps and every level of the stack. Prior to Optimizely, Jon led Pacific Rim engineering teams at Microsoft to rethink visual search. Working at the intersection of machine learning, distributed systems, and image processing, he learned the power of quantitative analysis and constant A/B testing to make data-driven decisions.

Jon Noronha

VP, Product ManagementOptimizely


Yeesheen is a Senior Product Manager of Growth at Optimizely. In this role, she works to drive user engagement and growth with Optimizely's powerful progressive delivery and experimentation platform to help teams ship software faster, and with more confidence. Prior to joining Optimizely, Yeesheen was a researcher specializing in trends in futurism, life extension technologies, and art. She received a Ph.D. in Cultural Studies from U.C. San Diego and still loves tacos and yoga.

Yeesheen Yang

Senior Product ManagerOptimizely



Platinum Gold Community

Watch Test & Learn Sessions On-Demand

Welcome back. Not you?