The Experimentation Guide for Product Development Teams

Learn how today’s leading teams build and test products at scale.

A master class in experimentation

Test and Learn is a guide for product managers, data scientists, engineers, and designers to dive deep into the intersection of insights-driven product development and agile software delivery.

Grow your product

Learn how the most successful product teams in the world design, develop, launch, and grow digital products that drive business growth by implementing an experiment-driven product lifecycle.

Go beyond Agile

Understand how engineering teams can embrace techniques like experimentation and feature flagging to go beyond Agile and support continuous delivery.

Learn from the experts

Expert speakers from today's leading companies will share best practices, real stories, and new insights about creating a test and learn culture in product and engineering.


No results found

  • In an age of ever-increasing distractions, quickly creating customer habits is an important characteristic of successful products. How do companies create products people use every day? What are the secrets of building services customers love? How can we create products compelling enough to "hook" users?

    Nir Eyal, the bestselling author of "Hooked: How to Build Habit-Forming Products", has constructed a framework for designing better products and will share his years of research in this practical workshop. This webinar gives product managers, designers, and marketers a new way of thinking of the necessary components of changing user behavior by studying how the world's most engaging products keep users coming back again and again.

    Learn More
  • Quantitative measurement is the key to scaling businesses, processes, and products and making them better. It sounds easy: just pick a number, and improve it. It's not.

    Choosing a metric is an exploration of a many-dimensional space with no map and no guide. Until now. This talk will teach you the science of choosing metrics that will guide you to building a better product. When you attend this webinar, you will:

    • Learn how applying data is as much about questions as it is about answers
    • Discover the trade-offs between different metrics
    • Understand systematic ways to measure your product funnel
    • Learn how to isolate the most important metrics to maximize your product impact
    Learn More
  • Product-led companies are winning in this digital era by rallying their teams around a product North Star: the metric that connects the business outcomes they are trying to deliver to the work they do each day.

    In this talk, you’ll learn a practical framework for how to determine your product’s North Star and how it can shift your strategy.

    By attending this session, you will:

    • Learn how to define a North Star and why it is so important for product development teams
    • Learn how the best product teams use a narrow set of meaningful metrics to align product strategy with actual team initiatives
    • See examples of how North Stars change your product roadmap and drive your business forward
    • Walk through a specific example of setting and evolving a North Star
    • Feel ready to take learnings back to your teams
    Learn More
  • At, experimentation is an important part of our product development cycle. On a daily basis we implement, deploy to production, execute and analyse hundreds of concurrent randomized controlled trials — also known as A/B tests — to quickly validate ideas.

    From entire redesigns and infrastructure changes to the smallest bug fixes, these experiments allow us to develop and iterate on ideas safer and faster by helping us validate that our changes to the product have the expected impact on the user experience.

    Learn More
  • In creating experiments to increase product sales or customer acquisition, the end goal is clear: the more orders placed, the more dollars back to the business. But how should we think about prioritizing tests to increase product engagement?

    In a world where we can encourage users to take a wide variety of actions with unclear business benefits, it seems almost impossible to determine what is best. How can we promote behaviors that build habits and promote retention? The answer is simple: we analyze what habits matter most. In this session Olivia will share:

    • How she created the Wall Street Journal’s first habit model
    • How the team identified which actions influence member tenure
    • How WSJ uses habit data to inform their testing and engagement strategie s
    Learn More
  • The role of design is often overlooked on growth teams that are moving fast and running experiments at scale. When applied correctly, design can be your growth team's secret weapon. Join Angel Steger, growth design lead at Dropbox, to learn how to leverage design thinking and design craft to super-charge your growth team's velocity while driving high-quality output. We’ll walk through tools and case studies to give you ideas you can put into motion right away.

    Attendees will:

    • Learn how to leverage the Design role within a Growth team
    • Learn how design quality works in the context of a fast-moving team
    • Learn how to use design thinking to differentiate between haste and velocity as a cross-functional team
    • Walk away with tools to learn quickly while making meaningful progress against large unknowns
    Learn More
  • Across the web, businesses are competing for user attention and loyalty in marketplaces that are constantly growing and evolving. In this fast moving environment, being data-driven in improving customer experience is the only way to reliably get and stay ahead of the competition. This is easier said than done - issues of what to focus on next and whether your work is improving business outcomes inevitably arise. Acting quickly and acting with confidence can sometimes be at odds with each other. In this session, Chrix Finne (VP of Product - Heap) and Matt Gomes (Director of Marketing - Opploans) discuss how Opploans approaches these challenges and more.

    Learn More
  • When shipping new features, do you optimize for speed or quality? What if you could have more of both? In this presentation, Emily will describe how the use of feature flags, observability, and the principles of code ownership come together to help Honeycomb engineers ship quickly and safely. Adopting these tools can help engineers more rapidly iterate on new features, reduce deployment risk, and quickly resolve incidents, all while keeping customers happy. When you attend this webinar, you will:

    • Learn how to incorporate feature flags into your new feature release process
    • Understand how observability can help your engineers ship new features quickly and safely
    • Learn about the principles of code ownership and how to adopt them on your team
    • Find out how these three components will help your team increase safety, speed, and user happiness
    Learn More
  • Learn how product leaders at Mailchimp and Optimizely:

    • Use experimentation and insights to build products customers love
    • Create psychological safety for team members to fail fast and learn rapidly
    • Foster collaboration across Product, Analytics, Marketing, and Engineering teams
    Learn More

Nir Eyal

Keynote Speaker

Nir Eyal writes, consults, and teaches about the intersection of psychology, technology, and business. He is also the author of Hooked: How to Build Habit-Forming Products

As Senior Director of Product Management at Optimizely, Jon Noronha is responsible for leading a team of Product Managers to discover new ways for companies to experiment more across websites, apps and every level of the stack. Prior to joining Optimizely, Jon coordinated engineering teams across Seattle and Beijing to rethink visual search at Microsoft. As part of Bing’s Image Search team at Microsoft, Jon worked at the intersection of machine learning, distributed systems, and image processing and learned the power of usability studies, quantitative analysis, and constant A/B testing. Jon has a bachelor’s degree in Computer Science from Harvard University.

Jon Noronha

Senior Director, Product ManagementOptimizely


Ketan Gangatirkar leads Engineering at Indeed for Job Seeker products. For the last 10 years, he's been helping millions of people get jobs. Along the way, he's made some good product decisions and some terrible ones. He used to think he had great ideas, but being wrong all the time cured him of that. Now all he wants is scientific evidence. He was analytical before it was cool. It's not cool, and it never has been, but he's still hoping. Someday. Someday…

Ketan Gangatirkar

VP of Engineering – Job SeekerIndeed


Alicia is a Growth Product Manager at Amplitude where she focuses on driving business growth through experimentation. During her 5 years at Amplitude, she has met dozens of product teams and learned how they measure and drive business impact. She also co-wrote "Mastering Retention", a 155-page playbook on understanding and improving user retention in digital products that has reached over 25,000 readers globally. Alicia applies a scientific mindset to all of her work, stemming from her 7 years working in biomedical research.

Alicia Shiu

Growth Product ManagerAmplitude


Lukas is the Director of Experimentation at where he helps thousands of people run experiments across a diverse set of products that empower people to experience the world.

Lukas Vermeer

Director of


Olivia leads the digital testing for Wall Street and She’s helped inject testing into all aspects of the customer lifecycle, including acquisition, engagement, product, and retention. Her team has run over 500 experiments over the past 3 years and is continuing to find new ways drive the product roadmap and expand acquisition strategies.

Olivia Simon

Manager, Product OptimizationDow Jones


Passionate about building tools that help people live their best lives, Angel has led teams to tackle opportunities productivity, creativity, education, and consumer genomics. Her interest in experimentation and learning began in her early days at 23andMe, where she learned how to leverage user research to drive new paradigms in tandem with analytics measurement. Currently managing Growth Design teams at Dropbox and previously at Pinterest, her teams have focused on driving customer acquisition, activation, engagement, and retention through connecting customers with relevant, actionable value. Outside of work, you'll find her at Pilates, watching movies, grooming her 9' plant wall, and experimenting with food.

Angel Steger

Director of Product Design, Growth Dropbox


Chrix is VP Product at Heap, where he leads product management efforts. Previously, Chrix helped build products and product teams at Confluent and Optimizely, and spent 6 years at Google, where he launched mobile advertising products and the first version of Google Analytics for mobile apps. Chrix graduated from Harvard University with a degree in math and music theory.

Chrix Finne

VP, Product ManagementHeap


Matt is Director of Marketing at OppLoans, where he leads all CRO and testing efforts. Previously, Matt helped build and scale the Direct Mail pre-screen programs for both OppLoans and Avant, and spent 3 years at sitting between the marketing, analytics, and finance teams. Matt graduated from Brandeis University with a degree in Business and Medieval History.

Matt Gomes

Director of MarketingOppLoans


Emily manages the engineering & design teams at In the past, she's worked on javascript, performance and monitoring at other developer tools companies like Bugsnag & GitHub. In her free time, she organizes an unconference called AndConf, makes many checklists, and likes to talk about disaster preparedness.

Emily Nakashima

Director of


Lauren Schuman is the Director of Growth at MailChimp. In her free time, she's also the Co-Founder of ThinkFluency, an EdTech company that specializes in innovative digital products for teachers and students. As a customer-centric marketing leader with a passion for building best-in-class online customer experiences, she has spent the last 14 years driving growth for eCommerce and SaaS businesses of all sizes.

Lauren Schuman

Sr. Director of Product InsightsMailchimp


Claire Vo is the Chief Product Officer at Optimizely, the world’s leader in digital experience optimization. Previously, she was the CEO and co-founder of Experiment Engine which was acquired by Optimizely in 2017. Claire is an expert in high-velocity experimentation programs, product strategy and user experience, and a passionate advocate for women in technology.

Claire Vo

Chief Product OfficerOptimizely



Access all sessions and resources on-demand