248 - Feature Flags & A/B Testing: A Deep Dive with Ishan Khanna

12 snips

Jun 26, 2023

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Flags Enable Tests, Tests Require Flags

A/B testing is the data collection and analysis process, while feature flags are the technical mechanism to expose control and variant groups.
Feature flags without measurement are just gating; measurement makes them experiments.

ADVICE

Match Sample Size To Effect Size

Only run experiments when you can reach sufficient sample size for the effect size you want to detect.
Smaller expected changes need much larger samples or longer runtimes to reach statistical power.

ADVICE

Use Flags As Kill Switches

Feature flag everything you deploy to give yourself a kill switch and reduce release risk.
Flags let you decouple rollout from app releases and quickly disable bad changes in production.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

In this edition of Fragmented, we're thrilled to host Ishan Khanna, a software engineer at Tinder who possesses great enthusiasm for feature flags and A/B testing. Donn discusses why he invited Ishan on the show, highlighting Ishan's passion for feature flagging and A/B testing. The conversation kicks off with an insightful story from Ishan about feature flagging at Booking.com, leading to a discussion on the difference between A/B Testing and Feature Flags, when and why to introduce feature flagging, and how to measure its effectiveness. The show also focuses on the benefits and risks of feature flagging, along with ways to manage potential complexities in the codebase.

We then delve deeper into the topic of feature flagging, covering how to get started, what to look for in a tool, and the role of testing. Discussion points include the best practices for rollout percentages, considerations for multi-platform implementation, and the specifics of targeting in feature flagging. The conversation wraps up with an exploration of available tools for those looking to introduce feature flagging or A/B testing frameworks into their operations, examining when it might be necessary to build a bespoke solution.

The episode offers a wealth of resources for listeners, including links to an array of feature flagging and A/B testing tools, such as Firebase Remote Config, Optimizely, and LaunchDarkly. For more insight into the topics discussed, Ishan recommends his Droidcon Berlin talk on 'Customer Driven Development' and Stuart Frisby’s talk on A/B Testing. To reach out to Ishan, listeners can contact him via Twitter, LinkedIn, or his website.

Links

Here are the links mentioned in the document, in markdown format:

Contact Ishan

Donn's Git Course

Need to learn Git? Donn has the course for you. In this FREE course you'll learn everything you need to know in order to start working with Git everyday. Watch it here.

AndroidJobs.IO

Job postings are FREE on AndroidJobs.IO 🎉
Sign up to get notified of new jobs on a weekly basis as well.
AndroidJobs.IO

Software Freelancing

Contact

@fragmentedcast on Twitter or our YouTube channel

Donn

Kaushik

kau.sh (has links to all my networks)
twitter.kau.sh
mastodon.kau.sh
youtube.kau.sh (on YouTube)

Disclaimer: Many of the links we share to products are affiliate links. They help support the production of Fragmented. Thank you for your support.