AI doesn't do anything. We do. | Rumman Chowdhury on reclaiming agency and rejecting "moral outsourcing"

37 snips

Apr 28, 2026

Rumman Chowdhury, an ethical AI leader and founder of Human Intelligence PBC, reminds us that people—not algorithms—make choices. She discusses moral outsourcing, bias bounties and red teaming, how simple prompts break guardrails, why benchmarks mislead, and ways builders can reclaim agency with better evaluation, legal protections, and agentic AI.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

ANECDOTE

Bias Bounties Started From A Twitter Hackathon

Rumman built a nonprofit bias-bounty program after hosting Twitter's first algorithmic bias bounty and being laid off post-Musk takeover.
The program let non-programmers test models and show how user feedback should feed design.

INSIGHT

Grandma Hack Reveals Guardrail Mechanics

Hackers and ordinary users reveal the same failure mechanisms: manipulating a model's 'helpful, harmless, honest' goals.
The 'grandma hack' shows framing content as harmless can bypass guardrails and produce dangerous outputs.

ADVICE

Do Assurance Before Running Benchmarks

Treat assurance as product work, not just legal audit; define success first, then pick tests.
Rumman warns companies often put benchmarks before design, causing unpredictable agent behavior in production.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Rumman Chowdhury wants to remind you that “AI isn't doing anything.” We do things. AI is not to blame for layoffs or if you’re denied medical coverage. People are.

Eight years ago, Rumman coined the term “moral outsourcing” to describe this excuse where we blame tech for decisions that people make. Why do the semantics matter? Because, Rumman says:

In world one where, “AI did X,” it's very scary. It's like, “oh my gosh, this thing that is bigger and smarter than me has come and descended and now it's gonna wipe out every job. “ [But if we center on people, then we have agency and accountability and we can say] “no, you built a thing that was broken and flawed.”

Rumman is the founder and CEO of Human Intelligence PBC, which is building evaluation infrastructure to make Gen AI systems safe, trustworthy, and compliant. She also served as the U.S. Science Envoy for Artificial Intelligence under the Biden administration, led AI ethics teams at Twitter and Accenture, and is a Responsible AI Fellow at Harvard.

In this conversation:

Why "moral outsourcing" is the sneakiest trick in tech — and how execs use AI as a shield for decisions humans made
How to avoid — or at least how to mitigate — creating AI that’s biased
Red teaming AI and creating bias bounties
The "grandma hack" and other ways regular people accidentally jailbreak AI models
How AI companies are quietly rewriting their terms of service to dodge liability when things go wrong
Why the benchmarks you see when a new model drops are "basically spelling tests"
AI psychosis, parasocial chatbots, and the cold emails Rumman gets once a month from people who think AI is alive
What builders can do right now to take back agency — and why Rumman is more excited about agentic AI than anything that came before

Chapters:

(00:00) - "The thing I believe in the most is human agency"
(02:14) - Why builders have more agency than they realize
(04:00) - What is a bias bounty?
(06:41) - What 2,000 hackers at DEF CON found
(09:40) - The grandma hack
(11:30) - Why guardrails fall apart
(14:54) - Anthropic's new bug-finding model and the cat-and-mouse game
(19:10) - Why most evals are "basically spelling tests"
(21:30) - How to actually evaluate an AI agent
(27:16) - "Moral outsourcing" and the AI layoff lie
(29:41) - Inside Rumman's tenure as U.S. AI Science Envoy
(33:06) - The legal loophole AI companies use to dodge liability
(36:31) - AI psychosis and the cold emails Rumman gets
(39:36) - Why Google's AI overview is quietly dangerous
(45:31) - The problem with "AI literacy"
(49:01) - Can we trust anything we see anymore?
(51:11) - What builders can do right now to take back agency

Support Future Around & Find Out

Follow Dan on LinkedIn
Get the free newsletter
Become a paid subscriber and help future proof FAFO!