Prompting Strategies and Holding Models Accountable

Ryan Nystrom is a software engineer at Notion. He joined in December 2024 after Notion acquired Campsite, the team communication platform he co-founded with Brian Lovin. At Notion, he’s been a core builder of Notion AI and the Custom Agents feature launched in February 2026. He manages a team of six to seven engineers while still writing code himself, currently running Project Afterburner, a push to cut Notion’s CI time to a quarter of its current duration.

What you’ll learn:

1. How to build a Notion AI custom agent that auto-generates your daily standup pre-read by pulling from Slack, GitHub, Honeycomb metrics, and yesterday’s meeting transcript
2. How to configure subagents and MCP integrations within Notion AI
3. How Notion’s internal “Boxy” system lets engineers @mention Codex from within Notion comments and get a full pull request with screenshots in 20 minutes
4. The spec-first development workflow: dictate an idea into Whisper, have Codex format it as a proper spec, commit it to the repo, and let the agent implement and verify it autonomously
5. Why fast CI is absolutely critical in the age of AI coding agents
6. How to prompt AI coding agents to defend their reasoning under pushback
7. Why engineering managers and even senior executives should keep writing code

—

Brought to you by:

WorkOS—Make your app enterprise-ready today

Orkes—The enterprise platform for reliable applications and agentic workflows

—

In this episode, we cover:

(00:00) Introduction to Ryan Nystrom

(02:48) How AI has upended 12+ years of the same working routine

(04:30) Project Afterburner: Notion’s push to cut CI time to a quarter

(09:00) Why high-frequency, high-quality meetings beat lower-frequency standups

(11:10) How automated context surfaces every engineer’s work equally

(12:15) Why cutting meeting prep is a burnout protection mechanism

(14:26) The case for engineering managers writing code

(16:13) Inside “Boxy”: Notion’s internal VM-based background agent system

(20:30) Old World vs. New World code review

(24:51) Prompting Codex from Notion comments

(29:20) The emotions around code review

(31:01) Quick recap

(32:00) Spec-first development: writing and checking agent specs into the repo

(35:10) The spec as changelog: version control for how a feature actually works

(37:53) How engineers’ roles are evolving

(39:00) Lightning round

(45:21) Where to find Ryan

—

Tools referenced:

• Notion AI: https://www.notion.com/product/ai

• Notion Custom Agents: https://www.notion.com/blog/introducing-custom-agents

• Codex (OpenAI): https://openai.com/codex

• Claude Code (Anthropic): https://claude.ai/code

• Honeycomb (observability + MCP): https://www.honeycomb.io

• Whisper (OpenAI voice transcription): https://openai.com/research/whisper

• Slack: https://slack.com

• GitHub: https://github.com

—

Other references:

• How Stripe built “minions”—AI coding agents that ship 1,300 PRs weekly from Slack reactions | Steve Kaliski (Stripe): https://www.chatprd.ai/how-i-ai/stripes-ai-minions-ship-1300-prs-weekly-from-a-slack-emoji

• Notion 3.3 Custom Agents launch (February 24, 2026): https://www.notion.com/releases/2026-02-24

—

Where to find Ryan Nystrom:

X: https://x.com/ryannystrom

LinkedIn: https://www.linkedin.com/in/ryannystrom/

GitHub: https://github.com/rnystrom

—

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.