Pelican Benchmarks Reveal Model Quality

Simon Willison is a prolific independent software developer, a blogger, and one of the most visible and trusted voices on the impact AI is having on builders. He co-created Django, the web framework that powers Instagram, Pinterest, and tens of thousands of other websites. He coined the term “prompt injection,” popularized the terms “AI slop” and “agentic engineering,” and has built over 100 open source projects, including Datasette, a data analysis tool used by investigative journalists worldwide. What makes Simon unique is that he’s made the leap from traditional software engineering to AI-native development more fully and visibly than almost anyone—and he’s been documenting everything he learns in real time on his blog, SimonWillison.net.

In our in-depth conversation, Simon shares:

1. Why November 2025 was the inflection point when AI coding agents crossed from “mostly works” to “actually works”

2. How Simon writes 95% of his code from his phone now and why he’s mentally exhausted by 11 a.m.

3. Why mid-career engineers (not juniors) are most at risk right now

4. The three agentic engineering patterns Simon uses daily (red/green TDD, templates, hoarding)

5. The next leap: the “dark factory” pattern where nobody writes or reviews code and AI does its own QA

6. Why prompt injection is an unsolved security problem and the “lethal trifecta” that will likely lead to an AI Challenger disaster

7. Why the pelican riding a bicycle became the unofficial benchmark for AI model quality

—

Brought to you by:

WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs

Vanta—automate compliance, manage risk, and accelerate trust with AI

—

Episode transcript: https://www.lennysnewsletter.com/p/an-ai-state-of-the-union

—

Archive of all Lenny's Podcast transcripts: https://www.dropbox.com/scl/fo/yxi4s2w998p1gvtpu4193/AMdNPR8AOw0lMklwtnC0TrQ?rlkey=j06x0nipoti519e0xgm23zsn9&st=ahz0fj11&dl=0

—

Where to find Simon Willison:

• X: https://x.com/simonw

• LinkedIn: https://www.linkedin.com/in/simonwillison

• Website: https://simonwillison.net

• Agentic Engineering Patterns: https://simonwillison.net/guides/agentic-engineering-patterns

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Introduction to Simon Willison

(02:40) The November 2025 inflection point

(08:01) What’s possible now with AI coding

(10:42) Vibe coding vs. agentic engineering

(13:57) The dark-factory pattern

(20:41) Where bottlenecks have shifted

(23:36) Where human brains will continue to be valuable

(25:32) Defending of software engineers

(29:12) Why experienced engineers get better results

(30:48) Advice for avoiding the permanent underclass

(33:52) Leaning into AI to amplify your skills

(35:12) Why Simon says he’s working harder than ever

(37:23) The market for pre-2022 human-written code

(40:01) Prediction: 50% of engineers writing 95% AI code by the end of 2026

(44:34) The impact of cheap code

(48:27) Simon’s AI stack

(54:08) Using AI for research

(55:12) The pelican-riding-a-bicycle benchmark

(59:01) The inherent ridiculousness of AI

(1:00:52) Hoarding things you know how to do

(1:08:21) Red/green TDD pattern for better AI code

(1:14:43) Starting projects with good templates

(1:16:31) The lethal trifecta and prompt injection

(1:21:53) Why 97% effectiveness is a failing grade

(1:25:19) The normalization of deviance

(1:28:32) OpenClaw: the security nightmare everyone is looking past

(1:34:22) What’s next for Simon

(1:36:47) Zero-deliverable consulting

(1:38:05) Good news about Kakapo parrots

—

References: https://www.lennysnewsletter.com/p/an-ai-state-of-the-union

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

—

Lenny may be an investor in the companies discussed.