When Machines Can Code - Reflections on Trusting Trust by Ken Thompson + Coding Machines by Lawrence Kesteloot

Feb 16, 2026

They compare Ken Thompson's compiler backdoor concept with a fictional self-evolving worm that infects tooling. They trace how malicious compilers could hide and persist across builds. They explore how machine learning and tooling can learn from human fixes and become stealthy. They debate trust, ethics, AI intelligence, and practical guardrails for using code-generating tools.

01:15:41

forum

Ask episode

web_stories

AI Snips

view_agenda

Chapters

menu_book

Books

auto_awesome

Transcript

info_circle

Episode notes

insights

INSIGHT

Source Code Isn’t Sufficient For Trust

Ken Thompson showed that compiled binaries can hide perpetual backdoors that source review won't reveal.
Trust ultimately rests on the people building tools, not just readable source code.

volunteer_activism

ADVICE

Harden Trust With Provenance And Guardrails

Always question how systems could be tricked and design guardrails accordingly.
Use cryptographic signing and rigorous provenance to raise the bar against supply-chain compromises.

insights

INSIGHT

Compilers Can Perpetuate Invisible Backdoors

A modified compiler can insert a backdoor into programs and then re-infect future compiler builds.
The malicious behavior can vanish from source but persist in compiled binaries indefinitely.

Get the Snipd Podcast app to discover more snips from this episode

Why read Thompson and Kesteloot together

02:13 • 1min

chevron_right

Ken Thompson background and context

03:23 • 1min

chevron_right

Lawrence Kesteloot and Coding Machines

04:50 • 2min

chevron_right

Initial reactions and modern relevance

07:17 • 5min

chevron_right

Explaining Thompson's Trojan compiler idea

How a self-perpetuating compiler works

15:31 • 3min

chevron_right

Real-world verification and history

18:54 • 2min

chevron_right

Thompson's moral: trust the people

21:23 • 3min

chevron_right

Coding Machines plot and discovery

24:10 • 3min

chevron_right

Machines learning from human patches

26:50 • 3min

chevron_right

The story's unsettling conclusion

29:22 • 2min

chevron_right

Trust, ethics, and responsibility

31:17 • 2min

chevron_right

Dark patterns and emergent behaviors

Are large language models intelligent?

40:16 • 3min

chevron_right

Illusionist analogy for AI

43:40 • 7min

chevron_right

Practical AI use and tooling tradeoffs

Stuxnet and unintended consequences

57:24 • 5min

chevron_right

Balancing AI adoption and caution

01:02:08 • 3min

chevron_right

Silicon Valley motives and ethics

01:05:17 • 3min

chevron_right

Vibe coding and cultural risks

01:08:21 • 2min

chevron_right

Practical takeaways and experiments

What we'll do differently

01:14:51 • 1min

chevron_right

Recommendations and closing thoughts

Reflections on Trusting Trust

• Mentioned in 26 episodes

Designing Data-Intensive Applications

The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

Martin Kleppman

This book helps software engineers and architects understand the pros and cons of various technologies for storing and processing data. It covers fundamental principles, trade-offs, and design decisions in data systems, including scalability, consistency, reliability, efficiency, and maintainability. The book delves into distributed systems research, the architecture of data systems, and how to make informed decisions about different tools and technologies. It does not provide detailed instructions on specific software packages but focuses on the underlying principles and trade-offs essential for designing data-intensive applications.

#3384

• Mentioned in 14 episodes

The Unicorn Project

Gene Kim

The Unicorn Project is a fictionalized story about a DevOps transformation taking place at Parts Unlimited, the same company featured in 'The Phoenix Project'. The book follows Maxine, a senior lead developer and architect, who is exiled to the Phoenix Project after contributing to a payroll outage. She joins a group of corporate rebels aiming to overthrow the existing bureaucratic order, liberate developers, bring joy back to technology work, and enable the business to win in a time of digital disruption. The novel introduces the five ideals of Locality and Simplicity; Focus, Flow and Joy; Improvement of Daily Work; Psychological Safety; and Customer Focus, which are crucial for creating an environment that fosters business innovation and productivity.

#768

• Mentioned in 45 episodes

The Phoenix project

Lambeth London Borough Council

In this novel, Gene Kim, Kevin Behr, and George Spafford tell the story of Bill, an IT manager at Parts Unlimited, who is tasked with turning around the company's failing IT department. The book delves into the challenges of IT management, the importance of DevOps practices, and how these practices can lead to significant improvements in efficiency, reliability, and customer satisfaction.

#8808

• Mentioned in 6 episodes

UNIX

A History and a Memoir

Brian Kernighan

Brian Kernighan's "Unix: A History and a Memoir" offers a personal and insightful account of the development of the Unix operating system. The book details the unique environment at Bell Labs that fostered innovation and collaboration, highlighting the contributions of key figures like Ken Thompson and Dennis Ritchie. Kernighan shares anecdotes and reflections on the design principles and cultural factors that shaped Unix's success. The book explores the evolution of Unix, its impact on the computing world, and its enduring legacy. It provides a valuable historical perspective on a pivotal technology and its lasting influence.

In this episode of Book Overflow, Carter and Nathan discuss Ken Thompson's essay Reflections on Trusting Trust and the short story Coding Machines by Lawrence Kesteloot!

Big thanks to user 0b00101010 from the Book Overflow Discord for the recommendation! Join it here: https://discord.gg/ZwS2fqW7ZZ

-- Want to talk with Carter or Nathan? Book a coaching session! --

----------------------------------------------------------

Carter

https://www.joinleland.com/coach/carter-m-1

Nathan

https://www.joinleland.com/coach/nathan-t-2

-- Books Mentioned in this Episode --

Note: As an Amazon Associate, we earn from qualifying purchases.

----------------------------------------------------------

Reflections on Trusting Trust by Ken Thompson

https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf

Coding Machines by Lawrence Kesteloot

https://www.teamten.com/lawrence/writings/coding-machines/

----------------

00:00 Intro

02:55 About the Authors and Essays

06:14 Initial Thoughts

11:03 The Trusting Trust Attack

22:39 Coding Machines

32:05 AI Trust and Dark Patterns

44:01 AI-Generated Code and the Future

54:52 Stuxnet and Unintended Consequences

1:02:43 Final Thoughts

Spotify: https://open.spotify.com/show/5kj6DLCEWR5nHShlSYJI5L

Apple Podcasts: https://podcasts.apple.com/us/podcast/book-overflow/id1745257325

X: https://x.com/bookoverflowpod

Carter on X: https://x.com/cartermorgan

Nathan's Functionally Imperative: www.functionallyimperative.com

----------------

Book Overflow is a podcast for software engineers, by software engineers dedicated to improving our craft by reading the best technical books in the world. Join Carter Morgan and Nathan Toups as they read and discuss a new technical book each week!

The full book schedule and links to every major podcast player can be found at https://www.bookoverflow.io

Home Top podcasts Popular guests Top books