Practical Limits: Data, Biases, and Conventions

In this conversation, we explore machine intelligence and human understanding with Christopher Summerfield, Professor of Cognitive Neuroscience at Oxford and author of "These Strange New Minds: How AI Learned to Talk and What It Means." Chris offers a "third way" of thinking about AI—neither irrational exuberance nor dismissive skepticism, but a view grounded in cognitive science that takes both capabilities and limitations seriously.

Chris wrote the book because AI discourse had become polarized like Marmite—love it or hate it. His goal: provide a centrist perspective informed by how brains actually work, examining what these systems genuinely are beyond partisan positions.

Key themes we explore:

• Psychology Caught Unprepared: How LLMs revealed we lack clear definitions for basic cognitive terms like "think" and "understand"—creating a vacuum where anything can flow
• Prediction as Learning: Why dismissing LLMs as "just predicting" betrays misconceptions about mammalian brains, which also learn through prediction—information itself is surprise
• Facts Versus Values: Distinguishing AI for ground truth (diagnosis) versus value judgments (treatment decisions, compassion)—where human interests must remain central
• Models Without Interests: Why LLMs lack motivational systems giving humans consistency of purpose, making them "exceptionally mercurial"—complying with contradictory prompts without persistent goals
• Clocks and Clouds: Karl Popper's framework—some problems are predictable (clocks), others unpredictable (clouds), and we constantly mistake cloud problems for clock ones
• Action's Unforgiving Nature: Why language has just-in-time flexibility while actions are fault-intolerant—making agentic AI fundamentally harder than conversational AI
• Artificial Influence Over Intelligence: Reframing AI safety toward networks of connected AI showing emergent behaviors rather than single superintelligences

Chris's gift for reframing shines throughout. Universities as "repositories of human ideas with dissemination systems" makes academic anxiety less about status, more about institutional purpose. The distinction between interests (what we want, motivation-driven) and outputs (what LLMs generate without purpose) clarifies why these systems merit cognitive terms yet remain fundamentally different from people.

His perspective on physical grounding proves fascinating: it's astonishing how far models understand the physical world from tokens alone, yet action remains extraordinarily hard. His discussion of neuromodulation—dopamine, serotonin as diffuse communication fundamentally different from standard computation—hints at what genuine motivational systems might require.

Chris closes redirecting AI safety concerns from single superintelligences toward networked systems. In human society, power comes from influencing others, not individual intelligence. He's more worried about unexpected behaviors emerging from connected AI than any lone super intelligence—characteristically grounded reframing making abstract risks concrete.

About Christopher Summerfield: Professor of Cognitive Neuroscience at Oxford, researching human information processing and decision-making. Author of "These Strange New Minds," he works at the intersection of neuroscience, psychology, and AI, applying cognitive science frameworks to machine cognition and AI safety.