Enfeeblement Risk: Humans as Reward Selectors

What makes something truly *intelligent?* Is a rock an agent? Could a perfect simulation of your brain actually *be* you? In this fascinating conversation, Dr. Jeff Beck takes us on a journey through the philosophical and technical foundations of agency, intelligence, and the future of AI.

Jeff doesn't hold back on the big questions. He argues that from a purely mathematical perspective, there's no structural difference between an agent and a rock – both execute policies that map inputs to outputs. The real distinction lies in *sophistication* – how complex are the internal computations? Does the system engage in planning and counterfactual reasoning, or is it just a lookup table that happens to give the right answers?

*Key topics explored in this conversation:*

*The Black Box Problem of Agency* – How can we tell if something is truly planning versus just executing a pre-computed response? Jeff explains why this question is nearly impossible to answer from the outside, and why the best we can do is ask which model gives us the simplest explanation.

*Energy-Based Models Explained* – A masterclass on how EBMs differ from standard neural networks. The key insight: traditional networks only optimize weights, while energy-based models optimize *both* weights and internal states – a subtle but profound distinction that connects to Bayesian inference.

*Why Your Brain Might Have Evolved from Your Nose* – One of the most surprising moments in the conversation. Jeff proposes that the complex, non-smooth nature of olfactory space may have driven the evolution of our associative cortex and planning abilities.

*The JEPA Revolution* – A deep dive into Yann LeCun's Joint Embedding Prediction Architecture and why learning in latent space (rather than predicting every pixel) might be the key to more robust AI representations.

*AI Safety Without Skynet Fears* – Jeff takes a refreshingly grounded stance on AI risk. He's less worried about rogue superintelligences and more concerned about humans becoming "reward function selectors" – couch potatoes who just approve or reject AI outputs. His proposed solution? Use inverse reinforcement learning to derive AI goals from observed human behavior, then make *small* perturbations rather than naive commands like "end world hunger."

Whether you're interested in the philosophy of mind, the technical details of modern machine learning, or just want to understand what makes intelligence *tick,* this conversation delivers insights you won't find anywhere else.

---

TIMESTAMPS:

00:00:00 Geometric Deep Learning & Physical Symmetries

00:00:56 Defining Agency: From Rocks to Planning

00:05:25 The Black Box Problem & Counterfactuals

00:08:45 Simulated Agency vs. Physical Reality

00:12:55 Energy-Based Models & Test-Time Training

00:17:30 Bayesian Inference & Free Energy

00:20:07 JEPA, Latent Space, & Non-Contrastive Learning

00:27:07 Evolution of Intelligence & Modular Brains

00:34:00 Scientific Discovery & Automated Experimentation

00:38:04 AI Safety, Enfeeblement & The Future of Work

---

REFERENCES:

Concept:

[00:00:58] Free Energy Principle (FEP)

https://en.wikipedia.org/wiki/Free_energy_principle

[00:06:00] Monte Carlo Tree Search

https://en.wikipedia.org/wiki/Monte_Carlo_tree_search

Book:

[00:09:00] The Intentional Stance

https://mitpress.mit.edu/9780262540537/the-intentional-stance/

Paper:

[00:13:00] A Tutorial on Energy-Based Learning (LeCun 2006)

http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf

[00:15:00] Auto-Encoding Variational Bayes (VAE)

https://arxiv.org/abs/1312.6114

[00:20:15] JEPA (Joint Embedding Prediction Architecture)

https://openreview.net/forum?id=BZ5a1r-kVsf

[00:22:30] The Wake-Sleep Algorithm

https://www.cs.toronto.edu/~hinton/absps/ws.pdf

<trunc, see rescript>

---

RESCRIPT:

https://app.rescript.info/public/share/DJlSbJ_Qx080q315tWaqMWn3PixCQsOcM4Kf1IW9_Eo

PDF:

https://app.rescript.info/api/public/sessions/0efec296b9b6e905/pdf