BI 184 Peter Stratton: Synthesize Neural Principles

14 snips

Feb 20, 2024

The podcast discusses synthesizing neural principles for better AI, focusing on a 'sideways-in' approach for computational brains. It explores integrating diverse brain operations, the challenges in achieving general-purpose AI, advancements in robotics inspired by biological principles, and the complexities of spiking neural networks for artificial general intelligence.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Tiny Brains, Big Lessons

Small brains (flies, bees) achieve rich behavior with far fewer neurons than modern AI models suggest is needed.
Studying those brains can reveal compact computational principles useful for AI design.

INSIGHT

Scale Over Elegance

Deep learning's success largely comes from massive scale of models, data, and compute rather than fundamentally efficient algorithms.
Gradient descent remains powerful but is likely inefficient compared with biological learning principles.

INSIGHT

Embodiment Reveals Missing Computation

Robotics exposes gaps AI models face because embodied control requires closed-loop, dynamic sensing and action.
Understanding movement and body coupling is essential to scale intelligence beyond static datasets.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Support the show to get full episodes, full archive, and join the Discord community.

Peter Stratton is a research scientist at Queensland University of Technology.

I was pointed toward Pete by a patreon supporter, who sent me a sort of perspective piece Pete wrote that is the main focus of our conversation, although we also talk about some of his work in particular - for example, he works with spiking neural networks, like my last guest, Dan Goodman.

What Pete argues for is what he calls a sideways-in approach. So a bottom-up approach is to build things like we find them in the brain, put them together, and voila, we'll get cognition. A top-down approach, the current approach in AI, is to train a system to perform a task, give it some algorithms to run, and fiddle with the architecture and lower level details until you pass your favorite benchmark test. Pete is focused more on the principles of computation brains employ that current AI doesn't. If you're familiar with David Marr, this is akin to his so-called "algorithmic level", but it's between that and the "implementation level", I'd say. Because Pete is focused on the synthesis of different kinds of brain operations - how they intermingle to perform computations and produce emergent properties. So he thinks more like a systems neuroscientist in that respect. Figuring that out is figuring out how to make better AI, Pete says. So we discuss a handful of those principles, all through the lens of how challenging a task it is to synthesize multiple principles into a coherent functioning whole (as opposed to a collection of parts). Buy, hey, evolution did it, so I'm sure we can, too, right?

0:00 - Intro 3:50 - AI background, neuroscience principles 8:00 - Overall view of modern AI 14:14 - Moravec's paradox and robotics 20:50 -Understanding movement to understand cognition 30:01 - How close are we to understanding brains/minds? 32:17 - Pete's goal 34:43 - Principles from neuroscience to build AI 42:39 - Levels of abstraction and implementation 49:57 - Mental disorders and robustness 55:58 - Function vs. implementation 1:04:04 - Spiking networks 1:07:57 - The roadmap 1:19:10 - AGI 1:23:48 - The terms AGI and AI 1:26:12 - Consciousness