EP 192 David Krakauer on Science, Complexity and AI

27 snips

Jul 18, 2023

Guest

David Krakauer

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Super‑Human Models And The Uncanny Valley

Krakauer coins 'super-human' models that cross a statistical uncanny valley to generalize in ultra-high dimensions.
These models exploit high-dimensional regularities inaccessible to simple compressive descriptions.

INSIGHT

Huge Data Gap Between Humans And LLMs

Krakauer contrasts human training (1.5 MB) with LLMs (hundreds of GB) to highlight sample inefficiency.
This gap reframes arguments like Chomsky's poverty of the stimulus.

ANECDOTE

From Neural Surrogates To New Laws

Krakauer describes Miles Cranmer using graph neural nets then symbolic regression to derive new equations for dark energy.
This pipeline turned neural surrogates into parsimonious scientific laws.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Jim has a wide-ranging talk with David Krakauer about the ideas in his forthcoming paper "The Structure of Complexity in Machine Learning Science" and how AI may alter the course of science. They discuss data-driven science vs theory-driven science, a bifurcation in science, the protein folding problem, brute force methods, the origin of induction in David Hume, the origin of neural networks in deductive thinking of the '40s, super-Humean models, crossing the statistical uncanny valley, ultra-high-dimensionality, adaptive computation, why genetic algorithms might come back, Chomsky's poverty of the stimulus, the lottery ticket hypothesis, neural nets as pre-processors for parsimonious science, how human expertise constrains model-building, GPT-4's arithmetic problem, cognitive synergy, why LLMs aren't AGIs, incompressible representations, gravitational lensing, the new sciences LLMs will lead to, encoding adaptive history, Jim's ScriptWriter software, discovery engines vs libraries vs synthesizers, the history of science as a history of constraint, Occam's razor & meta-Occam, assembly theory, whether existential risk is a marketing ploy, the Idiocracy risk, using empirical precedent in tech regulation, networks of info agents, the outsourcing of human judgment, and much more. Episode Transcript JRS EP10 - David Krakauer: Complexity Science Darwin's Dangerous Idea: Evolution and the Meanings of Life, by Daniel Dennett JRS Currents 100: Sara Walker and Lee Cronin on Time as an Object David Krakauer's research explores the evolution of intelligence and stupidity on Earth. This includes studying the evolution of genetic, neural, linguistic, social, and cultural mechanisms supporting memory and information processing, and exploring their shared properties. President of the Santa Fe Institute since 2015, he served previously as the founding director of the Wisconsin Institutes for Discovery, the co-director of the Center for Complexity and Collective Computation, and professor of mathematical genetics, all at the University of Wisconsin, Madison.