From Orphanage to Google Brain: David Sussillo on Heroin, Neural Networks and the Mysteries of the Heart
Keen On America
Inherited Traits and Navigational Skills
Sussillo reflects on traits from his extended family that helped him navigate group home life and later success.
“I can point to things. But is that a systemic explanation? I think there the answer is a little less clear. I mean, surely people need love and all of that, but then there’s this risk of just devolving into platitude.” — David Sussillo
David Sussillo is a big time neural reverse engineer. The Stanford brain scientist worked at Google Brain with Geoffrey Hinton, and now is at Meta Reality Labs. What distinguishes Sussillo, however, is not his Silicon Valley good luck, but the bad luck of his origins. In his memoir, Emergent: A Memoir of Boyhood, Computation, and the Mysteries of the Mind, Sussillo begins at the Albuquerque Christian Children’s Home — a modern-day orphanage — and the Milton Hershey School, the boarding school endowed by the chocolate magnate for kids with nowhere else to go. Both his parents were addicts. His mom died young. His dad spent his life as an untrained preacher ministering to homeless people on the streets of Albuquerque while managing a lifelong heroin habit.
The book’s thesis borrows from the science he studies: “emergence” — simple things interacting to produce complex behaviour that none of them could produce alone. His life is both proof of and a challenge to this concept. He made it out. Most of the kids he grew up with didn’t. He can point to moments — a gifted-and-talented test in third grade, an aunt and uncle’s intervention at nine, a first love in college — but he can’t build an explanatory system from these haphazard events. The Sussillo quilt doesn’t have an innate pattern. It just has patches.
What makes Sussillo unusual as a memoirist is his refusal to sentimentalise. Twenty years of psychotherapy, he confesses, has taught him something most authors never learn: that understanding your own story doesn’t mean you’ve explained it. His science can’t explain his childhood either. “The big dirty secret of neuroscience,” he says, “is that we don’t really understand much in the ways that people would love us to understand.” The man who reverse-engineers neural networks can’t reverse-engineer himself.
I asked him whether having children would have been harder than writing the book. Yes, he said. With the book, you can take a break. With kids, you relive things through a very specific way of relating. He and his wife chose not to. His mentors all told him he’d have been great at it. He’s not so sure. That honesty — the willingness to say “I don’t know” and mean it — runs through everything Sussillo does. He says he’s happy, claiming to have found peace with his past. But he still carries the baggage. Who wouldn’t? He’s just learned to manage it. Emergent, not emerged.
Five Takeaways
• From Orphanage to Google Brain: Both parents were heroin addicts. Sussillo grew up in a modern-day orphanage in Albuquerque and then the Milton Hershey School. He went on to work at Google Brain with Geoffrey Hinton, now works at Meta Reality Labs, teaches at Stanford. Most of the kids he grew up with didn’t make it.
• Emergence as Autobiography: The book’s thesis borrows from the science he studies: simple pieces combining into complicated outcomes. His life is the proof of concept and the counter-example simultaneously. The quilt doesn’t have a pattern. It just has patches.
• The Dirty Secret of Neuroscience: The man who reverse-engineers neural networks can’t reverse-engineer himself. “We don’t really understand much in the ways that people would love us to understand.” Twenty years of therapy taught him more than the science.
• Would Kids Have Been Harder Than the Book? Yes. With the book, you can take a break. With kids, you relive trauma through a very specific way of relating. He and his wife chose not to have children. His mentors told him he’d have been great at it. He’s not so sure.
• Emergent, Not Emerged: Sussillo has found peace with his past. He’s happy. He still carries the baggage from his childhood. He’s just learned how to manage it. The emergence is ongoing.
About the Guest
David Sussillo is a research scientist at Meta Reality Labs and a consulting professor at Stanford University. He previously worked at Google Brain. His memoir is Emergent: A Memoir of Boyhood, Computation, and the Mysteries of the Mind. He grew up in the Albuquerque Christian Children’s Home and the Milton Hershey School. He lives in New Mexico.
References:
• Emergent: A Memoir of Boyhood, Computation, and the Mysteries of the Mind by David Sussillo — the book under discussion.
• The Albuquerque Christian Children’s Home — the group home where Sussillo spent five years of his childhood.
• The Milton Hershey School — founded in 1906 by the Hershey chocolate magnate for children with nowhere else to go. Sussillo spent four years there.
• Google Brain — the lab where Sussillo worked alongside Geoffrey Hinton on the neural network research that became the foundation of modern AI.
• John Conway’s Game of Life — the cellular automaton simulation Sussillo cites as an early example of emergence: complicated outcomes from simple rules.
About Keen On America
Nobody asks more awkward questions than the Anglo-American writer and filmmaker Andrew Keen. In Keen On America, Andrew brings his pointed Transatlantic wit to making sense of the United States — hosting daily interviews about the history and future of this now venerable Republic. With nearly 2,800 episodes since the show launched on TechCrunch in 2010, Keen On America is the most prolific intellectual interview show in the history of podcasting.
Chapters:
- (00:00) - Introduction
- (01:30) - The Albuquerque Christian Children’s Home and Milton Hershey School
- (03:30) - Why write a memoir? Five years and twenty years of therapy
- (05:00) - Heroin-addicted parents: the origin story
- (08:00) - A father as untrained preacher on the streets of Albuquerque
- (10:00) - Which parent had more impact?
- (12:00) - The gifted-and-talented test that changed everything
- (15:00) - From Milton Hershey to Carnegie Mellon: the jump
- (18:00) - Life falls apart at 23: panic attacks and psychotherapy
- (21:00) - Neural networks, Google Brain, and the dirty secret of neuroscience
- (25:00) - Would having kids have been harder than writing the book?
- (28:00) - The Albanian friend and the beach: what America gets right
- (31:00) - Silicon...
The AI-powered Podcast Player
