Effective Altruism: Ten Global Problems – 80,000 Hours (October 2021) Four: Brian Christian on artificial intelligence
02:54:28
Kids Over-Imitate On Purpose
- Children over-imitate even unnecessary actions when they infer a demonstrator had a reason.
- Three-year-olds copy pointless steps because they assume hidden causal reasons.
Self-Imitation Powers AlphaGo Zero
- AlphaGo Zero learned policy by imitating its own deliberative search outcomes, creating a feedback loop.
- Iterated distillation and amplification let systems surpass human training data.
Infer Goals Instead Of Actions
- Inverse reinforcement learning (IRL) infers a user's reward function from observed behavior.
- IRL can teach goals humans can't demonstrate, then optimize behavior to achieve them.
Get the Snipd Podcast app to discover more snips from this episode
Get the app 1 chevron_right 2 chevron_right 3 chevron_right 4 chevron_right 5 chevron_right 6 chevron_right 7 chevron_right 8 chevron_right 9 chevron_right 10 chevron_right 11 chevron_right 12 chevron_right 13 chevron_right 14 chevron_right 15 chevron_right 16 chevron_right 17 chevron_right 18 chevron_right 19 chevron_right 20 chevron_right 21 chevron_right 22 chevron_right 23 chevron_right 24 chevron_right 25 chevron_right 26 chevron_right 27 chevron_right 28 chevron_right 29 chevron_right 30 chevron_right 31 chevron_right 32 chevron_right 33 chevron_right 34 chevron_right 35 chevron_right 36 chevron_right 37 chevron_right 38 chevron_right 39 chevron_right 40 chevron_right 41 chevron_right 42 chevron_right 43 chevron_right 44 chevron_right 45 chevron_right 46 chevron_right 47 chevron_right 48 chevron_right 49 chevron_right 50 chevron_right 51 chevron_right 52 chevron_right 53 chevron_right 54 chevron_right 55 chevron_right 56 chevron_right 57 chevron_right 58 chevron_right 59 chevron_right 60 chevron_right 61 chevron_right 62 chevron_right 63 chevron_right 64 chevron_right 65 chevron_right 66 chevron_right 67 chevron_right
Introduction
00:00 • 2min
The Alliment Problem
01:42 • 2min
Ai Safety and Machine Learning Ethics - A Review
03:36 • 4min
Is the Future Going to Be Good?
07:14 • 2min
GPT
09:21 • 2min
Using a G I in the Cloud?
11:06 • 4min
Is Linearity in the Output a Problem?
14:44 • 2min
How to Train a Neural Network?
16:39 • 5min
How Can You Learn From Your Own Estimate?
21:22 • 2min
Dopomen Nurons
23:43 • 4min
How Can Reinforcement Learning Go Off the Rails?
27:43 • 2min
Reward Actions, Not Actions of the Agent
29:38 • 2min
Are Human Questions First?
31:50 • 2min
The Problem of Sparse Rewards
33:25 • 4min
Tree Senility
36:56 • 2min
Resonance Learning in the Real World
38:52 • 2min
How to Get Intelligent Behavior From a Eyes That Lack Curiousness
40:48 • 3min
How Do You Overcome This Problem of Sparsity?
43:36 • 2min
Can It Get Into a TV Screenana Game?
45:32 • 3min
The Effect of TV on the Visual System
48:34 • 2min
The Human Brain Is Like That, Right?
51:01 • 3min
A Novelty Seeking Agent - What Happens When You Play a Computer Game?
54:27 • 2min
Is It Possible to Create a Knowledge-Seeking Agent?
56:01 • 3min
Is It Safe to Build a Super Intelligent Knowledge Finder?
58:35 • 2min
Do What I Do
01:00:14 • 2min
Do You Know How to Model a Car?
01:01:50 • 2min
How to Automatically Identify Bad Drivers?
01:03:46 • 3min
Data Set Aggregation
01:06:33 • 3min
Imitation Is a Faily Mode of Imitation
01:09:07 • 4min
The Secret of Our Success
01:12:40 • 3min
Alpago Zero and Its Neural Networks
01:15:24 • 3min
Can You Guess What You're Going to Do?
01:18:40 • 3min
Learning by Imitating
01:21:32 • 2min
How to Do Things That We Can't Do, Right?
01:23:25 • 6min
Is It Possible to Teach Using This Method?
01:29:09 • 2min
How to Optimize Notification Delivery for a User Engagement Model?
01:30:44 • 2min
I Don't Want to Drink Alcohol
01:32:20 • 3min
Is There a Reward Function?
01:35:05 • 2min
How to Deactivate Part of the Network With Every Training Example
01:36:46 • 3min
Why Is It So Important to Have Ai Driven Cars?
01:39:54 • 3min
Is There a Risk of Uncertainty in Machine Decision Making?
01:43:06 • 3min
Is the Auto Pilot Mistaken?
01:46:02 • 2min
Is There a Difference Between Human Behavior and Human Irrationality?
01:47:45 • 3min
Ai Safety Grid Worlds
01:50:34 • 2min
Is There a Future for I Safety?
01:53:04 • 3min
Safety Research
01:56:32 • 4min
Are We Learning Something Important About Intelligence and What Humans Are Doing?
02:00:09 • 5min
Is It Possible to Flash That Out?
02:04:57 • 2min
Climate Change Is More Analogous Than the Same
02:07:24 • 3min
The Next Frontier in Social Choice Theory
02:10:09 • 3min
Is There a Stop on the Way Between Deception and Deception?
02:12:50 • 2min
Is There a Future for Transparency?
02:15:05 • 4min
A Good Black Mirror Episode, I Gess
02:18:37 • 3min
Yeye Daria on the Grando Ye, Episode Toooye Aan
02:21:13 • 2min
The Longton Future - Is There a Problem?
02:22:44 • 4min
I Think It's a Mistake to Go Down This Path.
02:26:54 • 2min
I Don't Think It's Going to Happen Any Time Soon
02:28:30 • 4min
Is There an Escapeye?
02:32:21 • 2min
How to Learn a Game Engine From Data?
02:34:05 • 2min
Is There Any Errors in the Book?
02:35:40 • 3min
Are There Any Important Challenges That You Can Contribute To?
02:38:19 • 3min
Machine Learning and Artificial Intelligence
02:41:00 • 2min
How to Solve a Reinforcement Learning Problem?
02:42:59 • 3min
The Deep Cued Networks Part of Machine Learning
02:46:17 • 2min
Effective Altruism and Ai Safety
02:47:58 • 2min
Is There a Path Dependent to Capital Constraints in Hedge Funds?
02:49:50 • 3min
The 80 Thousand Hours Podcast - Episodes 40 and 47
02:52:42 • 2min
Brian Christian is a bestselling author with a particular knack for accurately communicating difficult or technical ideas from both mathematics and computer science.
The 80,000 Hours team found his new book The Alignment Problem to be an insightful and comprehensive review of the state of the research into making advanced artificial intelligence useful and reliably safe, and we thought he'd be a great person to introduce the problem.
Full transcript, related links, and summary of this interviewThis episode first broadcast on the regular 80,000 Hours Podcast feed on March 5, 2021. Some related episodes include:
- #44 – Dr Paul Christiano on how OpenAI is developing real solutions to the 'AI alignment problem', and his vision of how humanity will progressively hand over decision-making to AI systems
- #3 – Dr Dario Amodei on OpenAI and how AI will change the world for good and ill
- #31 – Prof Allan Dafoe on defusing the political and economic risks posed by existing AI capabilities
- #47 – Catherine Olsson & Daniel Ziegler on the fast path into high-impact ML engineering roles
Series produced by Keiran Harris.