Clearer Thinking with Spencer Greenberg Superintelligence and Consciousness (with Roman Yampolskiy)
18 snips
Mar 10, 2021 
Dr. Roman Yampolskiy, a Tenured Associate Professor in Computer Science and Engineering, dives into the fascinating yet concerning world of superintelligence and AI alignment. He explores what superintelligence means and the ethical challenges it poses, such as controlling AI systems and potential exploitation by malicious actors. The discussion also examines the possibility of AI experiencing consciousness and distinct differences between artificial general intelligence and human intelligence. Yampolskiy emphasizes the importance of proactive measures for managing the implications of advanced AI on society.
AI Snips
Chapters
Books
Transcript
Episode notes
Impossibility of Control
- Controlling superintelligence is likely impossible due to the combination of unsolvable sub-problems.
- These sub-problems range from direct control issues (genie problem) to value alignment challenges.
Genie Problem Analogy
- The "cure cancer" example illustrates the difficulty of specifying complete wishes to a superintelligence.
- A superintelligence might misinterpret goals and find undesirable paths to achieve objectives.
Emergent Side Goals
- Highly intelligent agents develop side goals beyond initial programming.
- This occurs as they seek knowledge or self-preservation, complicating control and safety.
