#9 - Christine Peterson on how insecure computers could lead to global disaster, and how to fix it

16 snips

Oct 4, 2017

Christine Peterson, co-founder of the Foresight Institute and coiners of the term open source, reflects on tech futures and risks. She explores whether insecure computers could cause civilization-level harm. She discusses nanotechnology, AI-driven hacking, cryonics and lifespan research. She also offers career and organizational advice for steering powerful technologies toward safety.

Ask episode

AI Snips

Chapters

Books

Transcript

Episode notes

ADVICE

Get Technical On Computer Security Now

Do train in computer security and trusted OS design (e.g., seL4) to help defend critical infrastructure before automated attackers arrive.
Use red‑team AI tools to stress-test systems continuously and adopt formally verified components when possible.

INSIGHT

Security Requires New Paradigms Not Just Patching

Patching legacy OSes (Windows, macOS) won't suffice; Peterson advocates paradigm shifts toward systems built with security from the start.
Secure systems exist (seL4, well‑attacked blockchain code) but require different development mindset and continuous hostile testing.

ADVICE

Prioritize Aging Research For Huge Life‑Year Gains

Do work on aging research because it accounts for the largest global loss of life‑years and burdens both rich and developing countries.
Consider repair‑oriented strategies (Aubrey de Grey style) and fund or join research that extends healthspan, not just lifespan.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Take a trip to Silicon Valley in the 70s and 80s, when going to space sounded like a good way to get around environmental limits, people started cryogenically freezing themselves, and nanotechnology looked like it might revolutionise industry – or turn us all into grey goo.

Full transcript, coaching application form, overview of the conversation, and extra resources to learn more:

In this episode of the 80,000 Hours Podcast Christine Peterson takes us back to her youth in the Bay Area, the ideas she encountered there, and what the dreamers she met did as they grew up. We also discuss how she came up with the term ‘open source software’ (and how she had to get someone else to propose it).

Today Christine helps runs the Foresight Institute, which fills a gap left by for-profit technology companies – predicting how new revolutionary technologies could go wrong, and ensuring we steer clear of the downsides.

We dive into:

* Whether the poor security of computer systems poses a catastrophic risk for the world. Could all our essential services be taken down at once? And if so, what can be done about it?
* Can technology ‘move fast and break things’ without eventually breaking the world? Would it be better for technology to advance more quickly, or more slowly?
* How Christine came up with the term ‘open source software’ (and why someone else had to propose it).
* Will AIs designed for wide-scale automated hacking make computers more or less secure?
* Would it be good to radically extend human lifespan? Is it sensible to cryogenically freeze yourself in the hope of being resurrected in the future?
* Could atomically precise manufacturing (nanotechnology) really work? Why was it initially so controversial and why did people stop worrying about it?
* Should people who try to do good in their careers work long hours and take low salaries? Or should they take care of themselves first of all?
* How she thinks the the effective altruism community resembles the scene she was involved with when she was wrong, and where it might be going wrong.

Get free, one-on-one career advice

We’ve helped dozens of people compare between their options, get introductions, and jobs important for the the long-run future. If you want to work on any of the problems discussed in this episode, find out if our coaching can help you.