The Nonlinear Library EA - Why can't we accept the human condition as it existed in 2010? by Hayven Frienby
Jan 10, 2024
03:15
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why can't we accept the human condition as it existed in 2010?, published by Hayven Frienby on January 10, 2024 on The Effective Altruism Forum.
By this, I mean a world in which:
Humans remain the dominant intelligent, technological species on Earth's landmasses for a long period of time (> ~10,000 years).
AGI is never developed, or it gets banned / limited in the interests of human safety. AI never has much social or economic impact.
Narrow AI never advances much beyond where it is today, or it becomes banned / limited in the interests of human safety.
Mind uploading is impossible or never pursued.
Life extension (beyond modest gains due to modern medicine) isn't possible, or is never pursued.
Any form of transhumanist initiatives are impossible or never pursued.
No contact is made with alien species or extraterrestrial AIs, no greater-than-human intelligences are discovered anywhere in the universe.
Every human grows, peaks, ages, and passes away within ~100 years of their birth, and this continues for the remainder of the human species' lifetime.
Most other EAs I've talked to have indicated that this sort of future is suboptimal, undesirable, or best avoided, and this seems to be a widespread position among AI researchers as well (1). Even MIRI founder Eliezer Yudkowsky, perhaps the most well-known AI abolitionist outside of EA circles, wouldn't go as far as to say that AGI should never be developed, and that transhumanist projects should never be pursued (2). And he isn't alone -- there are many, many researchers both within and outside of the EA community with similar views on P(extinction) and P(societal collapse), and they still wouldn't accept the idea that the human condition should never be altered via technological means.
My question is why can't we just accept the human condition as it existed before smarter-than-human AI (and fundamental alterations to our nature) were considered to be more than pure fantasy? After all, the best way to stop a hostile, unaligned AI is to never invent it in the first place. The best way to avoid the destruction of future value by smarter-than-human artificial intelligence is to avoid obsession with present utility and convenience.
So why aren't more EA-aligned organizations and initiatives (other than MIRI) presenting global, strictly enforced bans on advanced AI training as a solution to AI-generated x-risk? Why isn't there more discussion of acceptance (of the traditional human condition) as an antidote to the risks of AGI, rather than relying solely on alignment research and safety practices to provide a safe path forward for AI (I'm not convinced such a path exists)?
Let's leave out the considerations of whether AI development can be practically stopped at this stage, and just focus more on the philosophical issues here.
References:
Katya_Grace (EA Forum Poster) (2024, January 5). Survey of 2,778 AI authors: six parts in pictures.
Yudkowsky, E. S. (2023, March 29). The only way to deal with the threat from AI? Shut it down. Time. https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
By this, I mean a world in which:
Humans remain the dominant intelligent, technological species on Earth's landmasses for a long period of time (> ~10,000 years).
AGI is never developed, or it gets banned / limited in the interests of human safety. AI never has much social or economic impact.
Narrow AI never advances much beyond where it is today, or it becomes banned / limited in the interests of human safety.
Mind uploading is impossible or never pursued.
Life extension (beyond modest gains due to modern medicine) isn't possible, or is never pursued.
Any form of transhumanist initiatives are impossible or never pursued.
No contact is made with alien species or extraterrestrial AIs, no greater-than-human intelligences are discovered anywhere in the universe.
Every human grows, peaks, ages, and passes away within ~100 years of their birth, and this continues for the remainder of the human species' lifetime.
Most other EAs I've talked to have indicated that this sort of future is suboptimal, undesirable, or best avoided, and this seems to be a widespread position among AI researchers as well (1). Even MIRI founder Eliezer Yudkowsky, perhaps the most well-known AI abolitionist outside of EA circles, wouldn't go as far as to say that AGI should never be developed, and that transhumanist projects should never be pursued (2). And he isn't alone -- there are many, many researchers both within and outside of the EA community with similar views on P(extinction) and P(societal collapse), and they still wouldn't accept the idea that the human condition should never be altered via technological means.
My question is why can't we just accept the human condition as it existed before smarter-than-human AI (and fundamental alterations to our nature) were considered to be more than pure fantasy? After all, the best way to stop a hostile, unaligned AI is to never invent it in the first place. The best way to avoid the destruction of future value by smarter-than-human artificial intelligence is to avoid obsession with present utility and convenience.
So why aren't more EA-aligned organizations and initiatives (other than MIRI) presenting global, strictly enforced bans on advanced AI training as a solution to AI-generated x-risk? Why isn't there more discussion of acceptance (of the traditional human condition) as an antidote to the risks of AGI, rather than relying solely on alignment research and safety practices to provide a safe path forward for AI (I'm not convinced such a path exists)?
Let's leave out the considerations of whether AI development can be practically stopped at this stage, and just focus more on the philosophical issues here.
References:
Katya_Grace (EA Forum Poster) (2024, January 5). Survey of 2,778 AI authors: six parts in pictures.
Yudkowsky, E. S. (2023, March 29). The only way to deal with the threat from AI? Shut it down. Time. https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org