The Nonlinear Library LW - Concrete positive visions for a future without AGI by Max H
Nov 9, 2023
13:39
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Concrete positive visions for a future without AGI, published by Max H on November 9, 2023 on LessWrong.
"There was a threshold crossed somewhere," said the Confessor, "without a single apocalypse to mark it. Fewer wars. Less starvation. Better technology. The economy kept growing. People had more resource to spare for charity, and the altruists had fewer and fewer causes to choose from. They came even to me, in my time, and rescued me. Earth cleaned itself up, and whenever something threatened to go drastically wrong again, the whole attention of the planet turned in that direction and took care of it.
Eliezer Yudkowsky, Three Worlds Collide
A common sentiment among people worried about AI x-risk is that our world is on track to stagnate, collapse, or otherwise come to a bad end without (aligned) AGI to save the day.
Scott Alexander:
[I]f we never get AI, I expect the future to be short and grim. Most likely we kill ourselves with synthetic biology. If not, some combination of technological and economic stagnation, rising totalitarianism + illiberalism + mobocracy, fertility collapse and dysgenics will impoverish the world and accelerate its decaying institutional quality.
@disturbance in a a recent LW post that got lots of comments:
Statement: I want to deliberately balance the caution and the recklessness in developing AGI, such that it gets created in the last possible moment so that I and my close ones do not die.
A seemingly straightforward implication of this view is that we should therefore be willing to take on some amount of risk in order to build towards AGI faster than we would in a world where we had the luxury to take our time.
I think some of these sentiments and their implications are based on a mistaken view of the relative difficulty of particular technical and social challenges, but here I want to focus on a totally different point: there are lots of ways that things could go well without AGI (at least for a while).
Even if positive scenarios without AGI are unlikely or unrealistic given our current circumstances and trajectory, it's useful to have a concrete vision of what a good medium-term future without AGI could look like. I think it's especially important to take a moment to reflect on these possible good futures because recent preliminary governance wins, even if they succeed without qualification, are mainly focused on restriction and avoidance of bad outcomes rather than on building towards particular positive outcomes.
The rest of this post is a collection of examples of technologies, ideas, projects, and trends unrelated to AGI that give me hope and joy when I see them being worked on or talked about. It's not meant to be exhaustive in any sense - mostly it is just a list of areas that I personally enjoy reading about, and would consider professional opportunities related to them.
Most of them involve solving hard technological and social problems. Some are quite speculative, and likely to be intractable or extremely unlikely to come to pass in isolation. But making incremental progress on any one is probably robustly positive for the world and lucrative and fulfilling for the people working on them[1]. And progress tends to snowball, as long as there's no catastrophe to stop it.
As you read through the list, try to set aside your own views and probabilities on AGI, other x-risks, and fizzle or stagnation scenarios. Imagine a world where it is simply a given that humanity has time and space to flourish unimpeded for a time. Visualize what such a world might look like, where solutions are permitted to snowball without the threat of everything being cut short or falling to pieces. The purpose of this post is not to argue that any such world is particularly likely to be actualized; it is intended to serve as a concrete reminder that there a...
"There was a threshold crossed somewhere," said the Confessor, "without a single apocalypse to mark it. Fewer wars. Less starvation. Better technology. The economy kept growing. People had more resource to spare for charity, and the altruists had fewer and fewer causes to choose from. They came even to me, in my time, and rescued me. Earth cleaned itself up, and whenever something threatened to go drastically wrong again, the whole attention of the planet turned in that direction and took care of it.
Eliezer Yudkowsky, Three Worlds Collide
A common sentiment among people worried about AI x-risk is that our world is on track to stagnate, collapse, or otherwise come to a bad end without (aligned) AGI to save the day.
Scott Alexander:
[I]f we never get AI, I expect the future to be short and grim. Most likely we kill ourselves with synthetic biology. If not, some combination of technological and economic stagnation, rising totalitarianism + illiberalism + mobocracy, fertility collapse and dysgenics will impoverish the world and accelerate its decaying institutional quality.
@disturbance in a a recent LW post that got lots of comments:
Statement: I want to deliberately balance the caution and the recklessness in developing AGI, such that it gets created in the last possible moment so that I and my close ones do not die.
A seemingly straightforward implication of this view is that we should therefore be willing to take on some amount of risk in order to build towards AGI faster than we would in a world where we had the luxury to take our time.
I think some of these sentiments and their implications are based on a mistaken view of the relative difficulty of particular technical and social challenges, but here I want to focus on a totally different point: there are lots of ways that things could go well without AGI (at least for a while).
Even if positive scenarios without AGI are unlikely or unrealistic given our current circumstances and trajectory, it's useful to have a concrete vision of what a good medium-term future without AGI could look like. I think it's especially important to take a moment to reflect on these possible good futures because recent preliminary governance wins, even if they succeed without qualification, are mainly focused on restriction and avoidance of bad outcomes rather than on building towards particular positive outcomes.
The rest of this post is a collection of examples of technologies, ideas, projects, and trends unrelated to AGI that give me hope and joy when I see them being worked on or talked about. It's not meant to be exhaustive in any sense - mostly it is just a list of areas that I personally enjoy reading about, and would consider professional opportunities related to them.
Most of them involve solving hard technological and social problems. Some are quite speculative, and likely to be intractable or extremely unlikely to come to pass in isolation. But making incremental progress on any one is probably robustly positive for the world and lucrative and fulfilling for the people working on them[1]. And progress tends to snowball, as long as there's no catastrophe to stop it.
As you read through the list, try to set aside your own views and probabilities on AGI, other x-risks, and fizzle or stagnation scenarios. Imagine a world where it is simply a given that humanity has time and space to flourish unimpeded for a time. Visualize what such a world might look like, where solutions are permitted to snowball without the threat of everything being cut short or falling to pieces. The purpose of this post is not to argue that any such world is particularly likely to be actualized; it is intended to serve as a concrete reminder that there a...