The Nonlinear Library EA - AI things that are perhaps as important as human-controlled AI (Chi version) by Chi
Mar 3, 2024
35:59
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI things that are perhaps as important as human-controlled AI (Chi version), published by Chi on March 3, 2024 on The Effective Altruism Forum.
Topic of the post: I list potential things to work on other than keeping AI under human control.
Motivation
The EA community has long been worried about AI safety. Most of the efforts going into AI safety are focused on making sure humans are able to control AI. Regardless of whether we succeed at this, I think there's a lot of additional value on the line.
First of all, if we succeed at keeping AI under human control, there are still a lot of things that can go wrong. My perception is that this has recently gotten more attention, for example
here,
here,
here, and at least indirectly
here (I haven't read all these posts. and have chosen them to illustrate that others have made this point purely based on how easily I could find them). Why controlling AI doesn't solve everything is not the main topic of this post, but I want to at least sketch my reasons to believe this.
Which humans get to control AI is an obvious and incredibly important question and it doesn't seem to me like it will go well by default. It doesn't seem like current processes put humanity's wisest and most moral at the top. Humanity's track record at not causing large-scale unnecessary harm doesn't seem great (see factory farming). There is reasonable disagreement on how path-dependent epistemic and moral progress is but I think there is a decent chance that it is very path-dependent.
While superhuman AI might enable great moral progress and new mechanisms for making sure humanity stays on "moral track", superhuman AI also comes with lots of potential challenges that could make it harder to ensure a good future. Will MacAskill talks about "grand challenges" we might face shortly after the advent of superhuman AI
here. In the longer-term, we might face additional challenges. Enforcement of norms, and communication in general, might be extremely hard across galactic-scale distances. Encounters with
aliens (or even merely humanity thinking they might encounter aliens!) threaten conflict and could change humanity's priorities greatly. And if you're like me, you might believe there's a whole lot of weird
acausal
stuff to get right. Humanity might make decisions that influence these long-term issues already shortly after the development of advanced AI.
It doesn't seem obvious to me at all that a future where some humans are in control of the most powerful earth-originating AI will be great.
Secondly, even if we don't succeed at keeping AI under human control, there are other things we can fight for and those other things might be almost as important or more important than human control. Less has been written about this (although
not
nothing.) My current and historically very unstable best guess is that this reflects an actual lower importance of influencing worlds where humans don't retain control over AIs although I wish there was more work on this topic nonetheless. Justifying why I think influencing uncontrolled AI matters isn't the main topic of this post, but I would like to at least sketch my motivation again.
If there is alien life out there, we might care a lot about how future uncontrolled AI systems treat them. Additionally, perhaps we can prevent uncontrolled AI from having actively terrible values. And if you are like me, you might believe there are weird
acausal
reasons to make earth-originating AIs more likely to be a nice acausal citizen.
Generally, even if future AI systems don't obey us, we might still be able to imbue them with values that are more similar to ours. The AI safety community is aiming for human control, in part, because this seems much easier than aligning AIs with "what's morally good". But some properties that result in moral good...
Topic of the post: I list potential things to work on other than keeping AI under human control.
Motivation
The EA community has long been worried about AI safety. Most of the efforts going into AI safety are focused on making sure humans are able to control AI. Regardless of whether we succeed at this, I think there's a lot of additional value on the line.
First of all, if we succeed at keeping AI under human control, there are still a lot of things that can go wrong. My perception is that this has recently gotten more attention, for example
here,
here,
here, and at least indirectly
here (I haven't read all these posts. and have chosen them to illustrate that others have made this point purely based on how easily I could find them). Why controlling AI doesn't solve everything is not the main topic of this post, but I want to at least sketch my reasons to believe this.
Which humans get to control AI is an obvious and incredibly important question and it doesn't seem to me like it will go well by default. It doesn't seem like current processes put humanity's wisest and most moral at the top. Humanity's track record at not causing large-scale unnecessary harm doesn't seem great (see factory farming). There is reasonable disagreement on how path-dependent epistemic and moral progress is but I think there is a decent chance that it is very path-dependent.
While superhuman AI might enable great moral progress and new mechanisms for making sure humanity stays on "moral track", superhuman AI also comes with lots of potential challenges that could make it harder to ensure a good future. Will MacAskill talks about "grand challenges" we might face shortly after the advent of superhuman AI
here. In the longer-term, we might face additional challenges. Enforcement of norms, and communication in general, might be extremely hard across galactic-scale distances. Encounters with
aliens (or even merely humanity thinking they might encounter aliens!) threaten conflict and could change humanity's priorities greatly. And if you're like me, you might believe there's a whole lot of weird
acausal
stuff to get right. Humanity might make decisions that influence these long-term issues already shortly after the development of advanced AI.
It doesn't seem obvious to me at all that a future where some humans are in control of the most powerful earth-originating AI will be great.
Secondly, even if we don't succeed at keeping AI under human control, there are other things we can fight for and those other things might be almost as important or more important than human control. Less has been written about this (although
not
nothing.) My current and historically very unstable best guess is that this reflects an actual lower importance of influencing worlds where humans don't retain control over AIs although I wish there was more work on this topic nonetheless. Justifying why I think influencing uncontrolled AI matters isn't the main topic of this post, but I would like to at least sketch my motivation again.
If there is alien life out there, we might care a lot about how future uncontrolled AI systems treat them. Additionally, perhaps we can prevent uncontrolled AI from having actively terrible values. And if you are like me, you might believe there are weird
acausal
reasons to make earth-originating AIs more likely to be a nice acausal citizen.
Generally, even if future AI systems don't obey us, we might still be able to imbue them with values that are more similar to ours. The AI safety community is aiming for human control, in part, because this seems much easier than aligning AIs with "what's morally good". But some properties that result in moral good...