The Nonlinear Library EA - #173 - Digital minds, and how to avoid sleepwalking into a major moral catastrophe (Jeff Sebo on the 80,000 Hours Podcast) by 80000 Hours
Nov 29, 2023
25:51
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: #173 - Digital minds, and how to avoid sleepwalking into a major moral catastrophe (Jeff Sebo on the 80,000 Hours Podcast), published by 80000 Hours on November 29, 2023 on The Effective Altruism Forum.
We just published an interview: Jeff Sebo on digital minds, and how to avoid sleepwalking into a major moral catastrophe. Listen on Spotify or click through for other audio options, the transcript, and related links. Below are the episode summary and some key excerpts.
Episode summary
We do have a tendency to anthropomorphise nonhumans - which means attributing human characteristics to them, even when they lack those characteristics. But we also have a tendency towards anthropodenial - which involves denying that nonhumans have human characteristics, even when they have them. And those tendencies are both strong, and they can both be triggered by different types of systems. So which one is stronger, which one is more probable, is again going to be contextual.
But when we then consider that we, right now, are building societies and governments and economies that depend on the objectification, exploitation, and extermination of nonhumans, that - plus our speciesism, plus a lot of other biases and forms of ignorance that we have - gives us a strong incentive to err on the side of anthropodenial instead of anthropomorphism.
Jeff Sebo
In today's episode, host Luisa Rodriguez interviews Jeff Sebo - director of the Mind, Ethics, and Policy Program at NYU - about preparing for a world with digital minds.
They cover:
The non-negligible chance that AI systems will be sentient by 2030
What AI systems might want and need, and how that might affect our moral concepts
What happens when beings can copy themselves? Are they one person or multiple people? Does the original own the copy or does the copy have its own rights? Do copies get the right to vote?
What kind of legal and political status should AI systems have? Legal personhood? Political citizenship?
What happens when minds can be connected? If two minds are connected, and one does something illegal, is it possible to punish one but not the other?
The repugnant conclusion and the rebugnant conclusion
The experience of trying to build the field of AI welfare
What improv comedy can teach us about doing good in the world
And plenty more.
Producer and editor: Keiran Harris
Audio Engineering Lead: Ben Cordell
Technical editing: Dominic Armstrong and Milo McGuire
Additional content editing: Katy Moore and Luisa Rodriguez
Transcriptions: Katy Moore
Highlights
When to extend moral consideration to AI systems
Jeff Sebo: The general case for extending moral consideration to AI systems is that they might be conscious or sentient or agential or otherwise significant. And if they might have those features, then we should extend them at least some moral consideration in the spirit of caution and humility.
So the standard should not be, "Do they definitely matter?" and it should also not be, "Do they probably matter?" It should be, "Is there a reasonable, non-negligible chance that they matter, given the information available?" And once we clarify that that is the bar for moral inclusion, then it becomes much less obvious that AI systems will not be passing that bar anytime soon.
Luisa Rodriguez: Yeah, I feel kind of confused about how to think about that bar, where I think you're using the term "non-negligible chance." I'm curious: What is a negligible chance? Where is the line? At what point is something non-negligible?
Jeff Sebo: Yeah, this is a perfectly reasonable question. This is somewhat of a term of art in philosophy and decision theory. And we might not be able to very precisely or reliably say exactly where the threshold is between non-negligible risks and negligible risks - but what we can say, as a starting point, is that a risk...
We just published an interview: Jeff Sebo on digital minds, and how to avoid sleepwalking into a major moral catastrophe. Listen on Spotify or click through for other audio options, the transcript, and related links. Below are the episode summary and some key excerpts.
Episode summary
We do have a tendency to anthropomorphise nonhumans - which means attributing human characteristics to them, even when they lack those characteristics. But we also have a tendency towards anthropodenial - which involves denying that nonhumans have human characteristics, even when they have them. And those tendencies are both strong, and they can both be triggered by different types of systems. So which one is stronger, which one is more probable, is again going to be contextual.
But when we then consider that we, right now, are building societies and governments and economies that depend on the objectification, exploitation, and extermination of nonhumans, that - plus our speciesism, plus a lot of other biases and forms of ignorance that we have - gives us a strong incentive to err on the side of anthropodenial instead of anthropomorphism.
Jeff Sebo
In today's episode, host Luisa Rodriguez interviews Jeff Sebo - director of the Mind, Ethics, and Policy Program at NYU - about preparing for a world with digital minds.
They cover:
The non-negligible chance that AI systems will be sentient by 2030
What AI systems might want and need, and how that might affect our moral concepts
What happens when beings can copy themselves? Are they one person or multiple people? Does the original own the copy or does the copy have its own rights? Do copies get the right to vote?
What kind of legal and political status should AI systems have? Legal personhood? Political citizenship?
What happens when minds can be connected? If two minds are connected, and one does something illegal, is it possible to punish one but not the other?
The repugnant conclusion and the rebugnant conclusion
The experience of trying to build the field of AI welfare
What improv comedy can teach us about doing good in the world
And plenty more.
Producer and editor: Keiran Harris
Audio Engineering Lead: Ben Cordell
Technical editing: Dominic Armstrong and Milo McGuire
Additional content editing: Katy Moore and Luisa Rodriguez
Transcriptions: Katy Moore
Highlights
When to extend moral consideration to AI systems
Jeff Sebo: The general case for extending moral consideration to AI systems is that they might be conscious or sentient or agential or otherwise significant. And if they might have those features, then we should extend them at least some moral consideration in the spirit of caution and humility.
So the standard should not be, "Do they definitely matter?" and it should also not be, "Do they probably matter?" It should be, "Is there a reasonable, non-negligible chance that they matter, given the information available?" And once we clarify that that is the bar for moral inclusion, then it becomes much less obvious that AI systems will not be passing that bar anytime soon.
Luisa Rodriguez: Yeah, I feel kind of confused about how to think about that bar, where I think you're using the term "non-negligible chance." I'm curious: What is a negligible chance? Where is the line? At what point is something non-negligible?
Jeff Sebo: Yeah, this is a perfectly reasonable question. This is somewhat of a term of art in philosophy and decision theory. And we might not be able to very precisely or reliably say exactly where the threshold is between non-negligible risks and negligible risks - but what we can say, as a starting point, is that a risk...