The Nonlinear Library LW - On the abolition of man by Joe Carlsmith
Jan 18, 2024
01:09:02
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On the abolition of man, published by Joe Carlsmith on January 18, 2024 on LessWrong.
(Cross-posted from my website. Podcast version here, or search for "Joe Carlsmith Audio" on your podcast app.
This essay is part of a series that I'm calling "Otherness and control in the age of AGI." I'm hoping that the individual essay can be read fairly well on their own, but see here for brief summaries of the essays that have been released thus far.)
Earlier in this series, I discussed a certain kind of concern about the AI alignment discourse - namely, that it aspires to exert an inappropriate degree of control over the values that guide the future. In considering this concern, I think it's important to bear in mind the aspects of our own values that are specifically focused on pluralism, tolerance, helpfulness, and inclusivity towards values different-from-our-own (I discussed these in the last essay).
But I don't think this is enough, on its own, to fully allay the concern in question. Here I want to analyze one version of this concern more directly, and to try to understand what an adequate response could consist in.
Tyrants and poultry-keepers
Have you read The Abolition of Man, by C.S. Lewis? As usual: no worries if not (I'll summarize it in a second). But: recommended. In particular: The Abolition of Man is written in opposition to something closely akin to the sort of Yudkowskian worldview and orientation towards the future that I've been discussing.[1] I think the book is wrong about a bunch of stuff.
At its core, The Abolition of Man is about meta-ethics. Basically, Lewis thinks that some kind of moral realism is true. In particular, he thinks cultures and religions worldwide have all rightly recognized something he calls the Tao - some kind of natural law; a way that rightly reflects and responds to the world; an ethics that is objective, authoritative, and deeply tied to the nature of Being itself. Indeed, Lewis thinks that the content of human morality across cultures and time periods has been broadly similar, and he includes, in the appendix of the book, a smattering of quotations meant to illustrate (though not: establish) this point.
"Laozi Riding an Ox by Zhang Lu (c. 1464--1538)" (Image source here)
But Lewis notices, also, that many of the thinkers of his day deny the existence of the Tao. Like Yudkowsky, they are materialists, and "subjectivists," who think - at least intellectually - that there is no True Way, no objective morality, but only ... something else. What, exactly?
Lewis considers the possibility of attempting to ground value in something non-normative, like instinct. But he dismisses this possibility on familiar grounds: namely, that it fails to bridge the gap between is and ought (the same arguments would apply to Yudkowsky's "volition"). Indeed, Lewis thinks that all ethical argument, and all worthy ethical reform, must come from "within the Tao" in some sense - though exactly what sense isn't fully clear. The least controversial interpretation would be the also-familiar claim that moral argument must grant moral intuition some sort of provisional authority.
This part of the book is not, in my opinion, the most interesting part (though: it's an important backdrop). Rather, the part I find most interesting comes later, in the final third, where Lewis turns to the possibility of treating human morality as simply another part of nature, to be "conquered" and brought under our control in the same way that other aspects of nature have been.
Here Lewis imagines an ongoing process of scientific modernity, in which humanity gains more and more mastery over its environment.
In reality, of course, if any one age really attains, by eugenics and scientific education, the power to make its descendants what it pleases, all men who live after it are the pat...
(Cross-posted from my website. Podcast version here, or search for "Joe Carlsmith Audio" on your podcast app.
This essay is part of a series that I'm calling "Otherness and control in the age of AGI." I'm hoping that the individual essay can be read fairly well on their own, but see here for brief summaries of the essays that have been released thus far.)
Earlier in this series, I discussed a certain kind of concern about the AI alignment discourse - namely, that it aspires to exert an inappropriate degree of control over the values that guide the future. In considering this concern, I think it's important to bear in mind the aspects of our own values that are specifically focused on pluralism, tolerance, helpfulness, and inclusivity towards values different-from-our-own (I discussed these in the last essay).
But I don't think this is enough, on its own, to fully allay the concern in question. Here I want to analyze one version of this concern more directly, and to try to understand what an adequate response could consist in.
Tyrants and poultry-keepers
Have you read The Abolition of Man, by C.S. Lewis? As usual: no worries if not (I'll summarize it in a second). But: recommended. In particular: The Abolition of Man is written in opposition to something closely akin to the sort of Yudkowskian worldview and orientation towards the future that I've been discussing.[1] I think the book is wrong about a bunch of stuff.
At its core, The Abolition of Man is about meta-ethics. Basically, Lewis thinks that some kind of moral realism is true. In particular, he thinks cultures and religions worldwide have all rightly recognized something he calls the Tao - some kind of natural law; a way that rightly reflects and responds to the world; an ethics that is objective, authoritative, and deeply tied to the nature of Being itself. Indeed, Lewis thinks that the content of human morality across cultures and time periods has been broadly similar, and he includes, in the appendix of the book, a smattering of quotations meant to illustrate (though not: establish) this point.
"Laozi Riding an Ox by Zhang Lu (c. 1464--1538)" (Image source here)
But Lewis notices, also, that many of the thinkers of his day deny the existence of the Tao. Like Yudkowsky, they are materialists, and "subjectivists," who think - at least intellectually - that there is no True Way, no objective morality, but only ... something else. What, exactly?
Lewis considers the possibility of attempting to ground value in something non-normative, like instinct. But he dismisses this possibility on familiar grounds: namely, that it fails to bridge the gap between is and ought (the same arguments would apply to Yudkowsky's "volition"). Indeed, Lewis thinks that all ethical argument, and all worthy ethical reform, must come from "within the Tao" in some sense - though exactly what sense isn't fully clear. The least controversial interpretation would be the also-familiar claim that moral argument must grant moral intuition some sort of provisional authority.
This part of the book is not, in my opinion, the most interesting part (though: it's an important backdrop). Rather, the part I find most interesting comes later, in the final third, where Lewis turns to the possibility of treating human morality as simply another part of nature, to be "conquered" and brought under our control in the same way that other aspects of nature have been.
Here Lewis imagines an ongoing process of scientific modernity, in which humanity gains more and more mastery over its environment.
In reality, of course, if any one age really attains, by eugenics and scientific education, the power to make its descendants what it pleases, all men who live after it are the pat...