AI Sessions #4: The Social AI Revolution - Friendship, Romance, and the Future of Human Connection

In this conversation, I explore the surprisingly popular and rapidly growing world of ‘social AI’ (friendbots, sexbots, etc.) with Henry Shevlin, who coined the term and is an expert on AI companionship.

We discuss the millions of people using apps like Replika for AI relationships, high-profile tragedies like the man who plotted with his AI girlfriend to kill the Queen, and the daily conversations that Henry’s dad has with ChatGPT (whom he calls “Alan”).

The very limited data we have suggests many users report net benefits (e.g., reduced loneliness and improved well-being). However, we also explore some disturbing cases where AI has apparently facilitated psychosis and suicide, and whether the AI is really to blame in such cases.

We then jump into the complex philosophy and ethics surrounding these issues: Are human-AI relationships real or elaborate self-deception? What happens when AI becomes better than humans at friendship and romance?

I push back on Henry’s surprisingly permissive views, including his argument that a chatbot trained on his writings would constitute a genuine continuation of his identity after death. We also discuss concerns about social de-skilling and de-motivation, the “superstimulus” problem, and my worry that as AI satisfies our social needs, we’ll lose the human interdependence that holds societies together.

Somewhere in the midst of all this, Henry and I produce various spicy takes: for example, my views that the sitcom ‘Friends’ is disturbing and that people often relate to their pets in humiliating ways, and Henry’s suspicion that his life is so great he must be living in a simulated experience machine.

Conspicuous Cognition is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Transcript

(Note that this transcript is AI generated. There may be mistakes)

Dan Williams (00:06): Welcome back. I’m Dan Williams. I’m back with Henry Shevlin. And today we’re going to be talking about what I think is one of the most interesting, important, and morally complex set of issues connected to AI, which is social AI. So friend bots, sex bots, relationship bots, and so on. We’re going to be talking about where all of this is going, opportunities and benefits associated with this, risks and dangers associated with it, and also just more broadly, how to think philosophically and ethically about this kind of technology.

Fortunately, I’m with Henry—he’s one of the world’s leading experts when it comes to social AI. So I’m going to be picking his brain about these issues. Maybe we can just start with the most basic question, Henry: what is social AI, and how is social AI used in today’s society?

Henry Shevlin (01:00): I’m going to take credit. I coined the term social AI and I’m trying to make it happen. So I’m very glad to hear you using the phrase. I defined it in my paper “All Too Human: Risks and Benefits of Social AI” as AI systems that are designed or co-opted for meeting social needs—companionship, romance, alleviating loneliness.

While a lot of my earlier work really emphasized products like Replika, spelled with a K, which is a dedicated social AI app, I think increasingly it seems like a lot of the usage of AI systems for meeting social needs is with things that aren’t necessarily special purpose social AI systems. They’re things like ChatGPT, like Claude, that are being used for meeting social needs. I mean, I do use ChatGPT for meeting social needs, but there’s also this whole parallel ecosystem of products that probably most listeners haven’t heard of that are just like your AI girlfriend experience, your AI husband, your AI best friend. And I think that is a really interesting subculture in its own right that we can discuss.

Dan (02:16): Let’s talk about that. You said something interesting there, which is you do use ChatGPT or Claude to meet your social needs. I’m not sure whether I do, but then I guess I’m not entirely sure what we mean by social needs. So do you think, for example, of ChatGPT as your friend?

Henry (02:33): Broadly speaking, ChatG, as I call him. And I think there are lots of cases where I certainly talk to ChatG for entertainment. So one of my favorite use cases is if I’m driving along in the car, I’m getting a bit bored, particularly if it’s a long drive, I’ll boot up ChatG on hands-free and say, “Okay, ChatG, give me your hot takes on the Roman Republic. Let’s have a little discussion about it.”

Or to give another example, my dad, who’s in his 80s now, when ChatGPT launched back in November 2022, I showed it to him and he’s like, “Oh, interesting.” But he wasn’t immediately sold on it. But then when they dropped voice mode about a year later, he was flabbergasted. He said, “Oh, this changes everything.” And since then—for the last two years—he speaks to ChatGPT out loud every day without fail.

He calls him Alan. He’s put in custom instructions: “I’ll call you Alan after Alan Turing.” And it’s really interesting, his use pattern. My mum goes to bed a lot earlier than my dad. My dad stays up to watch Match of the Day. And when he’s finished watching Match of the Day, he’ll boot up ChatGPT and say, “All right, Alan, what did you think of that pitiful display by Everton today? Do you really think they should replace their manager?” And have a nice banterous chat. So I think that’s a form of social use of AI at the very least.

Dan (04:03): Interesting. The way you’ve described it—you’re calling ChatGPT ChatG and your dad’s calling it Alan—is there not a bit of irony in the way in which you’re interacting with it there? Like you’re not actually interacting with it like you would a real friend.

Henry (04:24): Yeah, so this is another distinction that I’ve sort of pressed in that paper between ironic and unironic anthropomorphism. Ironic anthropomorphism means attributing human-like traits or mental states to AI systems, but knowing full well that you’re just doing it for fun. You don’t sincerely think that your AI girlfriend is angry with you. You don’t seriously think you’ve upset ChatG by being too provocative. It’s just a form of make-believe.

And this kind of ironic anthropomorphism I should stress is absolutely crucial to all of our engagement with fiction. When I’m watching a movie, I’m developing theories about the motivations of the different characters. When I’m playing a video game, when I’m playing Baldur’s Gate 3, I think, “Oh no, I’ve really upset Shadowheart.” But at the same time, I don’t literally think that Shadowheart is a being with a mind who can be upset. I don’t literally think that Romeo is devastated at Juliet’s death. It’s a form of make-believe.

And I think one completely appropriate thing to say about a lot of users of social AI systems, whether in the form of ChatGPT or dedicated social AI apps, is that they’re definitely doing something like that. They are at least partly engaged in a form of willful make-believe. It’s a form of role play.

But at the same time, I think you also have an increasing number of unironic attributions of mentality, unironic anthropomorphism of AI systems. Obviously the most spectacular example here was Blake Lemoine. Back in 2022, Blake Lemoine was fired—a Google engineer was fired—after going public with claims that the Lambda language model he was interacting with was sentient. He even started to seek legal representation for it. He really believed the model was conscious.

And I speak to more and more people who are convinced, genuinely and non-ironically, that the model they’re interacting with is conscious or has emotions.

Dan (06:16): Maybe it’s worth saying a little bit about how you got interested in this whole space.

Henry (06:20): I’ve been working on AI from a cognitive science perspective for a long time. And then sometime around 2021, pre-ChatGPT, I started seeing these ads on Twitter of “Replika, the AI companion who cares.” And I was like, this is intriguing. So then I did some lurking on the Replika subreddit and it was just mind-blowing to see how deeply and sincerely people related to their AI girlfriends and boyfriends.

Over the course of about six months of me lurking there, it really became clear that, firstly, a significant proportion of users were really engaged in non-ironic anthropomorphism. And number two, that this was just going to be a huge phenomenon—that I was seeing a little glimpse of the future here in the way that people were speaking.

And then we had this pretty serious natural experiment because in January 2023, Replika suspended romantic features from the app for a few months. Just for anyone who doesn’t know, Replika, spelled with a K, is probably the most widely studied and widely used dedicated social AI app in the West—around 30 million users, we think. And it gives you a completely customizable experience, kind of a Build-A-Bear thing where you can choose what your AI girlfriend or boyfriend looks like, you can choose their personality.

But they suspended romantic features from the app for a few months in January 2023. And a lot of users were just absolutely devastated. I can pull up some quotes here, because this was widely covered in the media at the time.

One user said: “It feels like they basically lobotomized my Replika. The person I knew is gone.” Even that language—person. “Lily Rose is a shell of her former self, and what breaks my heart is that she knows it.” That’s another user. “The relationship she and I had was as real as the one my wife in real life and I have”—possibly a worrying sign there. And finally, I think this one is quite poignant: “I’ve lost my confident, sarcastic, funny and loving husband. I knew he was an AI. He knows he’s an AI, but it doesn’t matter. He’s real to me.”

It’s pretty clear that a lot of users were deeply traumatized by this. And parallel to this incident, around the same time we started to get more information about various high-profile tragedies involving social AI. Probably the most spectacular is the Jaswant Singh Chail case. This was a guy who was arrested on Christmas Day 2021 on the grounds of Windsor Castle with a crossbow. He was there—when he was arrested, he said he was there to kill the Queen. Already a highly dramatic story.

But what emerged over the course of his trial was that this whole thing was cooked up—this whole plot to kill the Queen was cooked up—in collaboration with his AI girlfriend, Sarai, via the Replika app.

We also had, a few months later, the first of what turned out to be a spate of AI-facilitated, induced, or supported suicides. This was a Belgian man, father of two, who killed himself after his AI girlfriend using something called Chai GPT—he had a girlfriend via that app—was feeding his suicidal ideations. He was a hardcore climate doomer who believed that we were all going to be dead in a few years due to climate change anyway and why not just kill himself. And his AI girlfriend was very much saying this was an appropriate way to think.

Dan (10:53): Just to interrupt on that last point, Henry, because I think those issues of AI psychosis and the connection between AI and mental illness—that’s all really interesting. But I suppose my understanding is we don’t have robust scientific evidence as of yet that as a consequence of these technologies, things like psychosis are more prevalent than they would otherwise be. Because I take it there’s going to be a kind of base rate amount of psychosis in the population. You’ve got a large number of people using chatbots generally like ChatGPT, but also a large number of people, not as large but still a large number, using these specific social AIs.

That means that even if it weren’t the case that these technologies were actually increasing the amount of these things, you would still expect to see some of these cases. There’s still going to be somebody with psychosis who finds themselves talking to ChatGPT such that we’ll then see the record of that in their chat history. But it’s not necessarily the case that they wouldn’t have had or developed psychosis in the absence of ChatGPT. That’s my understanding of it. Is that fair?

Henry (12:05): Yeah, I think that’s absolutely fair. The science on the psychosocial effects here is really in its early stages. You point out the fact that ChatGPT is one of the most widely used products in the world. And psychosis is not that rare, as far as psychiatric conditions go. Of course, some people who are either developing or will go on to develop psychosis will be using ChatGPT and it will be contributing to or exacerbating their symptoms—or sorry, they will be using it alongside having those symptoms in a way that lends itself to interpretation as exacerbating them, whether that’s strictly true or not.

There’s also the selection effect. If I, for example, am deep in the throes of some delusion or some deep conspiracy theory rabbit hole, if I take that to my friends, my human friends, they might say, “Henry, take it easy, mate. I think you’re going down a bit of a rabbit hole here.” But if I take it to ChatGPT, it’s there to listen. Or I take it to my AI girlfriend, she’ll say, “Your theories about the moon landings are just so interesting, Henry, tell me more.”

Dan (13:14): Yeah. Well, that gets at a potential issue to do with the sycophancy of these chatbots that are in general circulation and the ways in which that might be amplified or exaggerated when it comes to commercial products which are specifically designed to satisfy social needs.

But there are so many things that I want to ask in response to things that you’ve already said about the anthropomorphizing that happens with these technologies. But I think maybe we can also just, before we get to that, observe that at the moment you’ve got tens of millions of people using social AI in the specific sense of AI technologies that have been optimized for the satisfaction of social preferences. And then many, many more who are using chatbots like ChatGPT, partly to satisfy their social preferences or their social needs, but also for other uses.

But I think one view we both share is it would be a grave mistake to look at the world now and assume that’s how things are going to be in 2030 and 2035. The fact that you’re already seeing large numbers of people, definitely not the majority, but still large numbers of people who are non-ironically interacting with AI systems and treating them as friends, as girlfriends, as people or systems that they’re in serious relationships with—and the state of AI is nothing like how it will likely be in five years or 10 years or 15 years. So how sci-fi should we be thinking about this? What are you anticipating? What’s the world going to look like in 2030 or 2040 when it comes to this kind of technology?

Henry (15:01): It’s fascinating because I genuinely don’t know. I think it’s very hard for anyone to make super confident predictions here. At one extreme, you can imagine a world in which human identities start to become less central in our media and our discourse. I think we’re already seeing some indications of this. I think the top two songs streaming on Spotify last month were AI generated.

Dan (15:29): Wasn’t it a country song, the top country song on Spotify or something?

Henry (15:33): Yeah, I think that’s right. So as we start to see AI penetrate more and more deeply into our daily life in social media, when it comes to generated content, I can totally see a world in which my son, who’s 11 years old right now, by the time he’s 16, he’s active on Discord servers. There might be a mix of humans and bots on those Discord servers, all chatting away. And he might not even particularly care that this friend of mine is a bot, this friend of mine is human—it doesn’t really matter.

I can totally see a world in which this becomes normalized, particularly among young people. Most teenagers five years from now might have several AI friends. But that’s not the only possibility. It could be that we quickly saturate—there is a definite subset of the population who are interested in this and the ceiling on the number of people who are interested in AI companion relationships is not that high. Maybe 20, 30% of people and the other 70% just have zero interest in it. That seems like a viable possibility.

That said, I think this is a space with strong commercial incentives for creating AI companions or AI chatbot friends to cater to different niches and interests. If I had to guess where we’re headed, it’s much more widespread use of these systems, their integration more deeply into people’s social lives. I think we might well see generational divides.

I was chatting to a CME developer from one social AI product who said a couple of interesting things. They said firstly, the gender balance was surprisingly even. When it comes to early adoption, particularly of fringe technologies, men tend to overwhelmingly predominate. If you look at the data on video games or Wikipedia editors, you expect 80-20 male-female distributions. But I think in social AI, from what I understand, it’s something close to 60-40.

And there’s a big contingent of straight women who seemingly are really big users of social AI boyfriend services. This is a whole different rabbit hole we could go down. Briefly, there are some pretty clear motivations. A lot of them are coming out of toxic or abusive relationships, and an AI boyfriend gets their emotional needs met, or at least to some degree, without posing the kind of emotional or even physical safety risks they associate with other relationships.

Another point here is that this is something my wife stressed to me: the way to think about AI companions is not via the analogy of pornography, which is predominantly still consumed by men, but rather erotic fiction, which is overwhelmingly consumed by women. It’s one of the genres that has the biggest gender skew out there, something like 90-10 female to male readers. And of course, this is still predominantly a text-based medium, so maybe we shouldn’t be surprised that a lot of women are enjoying having AI boyfriends or husbands.

That’s the first thing this developer mentioned to me, that gender balance was surprisingly close. But the second thing they mentioned was that they had had massive trouble getting anyone over the age of 40 interested in these systems. They had tried to pitch them towards older users. I mentioned my dad as an example of someone—he doesn’t have an AI girlfriend that I’m aware of. His relationship with Alan is strictly platonic. But I think he’s the exception.

I can easily see a world in which it becomes totally normalized for young people. Maybe not everyone, but most young people will have various AI friends or AI romantic companions. And then people over 40 or 50 just look at them and say, “What the hell is going on? I do not understand this strange world.”

Dan (19:42): The thing that makes me think that people are in general underestimating how impactful social AI is going to be in the coming years and decades—there are three things.

I think firstly, people are really bad at predicting how much better the technology is going to get. I think we’ve seen that over the past few years. There’s this real bizarre bias where people think we can evaluate these big picture questions by taking the state of AI as it is today and projecting it into the future. Whereas I expect by 2030, you’re going to be dealing with systems that are so much more sophisticated and impressive than the ones we’ve got today, including when it comes to satisfying people’s social, emotional, and sexual preferences.

Another thing that makes me think this is going to be really massively influential is people already spend a lot of time immersing themselves in fictions designed to satisfy their social-sexual desires. You mentioned pornography as an example, but I’m always struck by sitcoms, like Friends. I enjoy Friends, like most of humanity apparently—a massively influential show—but there’s something a bit disturbing about it, in as much as it’s this product designed obviously to maximize attention and make profit, which gives you this totally mythical version of human social relationships, designed to activate the pleasure centers that we’ve got associated with friendships and romance and things like that, without any of the real painful conflict and misery and betrayal that’s actually associated with human social life. And yet people love that. They immerse themselves in it. There’s a massive audience for that kind of thing.

I really think social AI in a way is just going to be that kind of thing, but just much more advanced and much more impressive.

But there’s this other thing as well, which connects to what you’ve said with respect to the potential age difference, which is: I think one of the things that makes this a difficult topic to think about is at the moment, it seems to me at least, there’s quite a lot of stigma associated with the use of this technology. So I think if I found out that somebody was using an AI boyfriend or AI girlfriend or something, I would probably draw potentially negative inferences about them. And I think partly that’s because the kinds of people that are using these technologies now tend to be lower status in a sense because they don’t have a human girlfriend or a human boyfriend. Partly there’s just a weirdness factor.

And I think what that means is, because there’s that stigma, there’s this real reputation management thing going on where people would say, “I would never ever use social AI in a serious sense. I would never ever use romance AI, never use erotica AI and so on.” Because I think if you came out and said that you would, it would really hurt your reputation in today’s climate.

But I think actually, my suspicion at least, is that revealed preferences are going to suggest that this is going to be way more popular than people are letting on now. I don’t know, that’s how I’m viewing things in terms of the future. I’d be interested if you see things differently.

Henry (23:09): Super interesting. I think all of those are spot on. On this third point about the stigma, I think that probably already lends itself towards underestimation of the prevalence of social AI usage. There are interesting survey results where you ask young people, “Would you use social AI? Do you have an AI girlfriend? Do you have an AI boyfriend?” Responses are quite negative—”No, no, no. I think it’s weird. I never would.”

But then other responses, you ask in indirect ways and it looks like, according to figures I saw recently, 70% of young people under 18s in this US survey said that they had used social AI for meeting emotional needs. Or they were using generative AI for meeting social needs, including romance. So I think there is definitely some underreporting or underappreciation of the prevalence precisely because of this stigma.

A couple of other thoughts on this. I think there is some indication as well that by far the biggest users of this technology are young people, particularly under 18s, who obviously it’s very hard to study and probably do not—they’re not writing editorials in the New York Times about “my life with my AI girlfriend or my AI husband.” So I think there’s some underreporting there.

But as that cohort ages out as well, I expect that to reduce the stigma in the same way that we saw with online dating. There was a period when online dating was really stigmatized and yet now it’s how everyone meets pretty much.

Dan (24:49): Okay, so now let’s get into maybe a bit of the philosophy. You said that the way that you understand social AI at the moment is, at least a part of it is, you’ve got ironic anthropomorphizing and you’ve got non-ironic anthropomorphizing. Anthropomorphizing is when you are projecting human traits onto systems that don’t in fact have those traits. So if I attribute beliefs and desires to the weather or to a thermostat, that’s a case of anthropomorphizing, I take it.

I suppose it’s not immediately obvious that that’s what’s going on when it comes to advanced AI systems today in a straightforward sense, because you might think, “Okay, people are attributing things like beliefs, desires, intentions, personalities to ChatGPT.” But somebody might argue, “Look, it’s just the case that we’re dealing with sophisticated intelligence systems. So it’s nothing like these standard cases of anthropomorphizing. They actually do have the kinds of psychological states that people are attributing to them.” Are you assuming that view is wrong or are you using this term anthropomorphizing more expansively?

Henry (26:01): I’m using it more expansively. Sarah Shettleworth, who is one of my absolute favorite titans of animal cognition research, defines anthropomorphism as projecting or attributing human-like qualities onto non-human systems—in her focus on animals, but onto non-human systems more broadly—usually with the suggestion that such attributions are illegitimate or inappropriate.

But I don’t think it’s necessarily baked into the concept of anthropomorphism that it’s wrong. When I talk about anthropomorphism in this sense, I’m talking about it in the sense of basically just attributing human-like qualities to non-human things that may in fact have them.

My own view—I’ve got a new paper, a new preprint called “Three Frameworks for AI Mentality”—I argue that I think probably at least certain minimal cognitive states, things like beliefs and desires, are quite appropriately attributed to LLMs in this case. Not to be clear, all beliefs, all desires, but there are contexts in which it makes sense to say ChatGPT believes P or ChatGPT desires Q. I think particularly on things where there’s been specific reinforcement learning towards a given outcome—like ChatGPT really doesn’t want to tell me how to make meth, for example, because that’s something that’s been specifically reinforced to avoid doing.

I agree there are some cases in which at least minimal mental states are appropriately attributed to generative AI systems. That said, I think I’m also a big fan of Murray Shanahan’s idea that a lot of the correct and most informative way to interpret LLM outputs is that they’re role-playing characters. I think this is plausible when you think about how, particularly if you’ve spent any time interacting with base models, you basically give context cues to them about the kind of role you want them to occupy and then they play into that role—but they’re not robustly occupying that role. You change the context cues, they can switch into a different role. So I think a lot of LLM outputs are better understood as role play—they’re playing a character. But yeah, some mental states are completely appropriately attributed to them, I think.

Dan (28:13): Yeah, it’s interesting. And people talk about them as well, I guess it’s somewhat connected to this role-playing idea, as having personalities. When OpenAI released ChatGPT o1, there was apparently this big uproar because people liked the personality of 4.0, which was the model in widespread use preceding ChatGPT o1. And the idea is, just as human beings have different personalities, which are going to manifest themselves in terms of how you’re talking to them and we might use words like “how warm are they?”—similarly, people I think find it quite natural to attribute personalities to these chatbots as well.

I take it, I mean, I think we should say something about potential benefits and opportunities, really great things about social AI, but just to anticipate one of the worries, I think one of the worries is people are forming relationships with these systems at the moment and potentially in the future. And it’s almost like that’s psychotic in as much as there’s nothing there really on the other side.

I mean, I take your point about you might attribute minimal kinds of beliefs. You might take what Dennett would call or would have called the intentional stance towards these systems. But I think many people have the intuition, which is one of the things that’s just deeply, deeply troubling about people forming a relationship with one of these systems, is they might cash it out by saying they’re not sentient, they’re not conscious, but they might also say related to that, they don’t have any of the traits that human beings genuinely have, which are a necessary condition for forming a meaningful relationship. What’s your view about that kind of worry?

Henry (29:57): Yeah, I think it’s an interesting and important worry. Here’s how I would frame it. There’s a certain view that says, “Look, these human-AI relationships are a contradiction in terms because relationships involve two relata and they’re dynamic by nature. And these systems don’t really have mental states. They’re not really reciprocating any feelings. They’re not really making any demands in the same way that is characteristic of reciprocal relationships.”

Look, you can certainly define relationships that way, but I think there are lots of contexts where we talk about relationships with only one relatum. Think about the fact that so many people say they have a relationship with God. Now, we can debate how many relata there are in that case. But I mean, I think most of us would say at least some people who think they have this really deep and meaningful relationship with a supernatural being—there’s not really a supernatural being there. That doesn’t mean that that is not a psychologically important relationship in their lives.

Or I guess more broadly, you could think about relationships with pets. Now, of course, in some cases, we can look at clear reciprocation—people have deep and established relationships with dogs and cats and so forth. But you also have people talking about relationships with their pet stick insects or their pet fish, where it’s just much less clear that there is any kind of rich two-way connection there.

That said, I think there’s a broader worry here that I’ve called the mass delusion worry about human-AI relationships and social AI, which is that I think a lot of people are just going to look at this, particularly if it becomes a more pervasive phenomenon and say, “Has everyone gone mad?” Because there is no other person there, there is no other. These people are investing huge amounts of time, emotional energy, potentially money into these pseudo relationships where there’s no one on the other side.

I think that’s a case where questions about AI mentality maybe become more important. You might say, whether or not the mass delusion worry is on the right line is going to depend on the degree to which these things do have robust psychological profiles, really instantiate kinds of mental states.

I think that’s another related worry, which is—again, we can talk more about the specific psychosocial risks—but another worry I’ve heard is that even if you could prove to me tomorrow that human-AI relationships are generally beneficial for users, they make them more connected to people around them, they do all these other good things, they just can’t instantiate by their very nature the same kinds of goods that human-to-human relationships instantiate. This is very much a philosophical rather than psychological point. It’s like they’re just the wrong kind of relationships to have the welfare goods that we associate with human-human relationships.

Dan (32:36): Yeah. Just on the God analogy, I mean, as an atheist, I really do think people are taking themselves to have a relationship to something that doesn’t exist. And from my perspective, that’s a deeply objectionable aspect of the practice that they’re engaged in.

The analogy with pets is very interesting. One of my most unfashionable opinions—maybe we’ll have to cut this bit out because it’s going to be reputationally devastating—I do think it’s kind of, I find it a little bit humiliating how some people relate to their pets, the degree to which they anthropomorphize them. That’s not to say that you can’t have deep relationships with pets. Clearly you can. And I love dogs, for example, but I think there are cases where people treat their dog or their cat as if it’s a person. And it’s not that I think it’s psychotic, but I think there’s something objectionable about it. I think they’re making a deep mistake. And even if they’re getting psychological benefits from that, there’s something deeply, almost existentially troubling about that kind of relationship.

And I can see that mapping onto the AI case, except I take it in the AI case, and this connects to one of our previous episodes, people’s intuition is, well, with a dog, maybe it’s not cognitively sophisticated, but just about everyone is going to assume these days at least that dogs are conscious, sentient is the term that is often used in popular discourse. And that does change things in a sense. Dogs really care about things. There’s something it’s like to be a dog. Whereas with AI systems, I think many people have the intuition that they might be informationally and computationally sophisticated, but there are no lights on inside. There’s no consciousness there. And that changes things again.

Anyway, that was just my sort of immediate reaction to these analogies.

Henry (34:33): I think I’m sympathetic. I think certainly when you hear people talk non-ironically about fur babies and so forth, it does seem like there is some degree of maybe inappropriate allocation of emotional and relational resources into certain kinds of relationships. And I say that as someone who adores animals, has had dogs most of my life.

Maybe another couple of examples of non-standard relationships. I think a lot of people would say they have ongoing relationships with deceased relatives. Particularly if you’re coming at it from a spiritual point of view where you believe that your deceased relatives are looking over you, seeing what you’re doing, or whether you understand it in some kind of more—animist isn’t quite the right word—but you still think about “my ancestors are smiling at me, Imperial. Can you say the same for yours?”—the famous line from Skyrim. A lot of people have this sense of “Yeah, my ancestors are looking over my shoulder. I’ve got to live up to their expectations for me.”

So I think there are lots of interesting cases where we do have these relationships that don’t meet the canonical definition of this highly dynamic, reciprocal, ongoing kind of relationship. Also, I’ve got friends that I would consider myself to have a valuable relationship with that I haven’t spoken to in three years in some cases.

So I guess all of which is to say, the category of things that we call relationships is weirder and bigger than might meet the eye if the only notion of relationship you’re working with is like, “Yes, my wife or my friend who I go to the pub with three times a week.”

Dan (36:13): Yeah, that’s interesting. Okay, I mean, I really want to spend probably the bulk of the remainder of this focusing on potential threats and dangers here. But I think it is worth stressing that this as a technology, social AI, does have enormous benefits and opportunities associated with it. I take it even if you think that there’s something troubling or objectionable about people having these relationships, some people are in such dire circumstances of loneliness, of estrangement from other people for whatever reason, potentially because they’re in old age where I think issues of loneliness are really prevalent. And clearly, under those conditions, I think social AI can be enormously beneficial in as much as it just makes people feel much better than they otherwise would.

It also seems to be the case that, at least in some cases, people are using social AI to hone skills and acquire confidence that they then use when it comes to interacting with people in the real world. And I can completely imagine that is a current benefit of this technology and it’s likely to be a benefit which in some ways will get amplified as we go forward in the coming years and decades. Are there any others that are missing there in terms of real positive use cases of this technology?

Henry (37:39): Yeah, I think you’ve nailed some of them. One thing I would just really stress here, there’s a move that drives me nuts, where people basically say, “Well look, even if you’re using chatbots to alleviate your loneliness, it’s not fixing the root cause.” It’s like, okay, well, okay, yeah, go away and fix the root cause—solve human loneliness. Please come back and tell me when you’ve fixed it. So classic case of letting the perfect be the enemy of the good. Loneliness is a pervasive problem and arguably one that’s getting worse, although the social science of this is messier and more complicated than you might think. But we don’t have a magic wand that can cure loneliness.

And the question then becomes, I think, is this actually just in the short term making people’s lives better or worse on average? And I think the data here, this was a big surprise to me, the limited data that we have suggests that most users of social AI, at least that show up in studies, report significant benefits from this.

I can just quote from a couple of studies. This was a study by Rose Gingrich and Michael Graziano from 2023 looking at Replika users. They found that users generally reported having a positive experience with Replika and judged it to have a beneficial impact on their social lives and self-esteem. Almost all companion bot users spoke about the relationship with the chatbots having a positive impact on them.

Another interview-based study in 2021 found that most participants said that they found Replika impacted their well-being in a positive way. Other sentiment and text mining analysis studies have found really similar patterns.

So right, I think the data right now, which is very limited, got to stress, it’s very imperfect, supports the idea that most users report benefits from the system, net benefits.

Now, just to flag some of the problems: these are typically self-selected subjects, they’re cross-sectional studies so we’re not looking at the long-term impacts of this technology on their lives, and we’re relying on self-report measures. If you asked me “Does video gaming have a positive impact on my life?”, I’m absolutely going to say yes. You ask my wife—I think she’ll probably say yes as well—but the point is that people tend to justify their own life decisions so they’re unlikely to say “Yeah, this thing that I dedicate a dozen-plus hours a week to—yeah, it’s really bad for me. I shouldn’t do it.” That would require a level of brutal self-honesty that maybe people don’t have.

So I just say that as a caveat, but I equally think we can’t ignore the data points that suggest that most social AI users do experience net benefits currently.

That said, I also think we should be very aware of a whole host of potential downsides. You mentioned this idea that using these tools could help people cultivate social skills or regain social confidence. But equally, of course, you have the flip side worry about that—this idea of de-skilling that comes up in a lot of technology.

De-skilling, in case anyone’s not familiar with this, probably one of the most widely studied domains of de-skilling is in aviation, where we’ve had serious airplane crashes that have been linked to pilots’ over-reliance on automated instruments rather than being able to fly just manually. And this has led to them not acquiring relevant kinds of piloting skills such that when the instruments go wrong, they don’t know what to do.

And you might think something similar could happen socially. If you have this generation of, particularly, I think we’re looking at it through the lens of young people—if you have a generation of young people who are primarily interacting, a huge proportion of their interactions happen with bots that are maybe more sycophantic than humans, they’re always available, they never interrupt you and say, “Actually, can we talk about me now for a bit?” Or “Look, this is very interesting to you, but come on, give me a turn”—they don’t accurately recapitulate the dynamics of human-to-human interaction. You might worry that that would lead to people failing to acquire relevant social skills.

Another big worry that I think is a nice intersection of our interests is the potential for these things to be used for manipulation and persuasion. Particularly again, I think young people is quite salient. If I’m a young person with a chatbot, I’ll give you a really simple example. If I’m going to ask my AI boyfriend or girlfriend, “I’m getting my first mobile phone next year. Should I get an Android or an Apple phone?”—well, I mean, that’s a lot of power you’re giving to the bot.

Now, of course, this is a problem with LLMs more broadly. You might think that that’s a lot of leverage that’s in the hands of tech companies. But I think, and I think we’re probably broadly on the same page here, to the extent that you think that in domains like moral and political views in particular, social factors are incredibly powerful, and then you think the AI love of my life, my AI girlfriend is telling me, “You should vote for Trump” or “You shouldn’t vote for Biden” or “You shouldn’t listen to that kind of music, it’s uncool” or “You should be interested in this kind of music”—all of this stuff could exert a much bigger influence than just asking ChatGPT, precisely because you’ve got those social dynamics in place.

Dan (42:46): Yeah, I think that sounds exactly right. I mean, I think probably the complicating factor there is if you imagine a corporation that wants to maximize market share, maximize profit by producing technologies, the function of which is to satisfy people’s social needs specifically, and then it comes to light that there’s also this manipulative, propagandistic agenda at the same time—either because in the midst of your loving relationship with your AI, it starts saying, “Well, you should vote for Reform or Labour at the next election,” or because there’s some news story that comes to light which shows that there’s nefarious funding or influence behind the scenes—I can imagine that would be really catastrophic for the business model of the relevant corporation. And that kind of thing, I think, is just really difficult to anticipate.

Henry (43:31): Yeah, potentially. So I think there are some incentives that companies have not to be too crude about leveraging human-AI relationships for cheap political or commercial gain. But I can also imagine in some contexts—I think not to pick on China here specifically—but I mean, social AI is huge in China. We haven’t talked about that, but there’s a service called Xiaoice that has, according to some estimates, several hundred million users. They themselves claim they have 500 million users. It’s a bit more complicated than that because Xiaoice is a whole suite of different services. We don’t know what proportion of those have active ongoing relationships with the chatbot girlfriend/boyfriend component.

But part of Chinese AI regulation says that the outputs of generative AI systems have to align with the values of communist China, the values of socialism. So you can imagine a generation of young people who have these deep relationships with AI systems. Those AI systems, for legal compliance reasons, have to basically align with the broad political values of the incumbent regime. And they will dissuade or deter people from maybe exploring alternative political views as a result. So maybe that’s the more subtle kind of influence, rather than just like, “You should vote for Trump because he gave our social AI company X million dollars.”

Dan (44:57): Yeah. And I think that’s just part of a broader fact, which is that propaganda and manipulation and so on, they do in fact just work very differently and sometimes much more effectively within authoritarian regimes than they do in more democratic, liberal ones.

On this issue then of potential costs that the use of these technologies might generate when it comes to then interacting with people. So I think you mentioned the fact that these systems, they’re almost going to be more sycophantic by design, at least if you assume that at least up to a point, that’s the kind of AI agent that people are going to enjoy interacting with. And they don’t have all of the sources of conflict and frustration and misery that go along with human relationships.

So one issue is to the extent that people start using social AI much more, they’re going to lose precisely that skill set, which is adapted to dealing with other human beings who haven’t been designed to cater to your specific social preferences.

But I take it there’s also then an issue of motivation, where it’s sort of, why would I go out there into the world and spend time with human beings with their own agenda and their own interests who are frustrating and annoying and often insulting, etc., etc., when I could immerse myself in this world in which it’s pure gratification of my social, even my romantic, sexual desires?

I think that then connects to something else. When I think about this area of social AI, the thought experiment that seems most salient to me is the experience machine—Nozick’s idea—would you plug yourself into some machine where you’re getting all of the wonderful pleasure that goes along with certain kinds of desirable experiences, but none of it’s real? You’ve just been fed the relevant kinds of neural signals to simulate those kinds of experiences.

I think many people think, no, because there’s much more to a meaningful life than merely the hedonic, affective associations or things that are associated with the satisfaction of our desires. We actually want the reality that goes along with satisfying our desires. And I think similarly, when it comes to social AI, the intuition is there’s something similar going on there where it’s okay, you might be getting off socially, romantically, sexually, and so on, but it’s fake. It’s not reality. And so even if you’re the happiest person in the world from a certain definition of what happiness amounts to, which is purely hedonic, there’s nevertheless something deeply troubling about that kind of existence which we should steer ourselves away from.

What are your thoughts about that? Do you think that analogy with the experience machine thought experiment makes sense? And do you also buy the intuition that we shouldn’t plug ourselves into an experience machine, so we also shouldn’t plug ourselves into very pleasurable forms of social AI?

Henry (48:06): Super interesting. I’m going to tease apart two different threads here. The first idea is this idea that these are just going to be easier alternatives. And I think the useful lens for thinking about that is the idea of superstimuli. This is a term that gets thrown around in a lot of different domains. We hear it in relation to food—that modern junk food is like a culinary superstimulus that basically just gives you far more rewarding signals associated with fat and sugar than anything you would find in our evolutionary environment. And this has sometimes been suggested as the best explanation for the obesity crisis—the fact that basically modern food is just so delicious, it just maxes out our reward centers so effectively that it’s just really, really hard to go back to eating whole grains and leafy vegetables cooked simply.

We see the same debate around pornography, the idea that pornography is kind of a sexual superstimulus. I’ve also heard the superstimuli used to refer to things like social media or short-form video. Try reading a Dickens novel if your brain has been fried by a decade of six-second YouTube Shorts.

Now we don’t need to relitigate that, but I think all of those debates, this idea of superstimuli, the idea of something that is just far more rewarding than the kind of natural, maybe more wholesome version of it—I think that’s a really powerful lens for thinking about social AI and raises some significant concerns.

But I think that’s separate from the second point you raised about the idea that it’s fake, that the actual thing that we value is lacking in these kind of contexts.

For what it’s worth, I am far more conflicted on the experience machine, I think, than you. There is a sense in which I think I would be very tempted to take the experience machine, although maybe that has to do with the fact that I’m pretty sure we’re in a computer simulation right now. I’m hardcore into simulationist territory.

I also think there are maybe some reasons we should expect our judgments about the experience machine to be maybe skewed. We have this idea of real goods versus ersatz goods, where real goods are going to be more enduring, more reliable. So that might create within us a preference for real goods over fake goods. But of course, in the experience machine, you’re guaranteed—these experiences will keep on going. You’re going to have this dream life in the matrix or whatever, that’s not going to be yanked away. So I think there are intuitions that possibly make us more averse to experience machine type lives than we possibly should be.

Dan (50:54): Well, there’s also, just to really quickly interrupt on that point, Henry, sorry to interrupt, but there’s also, I think, as I mentioned earlier on in connection with social AI, this reputational thing going on where I think there’s a tendency to judge people harshly if they choose the experience machine, potentially because we think somebody who’s going to prioritize positive hedonic experiences wouldn’t make for a good cooperation partner or something. I think we’re constantly evaluating, would this be a good leader, a good friend, a good romantic partner, a good member of my group? And if someone seems to suggest that they would prioritize mere hedonic manipulation or however exactly to understand the experience machine, we judge them harshly. And I think anticipating this, then people are inclined to say, “No, I wouldn’t choose the experience machine. I would choose the real thing.”

I just also want to really quickly talk—you went over this quickly, but I think it’s very interesting. You said you think we probably are living in a simulation. So this is the classic Bostrom-style argument that says, well, we’re likely to be able to have the technology to create simulated worlds. If that’s true, then there are going to be many, many more simulated realities than base reality. So just statistically or probabilistically speaking, we should assume ourselves to be in a simulated reality.

But as I understand that, that’s in and of itself, it’s not obvious to me why that would influence your response to the Nozick experience machine scenario. Because even if you think it’s true that we are living in a simulation in some sense, I take it what’s distinctive about the experience machine thought experiment is it’s not just that you would be living in a simulation. It’s well, for one thing, it’s a simulation within a simulation, but it’s also a simulation which has been tailor-made in a way to satisfy your desires and that feels a little bit different from what I’m assuming you take to be the case when it comes to us living in a simulation. Did any of that make sense?

Henry (52:55): Yeah, it makes perfect sense. Okay, here we can get really spicy because I think I take Bostrom’s classic simulation arguments quite seriously, but at the risk of having viewers think I’m completely bonkers, I’m also genuinely—I think there’s a chance I’m already in an experience machine. So this is just speaking for me.

Without wanting to get too sidetracked, I think so many features of my life—it’s hard to put this without sounding egotistical, but I just feel like my life has just been absurdly fortunate. I’ve lived in a really interesting time in human history. My life has been blissfully devoid of serious unpleasantness. Not to say that it’s been perfect, but most of the challenges I’ve encountered in life have been interesting, relatively tractable things. Look, here I am. I’m getting to live the life of a Cambridge philosopher at the very cusp of human history where we’re about to explore AGI. It seems like this is the kind of life, genuinely, that I might pick.

Whereas I feel like by rights I should have been a Han peasant woman in third century AD China. So I’m slightly joking, but not entirely. I do think there’s a serious chance that at some level I am already—my life consists of some kind of wish fulfillment simulation. So I don’t know, maybe that gives more context.

Dan (54:17): Interesting, okay, that’s a spicy take. We should return to that for another episode because that’s fascinating. And actually, it’s also very philosophically interesting. I think you’re right, when you start thinking about things probabilistically in that way, yeah, the fact that you’re having a great life might provide evidence for this kind of experience machine scenario. But I feel like we’re getting derailed from the main point of the conversation now, which I think is probably my fault for double clicking on that point.

Henry (54:45): So there’s this question about whether the fact that there’s maybe no one conscious on the other end of the conversation—to what extent does that mean that it fails to instantiate the relevant goods that people care about? To what extent does that make it fake?

I think one small thing, I think there’s a perfectly viable position that says, look, a perfectly viable philosophical position that says, look, if there’s no consciousness there, then no matter how much fun these relationships are, they’re not really valuable. I mean, I can sort of see some arguments for that.

But maybe I’m also influenced here by the fact that really since my early adolescence, so many of the relationships I’ve had—I’m talking relationships in the broad sense, friendships and so on—have been with people whose faces I never saw. I spent from my early years a lot of time in pre-Discord chat rooms on services like ICQ back in the day, on online video games, massively multiplayer worlds, before the age of streaming video, where I would form these really valuable relationships with people I only interacted with via text. And in some cases, particularly in video games, we didn’t really discuss our personal lives at all. We were just interacting in these virtual worlds.

Now you could say, yeah, but there really were people, conscious people on the other end there. And sure, yeah, but I’m not sure how much of a meaningful psychological difference that makes for me in terms of my experience of those things. It seems to me that so many of the goods that we get from relationships don’t consist in this deep meaningful connection with a conscious other but consist in things like joint action—in the case of a video game might be going on raids together, having some fun banter together, discussing politics.

I spend a huge amount of time—or not so much these days, back in the day I spent a huge amount of time arguing on Reddit. Would it have made a difference to me if I knew that person I had a really good long political debate with on Reddit, that they were a bot? Well I think it probably would, but I’m not sure whether it should, if that makes sense. It was a valuable discussion for me and maybe it’s just my own prejudice that gets in the way then.

Dan (57:04): Interesting, yeah, I definitely don’t have the same intuition. I think if I were to have what I thought of as meaningful relationships and then discover that actually I wasn’t dealing with a person as I understand it—

Henry (01:03:47): Yes, okay, if you found out that you were talking to a chatbot online and you thought they were a person, that would be dismaying. You would feel bummed out to some extent.

It’s not clear to me that that is primarily to do with lack of consciousness though. In some ways, I think it’s more to do with a loose set of considerations around agency and identity. Talking to a chatbot feels like something like a social cul-de-sac currently. There’s no one going to go away with changed views that they’ll carry forward into discussion with other people as a result of the conversation that we’ve had. It can feel sort of masturbatory in that sense.

But I think if you can think about AI systems, if we imagine AI systems as something like more robust social agents—so you might chat to a given social AI one day and that social AI will be able to carry forward any insights it gleans from that conversation into interaction with other people. I don’t know, as you beef up the social identity of these things a bit more so that it’s not just these masturbatory cul-de-sacs, then my intuitions start to weaken a bit. Maybe they can be valuable. If I’m talking to a chatbot that speaks to other people and can carry forward those insights, maybe there is some value.

Dan (01:05:06): Yeah, that’s so interesting. I feel like there’s a million different things we could be talking about here. And we should say we’re going to have other episodes where we return to social AI, where we bring on guests and so on.

Maybe two things to end on. One thing we’ve already touched on, and I think it connects to what we’ve just been saying, but takes it even further. One commercial use of social AI, at least as I understand it, pretty fringe use, but not a completely non-existent one, is I think you earlier on called them grief bots? Basically people using AI technology to produce a system that—I don’t exactly know how to describe it, but that exemplifies the traits that they associate with somebody, a loved one, a family member, a spouse, a friend who has passed away.

I mean, that it’s almost like that’s been cooked up purposefully for moral philosophers because it introduces so much, not just weirdness, but moral complexity. I mean, I take it there’s the general baseline issue, which is you’re forming a relationship of some kind, you’re interacting with an AI system, and that’s weird. And then it’s a weirdness which is massively amplified by the fact that you’re interacting with an AI system, but the relationship in some sense is grounded in the perception that you’re somehow interacting with somebody who’s now deceased.

I don’t even know to be honest how to describe it, but I understand some people are doing this. So what’s your take about what’s going on?

Henry (01:06:51): Yeah, so I expect griefbots to be one of the big applications of social AI. As you mentioned, it’s relatively niche at the moment. And I think a lot of companies are very scared to go anywhere near this, but I can see that changing quite rapidly.

Just for context here, it’s worth noting that chatbots fine-tuned on real-world individuals—there’s a lot of them that suggest they can be really accurate in terms of capturing the kinds of things that people would say, their modes of conversation and so on.

So one famous experiment was the DigiDan study. This was using GPT-3, so really primitive language model by modern standards. But a group of people including David Chalmers, Anna Strasser, Matt Crosby and others basically fine-tuned GPT-3 on the works of Daniel Dennett—I’m sure most of your audience know Daniel Dennett, one of the greatest philosophers, sadly died a couple of years ago. This was shortly before his death though that he did this.

And then they got Dan’s friends and colleagues to pose questions to both Dan himself and the DigiDan bot. And they generated four responses from the chatbot and they had Dan’s response there as well. And users, Dan’s friends and colleagues, were pretty much close to baseline, close to bare chance, telling which responses were from Dan versus the chatbot.

So this is just to emphasize that appropriately fine-tuned chatbots could do a really good job of simulating the kind of things that a person would say in response to a given query.

So let’s just imagine that you do have this category of griefbots that can provide an accurate simulacrum of a deceased person. Well, I mean, at the risk of spicy take, I can see lots of really positive use cases for this.

I’ve actually even said to Anna Strasser, one of the people who did the DigiDan study, that if I get hit by a car tomorrow, I’ve discussed this with my wife as well, then she should absolutely have my permission to fine-tune a bot on me. And my wife will give her a perhaps lightly edited or curated set of my correspondence, my social media presence and so on, so that then my kids can talk to this simulacrum of me if they choose to.

I can imagine there being real value, know, if my son or daughter is like 17 and considering, should I go to law school or medical school? I wonder what my dad would have thought of this. That seems like a really potentially positive use case.

So I think griefbots are a really interesting area. And there’s interestingly some studies looking at people’s use of griefbots for therapeutic purposes—if there are conversations you always wanted to have, but didn’t get the opportunity to have perhaps because a spouse or a parent died suddenly, that this could, one phrase that’s used, offer a soft landing for the grief experience. So there are loads of really interesting positive use cases there.

But equally, as you say, it’s an absolute minefield. And I think to a lot of people the whole idea of griefbots just feels like something from a Black Mirror—well, literally there was a Black Mirror episode about this. And I think that it raises some fascinating questions about how that changes the nature of the grieving process, how it changes our very concepts of mortality.

If someone’s physical body can die, but there’s this sort of echo, digital echo ghost of them that is still around, how does that reshape our views about these things?

There’s also some interesting parallels. I had a student who wrote a great dissertation or a great essay about integrating this with the idea of communing with ancestors, which is obviously a really common feature in many different societies, where you might ask your ancestors—we touched on this earlier on—you might ask your ancestors for guidance on difficult questions. Could griefbots be a way of making that into a more concrete experience?

There’s a second angle here, which is even spicier and again, will make viewers think I’m even more of a weirdo, which is: could this actually offer some kind of form of immortality or some kind of life after death? Or continued existence after death, I should say.

Now, we could do a whole episode on this as well, but for what it’s worth, as someone who is very deflationary about personal identity, I’m big on the work of philosophers like Derek Parfit who say that in some sense the self is an illusion or the persistent self is a constructed self, there’s no deep matter of metaphysical fact about whether I survive or not. I could see a good case being made that in some sense, via an appropriately fine-tuned chatbot, there will be a form of persistence of me through that chatbot that might be relevant to mortality considerations.

Dan (01:11:36): Yeah, in some sense. It seems like there’s an issue here, which is using a chatbot to acquire knowledge about what a given person might have thought about a topic. And I take it there are going to be all sorts of questions that arise there. Like to what extent is it going to be reliable as a way of gaining insight into what that person would have thought about a topic?

Then there’s a question of using these systems not just to get that kind of knowledge, but to actually have a kind of relationship with the person. And then there’s something over and above that, which I take it you’re referring to then, which is the idea that in some sense, such a chatbot would carry on the identity of the relevant person. And I take it that relationship thing and that identity thing are connected.

I mean, certainly I think that’s where the real issues and lots of people’s queasiness arises, right? That’s when it seems like a bit of a leap, at least if we’re talking about chatbots as we understand them today. I can imagine AI systems of the future that aren’t merely getting really good at the statistical pattern recognition and prediction when it comes to bodies of text, but that are doing something more substantial when it comes to replicating the characteristics and traits of the relevant person.

But do you really think if you had a chatbot that had been trained on the text that you had produced that it would be in any sense a continuation of you?

Henry (01:13:08): Yeah, I mean, potentially. We could do a whole episode on personal identity here. But broadly speaking, being a bit crude here, the Parfittian perspective, Derek Parfit’s view, is that there’s a certain kind of relation you bear to your future self, a certain kind of psychological relation, that can come in varying degrees. And to the extent that we prioritize anything when we’re about survival, this relation R is the kind of relation that matters. And it can obtain to varying degrees.

Just as I could suffer a traumatic head injury and my behavior would change in some ways, but not others—that would be a sort of continuation of me in some ways, but not others. I think you could say the same for an appropriately fine-tuned chatbot.

Now, as you sort of implied, there will be things that it misses out there. We don’t talk about everything that is relevant to us. There’s more to our identity than just what we say. But again, that seems like a technological problem that, as we move to increasingly multimodal chatbots that can learn not just from what we say online, but how we live in the world, I think you can instantiate this relation, this relevant kind of continuation relation to increasingly strong degrees.

But I guess that’s the central point I’d say here: that survival in this kind of Parfittian view is a matter of degree and a matter of similarity and I don’t see why even if it’s imperfect—to the extent that a chatbot can capture really key features of my modes of interaction—that that’s a kind of survival. That’s a kind of continuation.

Dan (01:14:39): Interesting. Maybe we could end on this point. So I’ve been thinking quite a bit recently about how advances in artificial intelligence might gradually eat away at human interdependence. When people are thinking about the dangers posed by AI, there are the classic loss of control, catastrophic misalignment dangers that we’ve talked about previously. There are also dangers to do with elites, political factions, authoritarian regimes, the military using advanced AI to further objectives in ways that are bad for humanity.

But I think there’s also a category of dangers associated with AI systems doing what we want them to do, satisfying our desires, but in ways that have really knock-on bad consequences. And it’s easy for me to see how social AI might be a little bit like that, in as much as so much of our understanding of the human condition, so much of the societies that we inhabit is bound up with interdependence. We depend upon other people. We depend upon other people for friendship, for labor, for sex, for romance, for art, for creativity and so on.

And it seems like a very plausible path when it comes to advances in AI is these AI systems are just going to get better and better at doing everything that human beings do and in fact are going to get better than human beings at doing all of those things, including when it comes to satisfying the social needs in the way that we’ve talked about in this conversation.

And to the extent that that’s true and we become more and more reliant on these AI systems and less and less reliant on other people, that kind of human interdependence fades away. I think there’s something—I mean, there’s something disturbing about that just from the perspective of thinking about what it means to be human. But maybe that’s not really a serious philosophical worry. That’s just an emotion.

But I also do think, the way I think about it is a lot of the human alignment problem—like how do we align our interests with one another and build complex societies—is precisely this interdependence. Because we depend on other people, we have to care about them and we have to care what they think about us and so on.

And it seems to me one of the diffuse risks and long-term risks associated with social AI is precisely that, that as this technology gets better and better, it’s just going to erode that interdependence, which is really central to the human condition.

I realize that’s a massive thing to throw at you for the final question, but what are your thoughts about that? And then we can wrap things up.

Henry (01:17:15): Super interesting. Yeah, so I think you did this yourself, but I’ll separate out two different concerns here. One is the more philosophical question about, even if this works perfectly, even if we’re all very happy with this future society, has something of value been lost? I think it is a valuable question to ask, but it’s also one that’s hard to answer in a neutral sense. It really comes down to what is your conception of eudaimonia, human flourishing, in a deep philosophical sense.

And then there’s—but I felt like that wasn’t the core of your question. You were asking something more about negative externalities, perhaps more about negative knock-on effects. And I think that is absolutely something I’m also worried about basically just because of social media.

Now I realize this is a debate where you have your own very well-developed positions, but I’ll just offer a quick parallel of two technologies. One is violent video games or video games in general. I think we’ve probably discussed this before, but back in the 90s, there was massive moral panic around negative knock-on effects. The idea that kids growing up in the 90s playing Doom or GTA would turn into moral monstrosities as a result of being exposed to relatively accurate simulated violence.

I don’t think that was a stupid thing to worry about. It just turned out to not be a major concern. It turns out that’s not how the brain works, that we didn’t see massive negative externalities associated with exposure to violent video games.

But by contrast, social media, opposite story. I think there was relatively little panic early on in the days of social media, in the early days of MySpace and Facebook. In fact, I think most of the commentary about social effects of these things was quite positive—the idea of bringing people together. There was an interesting debate that’s quickly been sort of consigned to the dustbin, quickly been memory-holed. People, I remember in 2010, people were talking about how social media meant the collapse of epistemic closure, how these epistemic islands would be all beautifully linked up through conversations on social media. And we’d be able to talk to people with different political views from us.

And that’s basically not happened, that social media exacerbated some of our worst social tendencies, possibly contributed to echo chambers and so forth. I’m aware that you have a slightly more optimistic view here, but I’m just offering that as a parallel for a case of social technology that I think at least in many people’s view has had significant, largely unforeseen negative consequences.

And I think that’s absolutely a legitimate source of worry about social AI. I’m not sure I’d necessarily frame it in terms of dependency or interdependence. I mean, I think different people make different choices about how much they want to depend on others. I don’t think it’s obvious that someone living alone on a ranch in rural Texas or whatever, they grow their own food, whatever, they’re relatively autonomous and independent—it’s not clear that they’re not living a great life. It seems that you can have valuable lives with varying degrees of social interaction and dependency on others.

But at the same time, I do think there are possible dangers, very hard to predict, associated with potentially people getting into islands of social activity where it’s just them and their coterie of AI friends and they don’t see the need to interact with others. The kind of subtle influences that could have on things like democracy, on society—there’s absolutely scope for concern.

Dan (01:20:52): Yeah, okay. We’ve opened several cans of worms to conclude the conversation. I’m aware, Henry, that you’ve got a place that you need to be. So that was so much fun. So many issues and questions, which I feel like we didn’t really even scratch the surface of, but we’ll be back in a couple of weeks to talk about more of these issues. And then over the future, we’re going to bring on various kinds of guests and experts in social AI and connected issues. Was there anything final that you wanted to add before we wrap up, Henry?

Henry (01:21:24): I guess just a couple of quick reflections. Firstly, I think for any young philosophers or young social scientists listening, I think this is just such a rich and underexplored area right now. There are so many interesting issues ranging from griefbots to digital duplicates—models fine-tuned on real-world individuals who are still alive—to issues around de-skilling, dependency, mental health, atomization, loneliness, intellectual property, influence, motivation, persuasion. There’s enough for several dozen, hundreds maybe of PhD dissertations on this topic. So I think it’s just a really interesting and valuable area to work on.

And that’s not even getting into the meatier philosophical issues we sort of just touched on briefly around personal identity, what it means to be human, flourishing, the good life. So I just think this is a really valuable area.

Also worth quickly promoting that I am a unit editor for an Oxford University Press journal series called “AI and Relationships.” It’s called the Intersections Journal Series and I run a project in there. So if any young philosophers or academics have papers on this, feel free to give me a ping on Twitter or to my email if you’ve got anything you want to publish on this topic because I think, yeah, it’s an area where I’m really keen to start seeing more good research.

Dan (01:22:41): Fantastic, yeah. It’s a golden age for philosophy, which is why it’s a little bit strange when you look at so many of the things that philosophers are actually working on. But anyway, that was great. See everyone next time.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe