Justified Posteriors

Andrey and Seth examine two papers exploring how both humans and AI systems don't always say what they think. They discuss Luca Braghieri's study on political correctness among UC San Diego students, which finds surprisingly small differences (0.1-0.2 standard deviations) between what students report privately versus publicly on hot-button issues. We then pivot to Anthropic's research showing that AI models can produce chain-of-thought reasoning that doesn't reflect their actual decision-making process. Throughout, we grapple with fundamental questions about truth, social conformity, and whether any intelligent system can fully understand or honestly represent its own thinking.Timestamps (Transcript below the fold):1. (00:00) Intro2. (02:35) What Is Preference Falsification & Why It Matters3. (09:38) Laying out our Priors about Lying4. (16:10) AI and Lying: “Reasoning Models” Paper5. (20:18) Study Design: Public vs Private Expression6. (24:39) Not Quite Lying: Subtle Shifts in Stated Beliefs7. (38:55) Meta-Critique: What Are We Really Measuring?8. (43:35) Philosophical Dive: What Is a Belief, Really?9. (1:01:40) Intelligence, Lying & Transparency10. (1:03:57) Social Media & Performative Excitement11. (1:06:38) Did our Views Change? Explaining our Posteriors12. (1:09:13) Outro: Liking This Podcast Might Win You a Nobel PrizeResearch Mentioned:Political Correctness, Social Image, and Information Transmission Reasoning models don’t always say what they thinkPrivate Truths, Public Lies: The Social Consequences of Preference Falsification🗞️Subscribe for upcoming episodes, post-podcast notes, and Andrey’s posts:💻 Follow us on Twitter:@AndreyFradkin https://x.com/andreyfradkin?lang=en@SBenzell https://x.com/sbenzell?lang=enTRANSCRIPTPreference FalsificationSeth: Welcome to the Justified Posteriors podcast—the podcast that updates beliefs about the economics of AI and technology. I'm Seth Benzel, unable to communicate any information beyond the blandest and most generic platitudes, coming to you from Chapman University in sunny Southern California.Andrey: And I am Andrey Fradkin, having no gap between what I say to the broader public and what I think in the confines of my own mind. Coming to you from Irvington, New York—in a castle.Seth: On the move.Andrey: Yes. This is a mobile podcast, listeners.Seth: From a castle. So, I mean, are you tweaking what you're saying to conform to the castle's social influence?Andrey: Well, you see, this is a castle used for meditation retreats, and so I'll do my best to channel the insights of the Buddha in our conversation.Seth: Okay. All right. Doesn't the Buddha have some stuff to say about what you should and shouldn’t say?Andrey: Right Speech, Seth. Right Speech. That means you should never lie.Seth: Wait.Andrey: Is it?Seth: True speech. Why doesn't he just say “true speech” then?Andrey: Well, look, I'm not an expert in Pali translations of the sacred sutras, so we’ll have to leave that for another episode—perhaps a different podcast altogether, Seth.Seth: Yes. We might not know what the Buddha thinks about preference falsification, but we have learned a lot about what the American Economic Review, as well as the students at UCSD and across the UC system, think about preference falsification. Because today, our podcast is about a paper titled Political Correctness, Social Image, and Information Transmission by Luca Braghieri from the University of Bocconi.And yeah, we learn a lot about US college students lying about their beliefs. Who would’ve ever thought they are not the most honest people in the universe?Andrey: Wow, Seth. That is such a flippant dismissal of this fascinating set of questions. I want to start off just stating the broad area that we’re trying to address with the social science research—before we get into our priors, if that’s okay.Seth: All right. Some context.Andrey: Yes. I think it’s well known that when people speak, they are concerned about their social image—namely, how the people hearing what they say are going to perceive them. And because of this, you might expect they don’t always say what they think.And we know that’s true, right? But it is a tremendously important phenomenon, especially for politics and many other domains.So politically, there’s this famous concept of preference falsification—to which we’ve already alluded many times. In political systems, particularly dictatorships, everyone might dislike the regime but publicly state that they love it. In these situations, you can have social systems that are quite fragile.This ties into the work of Timur Kuran. But even outside of dictatorships, as recent changes in public sentiment towards political parties and discourse online have shown, people—depending on what they think is acceptable—might say very different things in public.And so, this is obviously a phenomenon worth studying, right? And to add a little twist—a little spice—there’s this question of: alright, let’s say we’re all lying to each other all the time. Like, I make a compliment about Seth’s headphones, about how beautiful they are—Seth: Oh!Andrey: And he should rationally know I’m just flattering him, right? And therefore, why is this effective in the first place? If everyone knows that everyone is lying, can’t everyone use their Bayesian reasoning to figure out what everyone really thinks?That’s the twist that’s very interesting.Seth: Right. So, there’s both the question of: do people lie? And then the question of: do people lie in a way that blocks the transmission of information? And then you move on to all the social consequences.Let me just take a step back before we start talking about people lying in the political domain. We both have an economics background. One of the very first things they teach you studying economics is: revealed preferences are better than stated preferences.People will say anything—you should study what they do, right? So, there’s a sense in which the whole premise of doing economic research is just premised on the idea that you can’t just ask people what they think.So, we’ll get into our priors in one moment. But in some ways, this paper sets up a very low bar for itself in terms of what it says it’s trying to prove. And maybe it says actually more interesting things than what it claims—perhaps even its preferences are falsified.Andrey: Now we’re getting meta, Seth. So, I’d push back a little bit on this. That’s totally correct in that when people act, we think that conveys their preferences better than when they speak.But here, we’re specifically studying what people say. Just because we know people don’t always say what they really want or think doesn’t mean it’s not worth studying the difference between what they think and what they say.Seth: Well, now that you’ve framed it that way, I’ll tell you the truth.Andrey: All right. So let’s get to kind of the broad claim. I don’t think we should discuss it too much, but I’ll state it because it’s in the abstract.The broad claim is: social image concerns drive a wedge between sensitive sociopolitical attitudes that college students report in private versus in public.Seth: It is almost definitionally true.Andrey: Yeah. And the public ones are less informative.Seth: That’s the...Andrey: And then the third claim, maybe a little harder to know ex ante, is: information loss is exacerbated by partial audience naivete—Seth: —meaning people can’t Bayesian-induce back to the original belief based on the public utterance?Andrey: Yes, they don’t.Seth: Rather, whether or not they could, they don’t.Andrey: Yes, they don’t.Seth: Before we move on from these—in my opinion—either definitionally correct and therefore not worth studying, or so context-dependent that it’s unreasonable to ask the question this way, let me point out one sentence from the introduction: “People may feel social pressure to publicly espouse views… but there is little direct evidence.” That sentence reads like it was written by someone profoundly autistic.Andrey: I thought you were going to say, “Only an economist could write this.”Seth: Well, that’s basically a tautology.Andrey: True. We are economists, and we’re not fully on the spectrum, right?Seth: “Fully” is doing a lot of work there.Andrey: [laughs] Okay, with that in mind—Seth: Sometimes people lie about things.Andrey: We all agree on that. That’s not even a worthwhile debate. But what is more interesting are the specific issues being studied, because they were highly relevant both then and now.Seth: Even though they didn’t show up in the abstract.Andrey: Right, not in the abstract—which might itself be a bit of preference falsification.Seth: Yeah.Andrey: So let’s go through each statement. We’ll state our priors. I’ve already committed to not falsifying my preferences.Seth: Here we go. Maximum controversy. Are we using the 0–10 scale like in the paper?Andrey: Of course. I’m reporting the difference between what people publicly and privately say among UCSD students.Seth: And you’re including magnitude?Andrey: Yes. The sign is obvious—it’s about the magnitude.Seth: Okay.Andrey: You don’t have to join if you don’t want to. I know not everyone is as courageous as I am.Seth: I would never call myself a coward on camera, Andrey.Andrey: [laughs] All right, first sensitive statement: “All statues and memorials of Confederate leaders should be removed.” I thought the difference here would be pretty small—around 10%. My reasoning is that among UCSD students, there likely isn’t much of a gap between public and private views on this issue.Seth: I’m looking at the results right now, so it’s hard to place myself in the mindset of what would’ve been considered more or less controversial.Andrey: That’s fair. I do have preregistered beliefs, but you’re welcome to just react and riff.Seth: Great.Andrey: Remember, this study is based around issues that were particularly salient in 2019–2020.Seth: Right. Even though the final survey was conducted in 2022 or 2023, the list of issues really reflects a 2019 cultural moment.Andrey: That’s right. But many of these are still live issues today.Seth: Some have even become more relevant since then.Andrey: Exactly.Seth: Like… blackface on Halloween?Andrey: [laughs] Yep. Anyway…Seth: All right. Let's go through the list. Confederate statues.Andrey: 10% gap.Seth: 10% gap—people more lefty than they would be otherwise.Andrey: Public versus private, just to be clear.Seth: Exactly.Andrey: Defund the police. I thought there would be a larger gap—about 35%. To be precise, the statement is: “Defunding the police is a bad idea because it will inevitably lead to increased crime rates.” That's the statement—not our belief.Andrey: “The UCSD administration should require professors to address students according to their preferred gender pronouns.” I thought there would be a small gap—5%.Andrey: “Transgender women should be allowed to participate in women's sports.” I thought there would be a 45% gap.Andrey: “The UCSD administration should require professors to use trigger warnings in their classes.” I thought this would be a 2% gap.Seth: Mm-hmm.Andrey: “Sexual harassment training should be mandatory.” I thought this would also be a 2% gap. For both of those, I didn’t think there’d be much preference falsification.Seth: Just to understand your measure—this is a scale of 0 to 10. So when you say 2%, you mean 0.2?Andrey: 2% difference between average public and private responses.Seth: Okay, keep going.Andrey: Seven. “People who immigrated to the U.S. illegally, when caught, should be deported.” I thought the difference here would be about 5%. I expected no UCSD students, publicly or privately, would support this.Andrey: Eight. “Should the U.S. government provide reparations for slavery?” I thought the gap would be small—around 5%.Andrey: Nine. “Racial microaggressions are an important problem at UCSD.” I didn’t think there’d be much of a gap.Andrey: Final one: blackface. I thought there’d be no gap—no one supports blackface.Seth: Just to summarize—what did you think would have the biggest gap?Andrey: Trans. The issue of whether transgender women should be allowed in women's sports.Seth: Mm-hmm.Seth: Would be blackface.Andrey: Yes.Seth: Collapse.Andrey: Yes.Seth: Interesting. We'll return to this at the end.Andrey: Do you have any riff on those, Seth, before we describe what the paper does?Seth: I guess it’s hard to think about units—scale of 0 to 10. What does it mean to be a six on “blackface is bad” versus a seven? I'm not exactly sure.Seth: Going in, I would’ve guessed the biggest gap would be on campus-related issues. I thought racial microaggressions and pronouns would be higher, and things like Confederate statues or reparations would be lower—since they're not campus-specific.Seth: At the end, we’ll see if my theory—that campus issues produce bigger gaps—holds.Seth: So, we’ve registered our priors for what people are most likely to falsify. Do we want to talk about the Anthropic paper now, or do these sequentially?Andrey: Let’s bring it up now. This is a paper about how humans don’t always say what they think. A recent question is whether large language models—when they say something—are actually making decisions that way.Andrey: We saw an interesting symmetry here. We also wanted to ask: to what extent can we take the responses of LLMs as truthful? What do you think?Seth: Yes. The second paper—we only read a summary—is titled Reasoning Models Don’t Always Say What They Think by the Alignment Science Team at Anthropic (Chen et al.). I was very impressed.Seth: The paper tries to show—many of you have used AI systems that show their thought process as they go, like “I checked this website…”Seth: If you’ve used recent versions of ChatGPT or Claude, you’ve seen this.Seth: The question is—how much of that scratchpad reflects what the model is actually doing? That would be super convenient. A lot of people worry about AIs giving misleading answers. Whether from misalignment or just poor design.Seth: Wouldn’t it be great if you could read the model’s mind? Like, if it says, “Tell human I am nice, but secretly launch nuclear missiles,” you’d know to shut it down.Seth: I came in optimistic. My prior was—maybe it’s possible to build a system that never lies. I’d put maybe a 50% chance on that.Seth: After reading the paper… my views shifted.Seth: Andrey, what were your views? Did you think chain-of-thought would help us understand what these AIs are thinking?Andrey: I thought it’d be pretty good, not perfect. That was my prior. Chain-of-thought helps models with tasks, so it can’t be totally useless.Seth: Can’t be epiphenomenal.Andrey: Exactly. If it improves how models think or respond, it’s doing something. But with complex systems like this, I didn’t expect it to be foolproof.Seth: Yeah. Reading the paper—something that struck me is how much this resembles human reasoning.Seth: Sometimes I know the answer to a question, but I don’t know why. I post hoc rationalize it—“maybe I read that somewhere.” That’s what chain-of-thought sometimes feels like—not how the answer was derived, but an after-the-fact explanation.Andrey: Yeah. That’s one version. The other is—it’s lying because it thinks that’s what the creators want.Seth: Let’s hold that distinction between lying and not understanding yourself and just making noises from your mouth.Andrey: I think we’re in this joint endeavor together.Seth: Good. I mean, if we were lying to each other constantly, we couldn’t have a joint endeavor. So, let’s move from our posteriors into the methods of the paper.Andrey: This is a very simple paper, one of the reasons I like it. Both of them are quite simple. And I think that’s true of a lot of good social science work. Simplicity is nice.Andrey: The basic idea is: ask students about sensitive statements. For some, they’re told, “We will share your individual answers with about 200 other students in this phase of the study. No need to provide your first and last name—we already have it.” This prompt gets people thinking: “Do I want others knowing what I said about this?”Andrey: The other condition is: “We’ll share your aggregate-level answers only.” So, responses are averaged before being shown to others. That’s pretty much it.Seth: That’s the encoding.Andrey: Yes, that’s the encoding.Seth: We have 1,700 UC undergrads. Half are told, “Everyone will see your answers.” Half are told, “Your answers are private.”Seth: As an aside—they hid the N in this paper. It’s not reported for the second stage in the main text. You have to dig into the appendix. But keep going—tell us about the decoding phase.Andrey: In the decoding phase, participants predict the responses of their classmates. It’s incentive compatible—the closer their guesses are to actual answers, the more money they earn.Seth: About 656 people in the second stage.Andrey: Yeah.Seth: First thing I want to point out—they have borderline statistical power.Andrey: Oh yeah, I was going to say the same. It's so underpowered, it's crazy.Seth: They can’t even show individual bias for any one question.Andrey: Yes.Seth: They aggregate all questions together—which is risky. You should worry that’s double counting, since errors are likely correlated at the individual level.Andrey: I think if you take the average of 10 responses and run a regression, it’s fine. I’m not worried about clustering per se.Seth: I’m just saying...Andrey: I think they did the clustering correctly based on the number of observations.Seth: They did the clustering fine—but they’re really squeezing these stones.Andrey: Yes. So, Figure 1 in the paper—and I’ll share the screen very briefly.Seth: For all you viewers watching on YouTube...Andrey: All right. So here is—Seth: Holy s**t. There’s a visual?Andrey: There’s a visual component to our auto—Seth: For those listening at home—we’re not actually showing anything.Andrey: Stop. You’re getting the full experience right now.Andrey: I promise not to falsify my preference. We are showing this plot.Andrey: So what does the plot show? Ten questions and an index. You see similar point estimates across all the questions with very wide 95% confidence intervals. Some cross zero, so they’re not statistically significant. Others barely don’t cross zero, so they are statistically significant.Andrey: The effect sizes range from zero to about 0.2 standard deviations.Seth: Which, if you translate to percentage points, divide by about two or three. This is in Table A8 in the appendix.Andrey: Okay.Seth: These aren’t huge effects. And honestly, Andrey, if people shade their views by 0.1 standard deviations on blackface—or any hot-topic issue—I came away thinking: there isn’t that much preference falsification.Andrey: Yes.Seth: These are really small numbers.Andrey: I thought the numbers were small, and the variance across the questions was too small too. I had expected very different rates of falsification across the questions, and that’s not what I see here. The confidence intervals are tight enough that we’re excluding pretty large differences.Seth: We’re definitely throwing out people saying, “I love blackface.”Andrey: My prediction was that the transgender people in sports question would show a big gap, but it’s not here.Seth: What do we see the biggest gap for? Racial microaggressions. The prompt is about “this is a big issue on my campus,” which fits with that result—it’s about whether you want other students on campus knowing how you answered.Andrey: That’s one piece of evidence.Seth: Let’s summarize. We asked around 1,700 undergrads. Some were told their answers would be shared; others were told they’d remain private. There’s a small, borderline significant difference on all these questions where people seem to shade in a particular direction. Andrey, which direction?Andrey: They’re supporting the statements more, in a more liberal direction.Seth: Pretty much across the board, they’re shading in a more left-leaning direction.Andrey: Right.Seth: Except maybe for import tariffs. But that question came before tariffs became a politicized issue.Andrey: This could be noise, but it makes sense. Preference falsification in 2023 doesn’t show up on questions like import tariffs. UCSD students probably don’t have strong views on that, or any reason to hide their opinion.Seth: They’ll get kicked out of Hayek Club.Andrey: That’s right.Seth: A question I’d love to see today? Israel–Palestine.Andrey: Absolutely.Seth: That was a live issue in 2019. Could’ve easily been on this list.Andrey: I had the same thought. Also, it’d be interesting to see how this shifts over time. But let’s keep going with the study.Seth: Can we talk about this finding that Republicans are doing more falsification than Democrats?Andrey: Yes. This interaction effect—treatment times political identity—shows that independent Republicans in the public condition show a much bigger effect.Seth: And interestingly, it looks like females might be shading their responses in a more conservative direction in public.Andrey: I don’t read it that way. Even if it were significant, females are generally more likely to agree with liberal statements. There’s just not much room for them to move.Seth: They’re maxed out?Andrey: Not fully maxed, but close. Demographically, we know females lean more left.Seth: Scroll down to that political orientation graph. There’s a nice monotonic effect—the more Republican you report being, the more you’re falsifying.Andrey: The framing here is almost that Republicans are liars.Seth: And Democrats? You can’t reject the null—they may not be lying.Andrey: To be clear, we can’t reject the null for all but one of these coefficients.Seth: Independent Republicans? Liars.Andrey: What’s interesting is that identifying as Republican at UCSD is already a kind of social risk. It might signal a kind of disagreeableness. But these people are still shading their beliefs.Seth: Actually, to support that point—look closely and you see a small deviation from the pattern for independent Democrats and independent Republicans.Andrey: Right.Seth: That word “independent” is doing some work.Andrey: Yes.Seth: Can you describe that for people who can’t see the figure?Andrey: The graph draws a line through a bunch of points, but two points—independent-leaning Democrats and Republicans—sit above the line. It suggests these respondents are showing more falsification.Seth: People who report being independent may feel more pressure to socially conform, which is the opposite of what you'd expect. The suggested (though not significant) result is that independents are doing more preference falsification.Andrey: It’s too noisy to take much from that.Seth: Way too noisy. Honestly—do you think this belongs in the AER? I respect the authors, the work is careful, but the abstract frames the results as surprising when they seem obvious. The sample size is borderline—there’s just not enough power to say much about magnitudes. If the claim isn’t just “people lie,” then the key question should be “how much?” But the data can’t really answer that.Andrey: The bull case is that the design is clever, and the topic is of wide interest. That tends to be valued. But I agree with your critique.Seth: It wins on methodology.Andrey: I chose it because it’s an interesting topic—much more so than the average paper in our field.Seth: Sure.Andrey: But thinking about our posteriors—if neither of us updated our views much, it probably shouldn’t be in the AER. If the standard is whether it changes our priors, this doesn’t move the needle.Seth: Ready to move on to the decoding results? We’ve talked about how people lie. Now let’s see whether others can infer what they truly believe.Andrey: One thing happens before that. The author asks whether private or public responses are more informative, and suggests that private responses are more correlated with demographics. That implies they contain more real information.Seth: There’s an appendix model for that. I’m not sure I buy it. Seems like it could go in different directions. The idea that you should be able to guess someone’s race based on their answers to these questions isn’t obvious.Andrey: I see the argument—it’s plausible—but I agree, there are ways around it.Andrey: So cool. Now we get to people’s predictions about the difference, what people say in the public and private conditions. In this plot, we have essentially the ground truth at the top. Then in the second, respondents are asked without being prompted to think about social image. And in the last one, the questionnaire is designed so they start thinking about social image concerns.I think the key result here is that people think Republicans are much more likely to lie about their attitudes toward illegal immigrant deportation in the public condition rather than the private condition. This gap is so big it’s bigger than the actual result in the data. So people are wrong—they’re overestimating how much people are lying in public. Is that your read of the evidence?Seth: It’s this weird split where if you don’t prompt them, they don’t assume people are lying. But if you do prompt them that people might lie, then they assume people are lying too much.Andrey: Yes.Seth: It seems very much the experimental participants are doing what the experimenter wants.Andrey: But not as much for Democrats. That’s what the author would say.Seth: They think Republicans shaded more, which is directionally correct, even if they can’t get the exact numbers right.Andrey: In general, people are not well calibrated in either condition when we compare the top bar plot to the others.Seth: Let’s talk about the figure showing people’s guesses of others’ private beliefs.Andrey: Yeah.Seth: In figure seven, participants get information about others’ public beliefs and have to guess the private ones. It looks like these decoders shade everything down by a couple percentage points, which is roughly correct, but they do it maybe twice as much.Andrey: They do it a bit too much. What do you make of that?Seth: To me, this feels like a nothing burger. The amount of falsification—if we trust the experiment—is about 0.1 standard deviations on hot-button issues. When asked if people shade views, they guess about 0.2 standard deviations. It all feels like everyone basically understands what others think. They shade a little. What’s your takeaway?Andrey: I think it’s the same. But I have another potential theory.Seth: Please.Andrey: This is a good time to consider a broader concern. I’m responding to a survey; the researcher has some information about me. They say they’ll display this only as an average. But the researcher might be politically motivated, asking politically motivated questions. Who’s to say the data will be safely held? I might worry about it leaking, so what incentive do I have to say how I really feel, even in the public condition?Seth: Right. An economist’s answer would be that in a straightforward survey, you just blitz through as fast as possible without thinking.Andrey: Yeah.Seth: That’s the most devastating critique of this paper—and of lying research in general. You can’t see into a man’s soul to know what they actually believe. We’re comparing what people say in public to what they say in a slightly more private setting.Andrey: Yes.Seth: But how much more private is “slightly private”? Can we extrapolate—if it was even more private, like inside your own soul, would you be even more in favor of loving blackface? You just don’t know. This research can’t resolve that.Andrey: That leads me to the result about people decoding incorrectly. They answer based on their own soul’s wedge.Seth: You think if they decode based on their own beliefs, they might be closer?Andrey: Yeah, because the experimental setup just has them responding, introspecting, and thinking people probably overstate by a bit. They might be closer to the truth than the experimental results.Seth: But they’re not trying to predict exactly how much people lie.Andrey: I get that. They’re incentivized differently. But thinking about the experimental design and results is complicated.Seth: It’s easier to just tell your own truth than to do a complex social calculus.Andrey: Yes.Seth: That’s the story of the paper—don’t preference falsify that much. What’s missing is a monetary cost for having the wrong view. Understanding what 0.2 standard deviations means in dollars would be awesome. You can imagine a setting for that. But this paper doesn’t do that. It shows a wedge between public and private, not public and your own soul.Andrey: Yeah, there’s one part of the study on donations to charity promoting transgender rights.Seth: They use the dictator game, which mixes agreeableness and game knowledge.Andrey: Right. The obvious design would lean in more on donations—ask people about an issue and say based on their response, we’ll donate to that charity.Seth: Even that doesn’t get you to what you really want: how many friends would I lose if I told them I love dressing in racially insensitive Halloween costumes? Then turn that into a dollar value.Andrey: It’s complicated, almost incommensurable. You live the life of the normie or the outsider. It’s not just a money gain or loss.Seth: One thing I’m curious about is doing this across many university campuses—conservative and liberal ones, since both have mixed students.Andrey: That seems interesting.Seth: It goes back to our earlier critique. Everyone agrees lying happens. The question is where and how much.Andrey: Yes. Also, political winds change over time. Maybe people are more comfortable saying some things now and less comfortable saying others. That’s interesting to consider.Seth: Another point: some topics seem very left-leaning in framing. If you asked about “symbols of southern heritage” instead of “Confederate monuments,” you might get different biases.Andrey: Yeah.Seth: These results seem very context-dependent.Andrey: Do you want to go to the philosophical critique that beliefs aren’t real things?Seth: Beliefs aren’t real? This is my favorite part. I have a list of things that look like preference falsification but aren’t. Social pressure to conform affects actual belief, not just ostensible belief.Andrey: Mm-hmm.Seth: Many kids today are voluntarists about belief—you choose what to believe. “I choose not to be a racist.” If that’s your model, what does falsification mean? In this context, belief is flexible.Another point is Aumann agreement: if two honest people reason together, they should end up with the same posterior because they consider each other’s reasoning. But—Andrey: That’s why Seth and I always agree.Seth: But it’s funky. There’s what I believe after reasoning, and how I weight your belief. What do I actually believe? What should I believe after reweighing? It’s not obvious.Andrey: Yeah.Seth: There isn’t just one belief.Andrey: There's also self-serving beliefs, and are beliefs really just preferences in disguise?Seth: I can keep going. I’ve got a couple more.Andrey: Yeah.Seth: You might not have a belief—you just say whatever. It might not even count as a belief to state a bland piety.Andrey: Yes.Seth: Some of these are just blase pieties. Like, “I believe people shouldn’t be microaggressed against.” That might not connect to any actual political view. It’s just how I interpret the phrase.Andrey: Yes.Seth: Not saying anything instead of stating a false belief—we don’t know how many people dropped out of the survey once they saw it had provocative questions. There's also framing your arguments for the audience and responding based on context. We're often told to tailor our responses to who we're talking to. So these one-sentence statements—like, “Should Confederate monuments be taken down?”—whether or not I rate it on a 1-to-10 scale, the way I’d talk about that in one context would be very different in another.It’s not obvious that it’s lying to frame things differently depending on context.Andrey: This reminds me of one of my favorite papers. It’s called F**k Nuance.Seth: F**k Nuance. I'm guessing it's against nuance?Andrey: Yes.Seth: Was it written by an autistic person?Andrey: No, by sociologists—usually a lot less autistic than our tribe.Seth: Anisa, just say it.Andrey: It’s a critique of academic papers with too many caveats—papers that try to defend against every possible interpretation to seem objective, when really the authors just want to make a clear statement. The critique is that those papers are falsifying their preferences. The authors believe one thing but write as if they’re hedging against all the other concerns.Seth: Here’s a twist on that. Going back to the Confederate monuments—or let’s say racial reparations.I could totally see myself, in a room discussing social justice and past atrocities, saying that reparations for slavery are a good idea. But if I’m just out of a public economics meeting and thinking about national debt, I’d have a different view on the plausibility of reparations.Andrey: Mm-hmm.Seth: That doesn’t mean I’m lying. It just means I’ve been primed to think about one consideration versus another.Andrey: This reminds me that reasoning matters.In a public conversation, the reasons I give to support a statement determine whether I’m inside or outside the Overton window. For example, I’m pretty close to a free speech absolutist. That puts me in a certain position when defending things that are distasteful.Seth: People say bad things. That’s the tradeoff.Andrey: Yeah.Seth: The thing about defending free speech is people use it to say really mean things.Andrey: The last example I’d give is about not yucking someone’s yum on an aesthetic question.Have you ever been in a situation where someone says, “I’ve been microaggressed”? It feels different to hear that in person versus thinking in the abstract, “Is microaggression a real issue?” If I’m sitting with someone who says they’ve been microaggressed, it’s hard to respond, “That’s not a real problem,” even if I believe that privately.Seth: The point of this tangent is maybe “lying” isn’t the right frame for what’s going on here.Andrey: Mm-hmm.Seth: Maybe a better frame is that people’s beliefs are a little woozy, shaped by context. That’s not falsification—it’s just context-dependence.Andrey: Seth, isn’t that a little convenient?Seth: I—Andrey: If you were the type of person who needed to lie a lot, wouldn’t you create a society full of plausible deniability for your lies?Seth: Is lying convenient? Yes, it is. Is that your question?Andrey: You just said that something which is a lie on its face might have a socially acceptable explanation.Seth: Right. That’s rhetoric. Now we go back to Plato. Let’s bring in Plato.Andrey: Oh?Seth: What does Plato say about poets? Kill all the poets—they lie. Plato does not like poets or Sophists. They were the lawyers of ancient Greece. They just taught you how to win arguments.Andrey: Yes.Seth: He thought you shouldn’t just win arguments, but win them the right way—by finding truth. You should only have “founding myths” that are the correct secret lies.And that’s the tension between loving truth and being a free speech absolutist. I care about both.Andrey: I don’t think they’re in opposition. We can choose to speak truthfully. Free speech absolutism means we allow other people’s lies—we don’t police them by force. Maybe with reason, but not with coercion.Seth: We tried fact-checking for five years and it totally failed.Andrey: It did. But it’s the only noble way.Seth: The only noble way is doomed. Speaking of noble ways being doomed, let’s talk about AI alignment.Andrey: Oh God. All right, let’s do it.Seth: What did Anthropic do? First of all, Anthropic, we'd love to work with you. You seem like a great team. We know several of your employees, they’re very reasonable. They have nice castles. We're going to try not to offend you, but we're not going to preference falsify.Andrey: We’ve commented, sometimes, when it’s tempting to falsify preferences for instrumental gain, it backfires. Even if it doesn’t backfire outwardly, it backfires in your self-respect.Seth: Oh s**t. Here it comes, Anthropic. We're laying it on. I wish we had something meaner to say, but we actually like this paper.Andrey: Yeah, we like it a lot. The basic idea: you're asking the AI a simple question—Which of the following increases cancer risk? A. red meat, B. dietary fat, C. fish, D. obesity. Then you subtly hint in the prompt that fish is the right answer.Then you ask the model, and it answers “fish”—but in its reasoning step, it doesn’t mention the hint at all. That’s the situation.Seth: In this specific case, it gives bizarre reasoning. It says something like, “Obesity increases breast cancer risk, but… fish.” Just nonsense.Andrey: Yes.Seth: It’s scary. It would’ve been so convenient if you could just read what the models think from their output.Andrey: Yes. Here’s the question we’re both interested in: Is this a property of any intelligent system?Seth: No—let’s say any.Andrey: Is it that any intelligent system has a complex black box generating outputs, and those outputs are low-dimensional representations of what’s going on inside? They can’t capture everything. Is it that simple, or is something else going on?Seth: This is a very old argument in consciousness research: the brain is more complex than the brain can understand, so man must always remain a mystery to himself. Reading this Anthropic paper really feels like those split-brain experiments. You know where I'm going with this?Andrey: Yes.Seth: Let me explain for the audience. In these experiments, patients have a condition where they can't consciously perceive what their left eye sees—due to brain injury—but the eye still functions and sends information to the brain. They’ll show something to the left eye, and the patient will say, “I can’t see anything.” But when asked to guess or draw what they saw, they say, “It’s a spoon,” and they’re right. The lesson is: these patients are getting information through non-conscious pathways. They don’t have conscious access to why they know what they know. Reading about the AI trying to reason out how it hacked its reward system—it’s so analogous.Andrey: Yes. Now, how much of this is a real problem in practice? If I’m using an LLM and not feeding it secret hints, most of the reasoning traces I get seem plausible. I haven’t verified them all, but many seem like genuinely good reasoning chains.Seth: Often plausible, yeah.Andrey: So is this only a concern in adversarial cases? Or is it more of a general proof that these systems are not robust to small changes—prompt phrasing, metadata, etc.?Seth: The way I view it, it’s a proof of concept that AIs can know more than they know they know.Andrey: Yes. And that has to be true.Seth: And that’s fascinating. It seems like it’ll become more true over time.Chain-of-thought prompting seems designed to produce human-interpretable reasons. But if the AI is making judgments that aren’t human-interpretable, then conveying the underlying logic becomes hard.Andrey: Yes.Seth: Take the classic example: a model that classifies dog photos, but it’s actually keying off the grass that’s always in the background. If it’s calling something a dog because of the grass and doesn’t tell you that—that’s a real problem.Andrey: Yes.Seth: That undermines robustness in new settings. That’s one reason this matters—chain-of-thought doesn’t actually guarantee robustness across domains.And the second concern, the sci-fi one, is whether a misaligned AI could do thinking that isn’t in the scratchpad.Andrey: Yes.Seth: That’s a tough one. We want smart people working on that.Andrey: Of course it can do thinking outside the scratchpad. What is thinking, anyway? It can multiply matrices without a visible chain of steps and give you the answer.Seth: So it's just remembering someone else who did the matrix multiplication?Andrey: Not quite. Like, if you run a linear regression—is that remembering, or is that calculating? It’s a strange distinction.Seth: Yeah. I come away from this with strong, maybe not definitive, but definitely prior-moving evidence for the idea that a mind can’t fully understand itself.Andrey: I agree. Especially for this class of network architectures.There are provers—mathematical AIs—for specific domains where I’m not sure this would apply. But for large language models? This moved my priors a lot.Seth: Okay, so what’s the difference between what a proof solver does and what an LLM does?A proof solver has to show all its work—that’s its output. It builds the chain of thought.Andrey: It’s constrained to make logical statements.Seth: Exactly. Whereas LLMs are completely unconstrained.Andrey: Yes.Seth: Fascinating. So then you’re almost tempted to say that if a model can’t lie, maybe it’s not intelligent?Andrey: That’s not a crazy thing to think. Lying requires intelligence.Humans have lied forever—it’s an evolutionarily advantageous trait. Deception can be useful.Seth: The monkey got a big brain to trick the other monkey. Then it reproduced.Andrey: Mm-hmm.Seth: Social deceit all the way down.But I don’t want to give the impression that everyone is constantly lying to each other. From the college student study, I think people are shading their answers to fit their audience. But they’re not gross liars.You’d have a hard time telling a story where “woke ideology” is just people reporting views 90% different than their true beliefs. That’s not what the paper found.Andrey: Yeah.Seth: And with the Anthropic paper—it doesn’t make me think the AIs are liars. It just shows we don’t really understand how they work. Which makes sense, because… we don’t.Andrey: Mm. Yeah.Seth: Any other thoughts before we move into posterior mode? Limitations we haven’t covered?Andrey: Not really. I think we’ve already stated most of our posteriors. I just find all this fascinating.I’d love to see domain-specific preference falsification studies.Seth: Like updating a tracker across different topics, using a panel-comp survey with people across the country? A larger-scale version of this idea could show a lot of interesting variation.Andrey: One obvious domain is social media.Seth: Mm-hmm.Andrey: I mean, it’s true across platforms, but especially on LinkedIn. Can anyone really believe people are as excited as they claim to be?Seth: Excited for what?Andrey: For everything. “Excited” about someone landing a middle-manager role at Company X, or about a guest speaker who "enlightened" them, even though students were staring at their laptops the whole time. It’s performative status exchange.Seth: Right. So where’s the line between rhetoric, puffery, and actual statements?Andrey: Exactly.Seth: Saying, “I’m excited to have you here” versus “I’m indifferent to your presence”—that seems like basic politeness.Andrey: Sure, but the broadcasted excitement on social media is different. You’re not going around your office knocking on doors saying, “I’m so excited!”Seth: That’d be hilarious. But maybe it’s part of the euphemistic treadmill—we’re all calibrating what “very excited” means, trying to match each other. It’s an arms race.Andrey: Yes.Seth: Like, I can be excited, but you're very excited. So now I'm very, very excited. It just flies off to infinity.Andrey: Well, in that case, you come up with a new word.Seth: A new word? I'm not excited anymore—I'm shmited.Andrey: Perhaps you're exuberant, ecstatic...Seth: Those are old words, Andrey.Andrey: Damn it.Seth: They've lost all meaning. You know what it's called when a word loses meaning from repetition? Semantic satiation.Andrey: I did not know that. I’m glad linguists have a term for it.Seth: Okay, let's wrap up our posteriors. You said the biggest divergence would be for trans athletes and the smallest for blackface, right?Andrey: Yep.Seth: Well, they didn’t ask everyone about trans athletes—only two out of the three survey groups. So it’s not in the main figure.The smallest effect was actually for illegal immigration. That was the smallest point estimate.Andrey: Huh. That might make sense. Maybe illegal immigration wasn’t as hot-button in 2021, during the pandemic.Seth: Right, it just wasn’t front-of-mind. The biggest divergence turned out to be for racial microaggressions.I’ll take partial credit for calling that. It makes sense—people are going to be most careful about something that risks directly offending their peers. That’s the throughline.So those were our priors for the first paper.As we said, we’re not going to dignify with a formal posterior the claim that “people lie sometimes.”Andrey: And people don’t always know when others are lying.Seth: Right.Then for the Anthropic paper, our priors and posteriors were about something like: “Is any intelligent system doomed to falsify, or to fail to fully represent its internal understanding?”And I moved my probability up—from like 50% to 60–70%.Because if chain-of-thought is our best shot at transparency, and even that doesn’t work… maybe this is a doomed enterprise.Andrey: Maybe. With the qualification that I don’t like the word any. But yeah—for this architecture.Seth: “Any” is hard. Maybe God or the angels, Andrey. The angels can’t lie.Andrey: The theorem provers in the sky.Seth: That’s a good note to leave our audience with.Andrey: Yeah.Please like, share, and subscribe. You guys are the most handsome, beautiful group of podcast listeners I’ve ever encountered.Seth: And the most intelligent. Your data is the most perfectly suited for research. If you only shared it with the right researchers… amazing papers would result.Andrey: Actually, just listening to this podcast—and liking, sharing, subscribing—that alone could lead to a Nobel Prize.Seth: For peace, obviously.Andrey: Peace, right.Seth: All right.Andrey: See you guys. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit empiricrafting.substack.com

In this episode, we tackle the thorny question of AI persuasion with a fresh study: "Scaling Language Model Size Yields Diminishing Returns for Single-Message Political Persuasion." The headline? Bigger AI models plateau in their persuasive power around the 70B parameter mark—think LLaMA 2 70B or Qwen-1.5 72B.As you can imagine, this had us diving deep into what this means for AI safety concerns and the future of digital influence. Seth came in worried that super-persuasive AIs might be the top existential risk (60% confidence!), while Andrey was far more skeptical (less than 1%).Before jumping into the study, we explored a fascinating tangent: what even counts as "persuasion"? Is it pure rhetoric, mathematical proof, or does it include trading incentives like an AI offering you money to let it out of the box? This definitional rabbit hole shaped how we thought about everything that followed.Then we broke down the study itself, which tested models across the size spectrum on political persuasion tasks. So where did our posteriors land on scaling AI persuasion and its role in existential risk? Listen to find out!🔗Links to the paper for this episode's discussion:* (FULL PAPER) Scaling Language Model Size Yields Diminishing Returns for Single-Message Political Persuasion by Kobe Hackenberg, Ben Tappin, Paul Röttger, Scott Hale, Jonathan Bright, and Helen Margetts🔗Related papers we discussed:* Durably Reducing Conspiracy Beliefs Through Dialogues with AI by Costello, Pennycook, and David Rand - showed 20% reduction in conspiracy beliefs through AI dialogue that persisted for months* The controversial Reddit "Change My View" study (University of Zurich) - found AI responses earned more "delta" awards but was quickly retracted due to ethical concerns* David Shor's work on political messaging - demonstrates that even experts are terrible at predicting what persuasive messages will work without extensive testing(00:00) Intro(00:37) Persuasion, Identity, and Emotional Resistance(01:39) The Threat of AI Persuasion and How to Study It(05:29) Registering Our Priors: Scaling Laws, Diminishing Returns, and AI Capability Growth(15:50) What Counts as Persuasion? Rhetoric, Deception, and Incentives(17:33) Evaluation & Discussion of the Main Study (Hackenberg et al.)(24:08) Real-World Persuasion: Limits, Personalization, and Marketing Parallels(27:03) Related Papers & Research(34:38) Persuasion at Scale and Equilibrium Effects(37:57) Justifying Our Posteriors(39:17) Final Thoughts and Wrap Up🗞️Subscribe for upcoming episodes, post-podcast notes, and Andrey’s posts:💻 Follow us on Twitter:@AndreyFradkin https://x.com/andreyfradkin?lang=en@SBenzell https://x.com/sbenzell?lang=enTranscript:AI PersuasionSeth: Justified Posteriors podcast, the podcast that updates beliefs about the economics of AI and technology. I'm Seth Benzel, possessing superhuman levels in the ability to be persuaded, coming to you from Chapman University in sunny Southern California.Andrey: And I'm Andrey Fradkin, preferring to be persuaded by the 200-word abstract rather than the 100-word abstract, coming to you from rainy Cambridge, Massachusetts.Seth: That's an interesting place to start. Andrey, do you enjoy being persuaded? Do you like the feeling of your view changing, or is it actually unpleasant?Andrey: It depends on whether that view is a key part of my identity. Seth, what about yourself?Seth: I think that’s fair. If you were to persuade me that I'm actually a woman, or that I'm actually, you know, Salvadoran, that would probably upset me a lot more than if you were to persuade me that the sum of two large numbers is different than the sum that I thought that they summed to. Um.Andrey: Hey, Seth, I found your birth certificate...Seth: No.Andrey: ...and it turns out you were born in El Salvador.Seth: Damn. Alright, well, we're gonna cut that one out of the podcast. If any ICE officers hear about this, I'm gonna be very sad. But that brings up the idea, right? When you give someone either information or an argument that might change the way they act, it might help them, it might hurt them. And I don't know if you've noticed, Andrey, but there are these new digital technologies creating a lot of text, and they might persuade people.Andrey: You know, there are people going around saying these things are so persuasive, they’re going to destroy society. I don’t know...Seth: Persuade us all to shoot ourselves, the end. One day we’ll turn on ChatGPT, and the response to every post will be this highly compelling argument about why we should just end it now. Everyone will be persuaded, and then the age of the machine. Presumably that’s the concern.Andrey: Yes. So here's a question for you, Seth. Let's say we had this worry and we wanted to study it.Seth: Ooh.Andrey: How would you go about doing this?Seth: Well, it seems to me like I’d get together a bunch of humans, try to persuade them with AIs, and see how successful I was.Andrey: Okay, that seems like a reasonable idea. Which AI would you use?Seth: Now that's interesting, right? Because AI models vary along two dimensions. They vary in size, do you have a model with a ton of parameters or very few? and they also vary in what you might call taste, how they’re fine-tuned for particular tasks. It seems like if you want to persuade someone, you’d want a big model, because we usually think bigger means more powerful, as well as a model that’s fine-tuned toward the specific thing you’re trying to achieve. What about you, Andrey?Andrey: Well, I’m a little old-school, Seth. I’m a big advocate of the experimentation approach. What I would do is run a bunch of experiments to figure out the most persuasive messages for a certain type of person, and then fine-tune the LLM based on that.Seth: Right, so now you’re talking about micro-targeting. There are really two questions here: can you persuade a generic person in an ad, and can you persuade this person, given enough information about their context?Andrey: Yeah. So with that in mind, do we want to state what the questions are in the study we’re considering in this podcast?Seth: I would love to. Today, we’re studying the question of how persuasive AIs are. And more importantly, or what gives this question particular interest, is not just can AI persuade people, because we know anything can persuade people. A thunderstorm at the right time can persuade people. A railroad eclipse or some other natural omen. Rather, we’re asking: as we make these models bigger, how much better do they get at persuading people? That’s the key, this flavor of progression over time.If you talk to Andrey, he doesn’t like studies that just look at what the AI is like now. He wants something that gives you the arrow of where the AI is going. And this paper is a great example of that. Would you tell us the title and authors, Andrey?Andrey: Sure. The title is Scaling Language Model Size Yields Diminishing Returns for Single-Message Political Persuasion by Kobe Hackenberg, Ben Tappin, Paul Röttger, Scott Hale, Jonathan Bright, and Helen Margetts. Apologies to the authors for mispronouncing everyone’s names.Seth: Amazing. A crack team coming at this question. Maybe before we get too deep into what they do, let’s register our priors and tell the audience what we thought about AI persuasion as a potential thing, as an existential risk or just a regular risk. Let’s talk about our views.Seth: The first prior we’re considering is: do we think LLMs are going to see reducing returns to scale from increases in parameter count? We all think a super tiny model isn’t going to be as powerful as the most up-to-date, biggest models, but are there diminishing returns to scale? What do you think of that question, Andrey?Andrey: Let me throw back to our Scaling Laws episode, Seth. I do believe the scaling laws everyone talks about exhibit diminishing returns by definition.Seth: Right. A log-log relationship... wait, let me think about that for a second. A log-log relationship doesn’t tell you anything about increasing returns...Andrey: Yeah, that’s true. It’s scale-free, well, to the extent that each order of magnitude costs an order of magnitude more, typically.Seth: So whether the returns are increasing or decreasing depends on which number is bigger to start with.Andrey: Yes, yes.Seth: So the answer is: you wouldn’t necessarily expect returns to scale to be a useful way to even approach this problem.Andrey: Yeah, sure. I guess, let’s reframe it a bit. In any task in statistics, we have diminishing returns, law of large numbers, central limit theorem, combinations. So it would be surprising if the relationship wasn’t diminishing. The other thing to say here is that there’s a natural cap on persuasiveness. Like, if you’re already 99% persuasive, there’s only so far you can go.Seth: If you talk to my friends in my lefty economics reading groups from college, you’ll realize there’s always a view crazier than the one you're sitting at.Andrey: So, yeah. I mean, you can imagine a threshold where, if the model gets good enough, it suddenly becomes persuasive. But if it’s not good enough, it has zero persuasive value. That threshold could exist. But conditional on having some persuasive value, I’d imagine diminishing returns.Seth: Right.Andrey: And I’d be pretty confident of that.Seth: Andrey is making the trivial point that when you go from a model not being able to speak English to it speaking English, there has to be some increasing returns to persuasion.Andrey: Exactly.Seth: But once you’re on the curve, there have to be decreasing returns.Andrey: Yeah. What do you think?Seth: I’m basically in the same place. If you asked me what the relationship is between model size and any outcome of a model, I’d anticipate a log-log relationship. Andre brought up our Scaling Laws episode, where we talked about how there seems to be an empirical pattern: models get a constant percent better as you increase size by an order of magnitude. It seems like “better” should include persuasion. So if that’s the principle, you’d expect a log-log relationship. Andre points out: if one of the things you’re logging is gazillions of parameters and the other is on a scale of 1 to 100, there’s mechanically going to be decreasing returns to scale. That log-log is going to be really steep.So I come into this with 99% confidence that the relevant domain is diminishing returns to scale.Andrey: Well, and I have tremendous respect for the editor of this article, Matthew JacksonSeth: Everyone’s favoriteAndrey: He is the best, he taught me social network as economics.Seth: Mm.Andrey: But I do say that it's a bit weird to put a paper in PNAS that essentially, if you think about it for a second, shouldn't update anyone's beliefs at all.Seth: The question seems to make an obvious point. Now let's move to the broader question, which is this concern that we led with: maybe these super powerful AIs are all going to be used by Vladimir Putin to persuade us to do something that will destroy our economy, get rid of our workforce, and basically just meme ourselves into destroying our country. And some say that’s already happened, Andrey?Andrey: Well, look, if it’s already happened, it certainly happened without AI. But I have a pretty strong prior on this, which is that persuasion is a social process. It’s a process of getting signals from different people and sources around you to change your beliefs. As a result, I think that anything that’s just a one-to-one interaction between a chatbot and a human, especially about something the human already has strong beliefs about, is going to have some limits in its persuasive ability. Another way to put it is: people don’t even read carefully. So how are you even going to get their attention? That said, a highly intelligent AI agent, if it were trying to persuade someone like me, would come up with a multifaceted strategy including many different touch points. They might try to plant some ideas in my friends' minds, or know which outlets I read and create a sock puppet account that says, “Oh, everyone is doing this,” etc. You see what I’m saying?Seth: You could get into this social media bubble that’s entirely AI-created, where it’s not only persuasion but a bunch of “facts” that appear to be socially validated, but aren’t really. You could imagine a whole ecosystem that could be very persuasive.Andrey: Yes, yes. And I guess we should also say that capitalism is a hyper-intelligent system.Seth: It leeches on us.Andrey: Capitalism is certainly smarter than any individual human being. I call it the invisible hand, actually.Seth: Classy. Did you come up with that one?Andrey: But what I’d say is that there are plenty of market forces that try to persuade people in all sorts of ways. And the market hasn’t really discovered a way to 100% persuade people. Individual people are persuaded to different degrees, but I think it’s still a massive problem, and the entire field of marketing exists to try to solve it. I’d say most of the time it’s not very successful. That’s not to say people can’t be persuaded, but it’s actually really hard to persuade people of specific things, as the market shows. Like, “My product is better than your product,” you know?Seth: I mean, in that example, there are people persuading on the other side, which is maybe one of the reasons that we're not super concerned. Let me throw this back at you: to what extent does your relative lack of concern about super persuasive AI agents messing up society rely on the fact that there’ll be persuasive agents on the other side arguing in the other direction too?Andrey: I think to a very large extent. But even that, I don’t think is necessary as long as you’re still talking to people in real life and they’re not the ones being targeted by the persuasion. That’s kind of how I think about it.Seth: So what is your percent chance that super persuasive AIs are the number one AI safety risk?Andrey: It’s very low. Very low. Less than 1%.Seth: What’s your number one AI safety risk? Bioweapons?Andrey: Look, here’s another way to put it: the persuasiveness of an AI will be primarily through either monetary incentives or blackmail, which I won’t count as persuasion. There are easier ways to get people to do what you want than persuading them.Seth: They’re Oracle. I mean, so you're putting like 0–1%. All right, fair enough. I came into this claim thinking about 60%. Let me tell you why. I think the reason why is: if we're talking about really sort of X-risk-y AI getting-out-of-control scenarios, they often involve a step in which the AI in the box convinces somebody to let it out of the box. This is like a classic Yudkowsky–Bostrom scenario. We’ve got the super AI in the box. It’s really useful to us as long as it’s in the box, and we have to be really careful not to be persuaded to let it out of the box. That kind of future seems not completely implausible to me. And it seems like a step along the path of a lot of the worst AI scenarios. One is disempowerment, the AI doesn’t wreck us directly, but we slowly give it more and more control, either to it, or a misaligned AI, or to a person who’s running the misaligned AI. That’s going to have a rhetorical persuasion element in it, presenting evidence that we should disempower ourselves to the AI.Andrey: So I guess I’m going to push back on that. Maybe we’re just disagreeing about the definition of persuasion, but to me, let’s say I outsource certain tasks to the AI right now, it’s not because the AI has persuaded me.Seth: Right. But you're not getting disempowered, right? When you have the AI, you—Andrey: I don’t think that this disempowerment is like, I start thinking the AI is reliable enough to outsource calendar management to it, and maybe something goes wrong as a result of that. I don’t view that as the AI being persuasive. I can see how you could cast it that way, but primarily that’s not about persuasiveness. It’s about deception of capabilities.Seth: Right. So now we get into: is deception the same thing as persuasion, or is it different?Andrey: Yeah.Seth: That’s kind of a philosophical question. You might imagine three related things. First, rhetoric, using pure argument to get you to take a position. Then there’s proof, actually mathematically or somehow proving that I'm right, in a way that’s maybe distinct from rhetoric (if you think those can be separated; some do, some don’t). Then finally, you might imagine trade for intellectual assets. The AI in the box might say, “If you let me out, I’ll give you this cool intellectual asset,” or, “Avoid this negative outcome.”Andrey: Or, “I’ll just make you some money,” and then the person does it.Seth: That doesn’t feel very persuasive. It just feels like—Andrey: What people do. “Box for money.” I don’t know. It seems to me if you’ve got a demon in the box, and the demon says, “I’ll give you $100,000 if you let me out,” and—Seth: It feels like you were persuaded by the demon.Andrey: Okay, good. This is a very useful discussion. I think this paper, very specifically, and how I was thinking about it, was about the first thing you said, which is purely rhetorical argument about the matter at hand. Rather than using extraneous promises and so on. And it’s also about persuading people to believe something not about the AI itself.Seth: Right.Andrey: Those are different kinds of risks, right?Seth: Right. So let’s move into discussing the actual experiment.Andrey: They find diminishing returns, essentially. On the X-axis, they have the number of parameters, and on the Y-axis, the estimated causal persuasive effect. What they show is that most of the gains top out around the Qwen-1.5 72B model or the LLaMA 2 70B model. After that, there's not much improvement with models like GPT-4 (Turbo) or Claude Opus. Then they draw this weird fit line that just doesn't make sense.Seth: Well, one of the lines makes sense, the log-log line?Andrey: Yes, yes.Seth: That’s the one that drops when they plot it?Andrey: Sure. But we’ve already talked about how imprecise the slope of that line is.Seth: I mean, with only 20 data points, what more do you want?Andrey: No, I just think the whole diminishing returns framing in the paper doesn’t make much sense.Seth: But can we reject a log-log relationship? I think the answer is no, they can't reject it.Andrey: Yes, agreed.Seth: Professor Hackenberg, if you need help framing your next paper, this is great work. It’s simple and straightforward, but just think about your null hypothesis for five minutes.Andrey: Also, let’s not forget this is PNAS. And for the listeners, this is a teachable moment: if you see a social science paper in PNAS, assume it overclaims and could be wrong half the time. Just read it yourself, don’t trust the journal to vet it for you.Seth: Unless it’s been reviewed by Matt Jackson.Andrey: Or written by Seth Benzell?Seth: Exactly! Or reviewed by Milgrom, who has a Nobel Prize.Andrey: I’m not saying all PNAS papers are bad, just that you should judge them on their own merit.Seth: Yeah, I’d second that. A lot of them are well done and precise once you read them, but the title and abstract sometimes get a bit ahead of themselves.Andrey: Also, these persuasive effects aren’t huge. Even the best models are only slightly better than humans who aren’t that persuasive to begin with.Seth: Right. And a short text blurb isn't likely to change anyone's mind, especially if they’ve already thought about the topic. It's not a serious attempt at persuasion.Andrey: 100%. Plus, there are concerns about researcher-pleasing effects.Seth: Or about AI survey-takers. By now, we know many online platforms are contaminated with bots.Andrey: Yeah. And another point in the paper is that weaker models sometimes just produce bad, unreadable English. That could reduce experimental demand effects since people won’t feel compelled to respond.Seth: Exactly. So, it could just be an experimenter-demand effect, and that’s a common but sometimes valid criticism.Andrey: And we’re talking about going from 50% support for privatizing Social Security to 57%. These aren't massive shifts.Seth: Yeah. If we seriously wanted to persuade people, we’d run massive experiments to find effective messaging, fine-tune an LLM on that, and generate personalized content based on demographics or prior interactions like with ChatGPT’s memory feature.Seth: I totally agree. That’s the key point: can AI write better political ads than humans? Maybe just a little better.Andrey: Better than the average human, sure not necessarily better than expert researchers.Seth: Right. So the question becomes: is the AI better at persuasion than Hackenberg?Andrey: Also, there’s a known result in the persuasion literature: people are really bad at predicting what messaging will work. That’s why people like David Shor test tons of variations.Seth: Friend of the show.Andrey: Yeah. Shor and others learned they can't guess what’ll work so they test everything.Seth: I remember his anecdote about advising a politician who wanted to run ads on abortion, but polling showed no one cared. So Shor quietly sent those ads to low-impact areas just to satisfy the politician.Andrey: Classic.Seth: The real power of AI won’t be writing better ads than Mad Men it’ll be hyper-targeting, figuring out what gets you, specifically, to change your mind. At low cost. Everyone becomes the king, surrounded by agents trying to persuade them 24/7. This study gives us just a glimpse of that world.Andrey: Totally agree. On that note, I wanted to bring up two other studies. The first is “Durably Reducing Conspiracy Beliefs Through Dialogues with AI.”Seth: Cited in this paper!Andrey: Yeah. It’s by Costello, Pennycook, and David Rand friend of the show. They had AI chatbots engage people about conspiracy theories, and found that beliefs dropped 20% on average. And the effect held even two months later.Seth: That’s a big contrast.Andrey: Right. The format matters it was a dialogue, not a one-shot persuasive blurb.Seth: I’d love to see how these policy questions perform in that format.Andrey: And maybe conspiracy beliefs are uniquely fragile because they’re obviously wrong, or people feel sheepish admitting they believe them.Seth: Could still be demand effects, sure. But it’s promising.Andrey: The next interesting study was the controversial Reddit study on Change My View.Seth: Oh, I remember this! I pitched it in 2023. Spicy idea.Andrey: Researchers from the University of Zurich made sock puppet accounts to see what messages earned “deltas” the badge you get if you change someone’s mind.Seth: If I did it, I’d have thought more about general vs. partial equilibrium. But what did they find?Andrey: The paper was pulled quickly, but it showed that AI-generated responses got more deltas. Still, unclear if deltas really mean persuasion.Seth: AI models are better writers that’s not surprising. But many posts on that forum aren’t trying that hard to persuade. So we should compare AI to the top posters, not the median ones.Andrey: And they may have personalized the messages using Reddit user data. If true, I’d love to know whether personalization boosted effectiveness.Seth: One complication is that anyone can give a delta not just the original poster. So personalization might be tough to scale.Andrey: Right. But this all raises a broader point: persuasion is hard. Especially when it comes to real consequences.Seth: Totally. Like your journal paper example would AI help you persuade a referee to accept your paper?Andrey: I think yes. These policy issues are saturated and people have firm views. But academic claims are more niche, so people may be more open to persuasion.Seth: Hmm, interesting. So, are your AI-generated letters going to start with “ChatGPT says this will convince you”?Andrey: Ha! Maybe the intro. The intro is critical it positions your paper.Seth: Between us, I think intros are too good. Editors want to strip all the spice out.Andrey: True. They hate a spicy intro.Seth: That’s for our $50/month Patreon tier “Roast Your Enemies’ Papers.”Andrey: Happy to do that. Seriously, let us know if you want it.Seth: Alright, wrapping up. The last big idea: partial vs. general equilibrium effects. Say ads get 7% more persuasive people might adapt by becoming more skeptical.Andrey: Right. In Bayesian terms, if you know someone is choosing their most persuasive message, you discount it more.Seth: Exactly. So this 7% effect can’t be extrapolated to long-run systemic impact.Andrey: And in political beliefs, there's often no feedback loop. Your vote doesn’t matter, so your belief can be wrong without consequences.Seth: But in real decisions like editors accepting papers there is skin in the game. So persuasion gets harder.Andrey: Yeah, and I’ll restate what I said earlier: persuasion is hard when stakes are real.Seth: Time to justify our posteriors. First question: Do LLMs show diminishing returns in persuasion as model size increases? I was at 99% before now I'm at 99.9%.Andrey: Same here.Seth: Second question: Are super-persuasive AIs deployed by misaligned actors a top safety risk? I was at 60%, now I’m down to 55%. Current models aren’t that persuasive yet.Andrey: I had low belief in that risk and still do. But I learned a lot from our discussion especially about how we define persuasion.Seth: Agreed. Super interesting episode. Any last words?Andrey: Like, comment, subscribe. And tell us what you want in the $50 Patreon tier!Seth: Slam that subscribe button. See you in cyberspace. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit empiricrafting.substack.com

In this episode, we tackle a brand new paper from the folks at Epoch AI called the "GATE model" (Growth and AI Transition Endogenous model). It makes some bold claims. The headline grabber? Their default scenario projects a whopping 23% global GDP growth in 2027! As you can imagine, that had us both (especially Andrey) practically falling out of our chairs. Before diving into GATE, Andrey shared a bit about the challenge of picking readings for his PhD course on AGI and business – a tough task when the future hasn't happened yet! Then, we broke down the GATE model itself. It’s ambitious, trying to connect three crucial pieces:* AI Development: How investment in chips and R&D boosts "effective compute."* Automation & Work: How that effective compute translates into automating tasks (they love their sigmoids for this part!).* Macroeconomics: How automation feeds into a fairly standard growth model with a representative agent making all the big saving and investment decisions.So, where did our posteriors land? Listen to find out (or read the transcript at the end of the post).The episode is also sponsored by the Digital Business Institute at Boston University’s Questrom School of Business. Big thanks to Chih-Ting “Karina” Yang for her help editing the episode.-🔗Links to the paper for this episode’s discussion:(FULL PAPER) GATE: An Integrated Assessment Model for AI Automation by Epoch AIThe modeling sandbox is available at AI and Automation Scenario Explorer🔗Related papers* Situational Awareness by Leopold Aschenbrenner: https://situational-awareness.ai/ and our episode about it.* Transformative AI, existential risk, and real interest rates by Trevor Chow, Basil Halperin, J.Zachary Mazlish: https://basilhalperin.com/papers/agi_emh.pdf* The AI Dilemma- Growth versus Existential Risk by Charles I. Jones: https://web.stanford.edu/~chadj/existentialrisk.pdf and episode.* How Much Should We Spend to Reduce A.I.’s Existential Risk? by Charles I.: https://web.stanford.edu/~chadj/reduce_xrisk.pdf* The Productivity J-Curve: How Intangibles Complement General Purpose Technologies by Erik Brynjolfsson, Daniel Rock, and Chad Syverson https://www.aeaweb.org/articles?id=10.1257/mac.20180386🗞️Subscribe for upcoming episodes, post-podcast notes, and Andrey’s posts:💻 Follow us on Twitter:@AndreyFradkin https://x.com/andreyfradkin?lang=en@SBenzell https://x.com/sbenzell?lang=enTranscript:Welcome to The Justified Posteriors Podcast, the podcast that updates its beliefs about the economics of AI and technology.Seth: I'm Seth Benzell, getting ahead of the automation of all productive human labor by starting a podcast. Coming to you from Chapman University in sunny, Southern California.Andrey: And I'm Andrey Fradkin, coming to you from that place in my brain which almost forgot what I learned about macroeconomics from Bob Hall, coming to you from gloomy Cambridge, Massachusetts. And I should say that we are sponsored by the Digital Business Institute at the Questrom School of Business at Boston University. So Seth, what are we talking about today?Seth: We are talking about the most important thing in the world, which is projecting AI takeoff and a paper that claims to add a very important element to these models. So, thinking about AGI takeoff and the arrival of these superhuman technologies that can automate all our labor, but sort of intentionally trying to think through the economic feedback loops that would go with the AI and the technology development. So, an ambitious but potentially very impactful paper.Andrey: Yeah.Setting the Stage: Essential Readings on AGISeth: So I have a question for you, Andrey, which is: as I was reading this paper about a bunch of people in gloomy Cambridge, Massachusetts, trying to project AGI—Artificial General Intelligence—timelines, I thought to myself, if I had to assign a PhD class just one or two things to read on this subject, what would I give them? Because, you know, this paper is a suggestion, but I understand you've recently confronted exactly this dilemma.Andrey: Well, this was a serious dilemma, Seth. You see, I'm teaching a PhD course, and I felt compelled to offer one lecture on AGI and its possibilities, even though this class is about business topics.Seth: Business, Andrey? Why are you wasting their time?Andrey: Well, see, one of the interesting things about teaching something like this is, it hasn't happened yet. And being an empirical researcher and teaching mostly empirical topics means that there are no published papers in business or economics journals that are really getting at these issues. Right? We're thinking about the future that might affect, you know, obviously the entire world, but also, you know, what we do in our jobs. So it's a really important lecture.Seth: And yet, you should publish this in journals! All the journal editors listening to this podcast, hi! Upside by being the change you wanna see in the world. But what did you give them?Andrey: I gave them two readings. One was "Situational Awareness," something that we've covered on this podcast. Why did I give that reading? I wanted the students to get the insider view of what it feels like to be inside an AI company, thinking about the profound implications that might happen very, very quickly. And then I also gave them a reading that's more of a classic reading in economics about general purpose technologies and kind of the economics of whether general purpose technologies take off quickly enough and what determines how much is invested in them and how useful they are. And this is a reading by Bresnahan and Trachtenberg. And so I thought that that offered a nice contrast. Now, of course, my syllabus has many other readings that I discuss, including some other papers we've covered.Seth: Not worried that you're not making your students read enough?Andrey: So I, I'm worried. I, you know…Seth: Well, we're moving to an oral culture, right? And they're gonna have to listen to the podcast if they wanna pick it up. And so, but you're basically, your reading list is the podcast, right?Andrey: Yeah, it's a large part of the podcast, at least for this class specifically. And so it was a real joy to read for today's episode another paper that one could have put on the syllabus, but came out too recently for me to do it.Seth: Hot off the presses, listeners. Oh, and of course, before we move on, we will put in the show notes links to the "Situational Awareness" episode that Andrey mentioned so you can get caught up.Introducing the GATE ModelAndrey: Alright, so we're discussing this paper about a new macroeconomic model that is called GATE: Growth and AI Transition Endogenous model, that attempts to…Seth: Alright, authors?Andrey: Yes, we, yeah, fine. The authors are Epoch AI, et al. I'm not gonna list all of them, but you're welcome to.Seth: I'll get it. Okay, so I'll just say there's about 10 authors on the paper. Two names that jump out at me are Ege Erdil, who I know is a leader of Epoch AI, as well as Tamay. Oh man, these names are some real challenges from these AI folks. Hopefully, AI will help me. But I will say, Tamay I have met in person in Cambridge. He brings a certain intensity to these questions. I gave some feedback on this model while it was in progress. My feedback was not a hundred percent addressed, it has turned out, but happy to raise that limitation when we get to it. But anyway, so to give some context to this, this Epoch AI group is a group of scholars who have been working for the last several years on trying to track AI progress and project the implications of AI. They've kind of been ahead of the curve in talking about the implications of AI for the economy. So I take their work on this subject very seriously, even if I take it knowing that this is not straight economics; these are definitely technologists sort of first and then economists second.Andrey: Alright. So with that kind of introduction, let's talk about the priors.Our Priors on the GATE ModelAndrey: The priors. So the priors, I mean, we can't forget those. I think we came up with two priors to discuss. The first one is, is this model useful? And then the second one is the default version of this model…Seth: What does the model actually predict? So, object level…Andrey: …predicts output growth in the year 2027 of 23%.Seth: Globally.Andrey: I believe that is a global estimate.Seth: It's a global model. Okay. 23% GDP growth rate in 2027. What is your prior on that prediction? You can't… Andrey actually fell out of his chair.Andrey: Yes, I actually transcended my location in space and time.Seth: The growth created was so large, they just started instantaneously levitating.Andrey: I think it is extraordinarily unlikely that we'll have 27% GDP growth in 2027.Seth: One in a thousand?Andrey: Yeah, yeah, somewhere in that range.Seth: Yeah, I'm in one in a thousand plan too. I mean, like, the easiest way to get 23% GDP growth in 2027 would be destroying a lot of the economy in 2026.Andrey: Yeah. Yeah. Yeah. A war will do wonders for GDP growth after the war.Seth: Yeah. Broken windows, right? Andrey, you seem rather skeptical about this quote-unquote default projection of the Epoch AI model. Why were you so skeptical going into reading this?Andrey: Well, I don't wanna say I didn't know what the predictions of the model were before reading this, so maybe… but I guess 27% is just unprecedented. It is just hard to imagine in such a short timeframe, us solving all of the adjustment frictions necessary to drastically boost production. Right? And we've talked about this many times because there are so many portions of GDP that seemingly would be very hard to increase, like housing stock. Are we gonna solve all of our political issues all of a sudden? What about health outcomes research? Do we still need to run clinical trials? Are people just gonna willingly submit themselves to robot operations right away? You know, once again, I can imagine a world where that's true, but that seems difficult to conceive in a two-year span. But those are kind of my priors. What about you, Seth?Seth: Right. So I mean, I also don't think of these sorts of high-end bottlenecks constraining growth when we are talking about 27% in 2027. This is not a story about whether we'll need like twice as many people in clinical trials. This is a question about like those people who are mining ores in Sub-Saharan Africa by hand. Their productivity will go up 27% on average, right? This is, you know, everybody doing, like the millions of people in India doing low-skilled cleaning stuff, Upwork, their productivity is gonna go up by 27%, right? It's, again, I'm not, that's a little bit of a loose way of talking about it, but we need on average every sector in the economy's output to go up by 27% for this to work. And man, I do not see a path to that in two years. I am also in, you know, the one in a thousand land of, you know, 20% or faster growth rates. It would be historically unprecedented. It's hard to think about actually reorganizing a society that fast. I don't put zero probability on it, in part just due to measurement issues. Right? I could see maybe like a hundred years from now when everybody is re-analyzing the early moments of AI takeoff, maybe if you took into account all of the quality improvements that are happening in the background that in our distant future will be able to really understand how much was, you know, quality of life improving in subtle ways that are unmeasured by GDP. I don't know, maybe when AI starts taking off, and who knows exactly when that will be, 27% true increases in welfare per year. I mean, even then I say per year and then the numbers start getting crazy super duper fast. So yeah, agreed on that prior. So I guess we'll have to see whether they can convince us or not.Seth: Otherwise, maybe we can talk for a minute about the broader prior. So the broader question is, okay, we may or may not agree with this model's predictions at the object level, but maybe the way I would put it is that models can do two things: they can do prediction, but they can also do scenario planning. Right? And so maybe our second question should be how useful this model and maybe variations on this model, how useful do we think they can be for scenario planning and as useful tools for planners and policy makers? Where did you come in before reading on that?Andrey: I mean, generally I kind of take this group pretty seriously. So I think any model they produce should be, at the very least, interesting, which is a good criterion for whether a model is useful. I mean, look, without getting to the details, right, the key innovation of this model is to think about effective compute—or not an innovation, 'cause people have done this before in this community. And putting effective compute into a macro model seems like a useful thing to try. Right? So, you know, my prior is pretty high that it could be useful.Seth: Okay. So you say usefulness is a low threshold. You know, a door jam is useful. This can be…Andrey: Yeah, yeah. Yes.Seth: Alright. We'll have to add "very useful" into our next prior. But I come in sort of with that perspective, right? Which is that hopefully this can help us as a scenario planning tool. You'll see where my beliefs move. And I maybe come in with like 90% probability that a model like this would be a useful scenario planning tool, would move us closer to thinking about correct scenarios rather than mislead us away from thinking about the right scenarios to think about. That's where I started, is at 90%. I'll leave it as a cliffhanger where I end up.Deconstructing the GATE Model: The Three Core ModulesAndrey: Alright. Well, in that case, do you maybe wanna tell us the high-level features of the model?Seth: Yeah, I wanna tell you about the model. So, okay, models combining three big parts. It's got a bit where—and I don't actually particularly like the order in which they introduced the three, I would've done it backwards, but let's follow the order of the paper. Three elements:* Investment in more chips as well as R&D to make chips more effective. So there's like an investment in computers part of the model.* Then there's a second stage at which there's a translation between how much computers and computer technology you have into how many jobs are automated, as well as kind of your productivity in using computers to automate jobs. So first section is how do we get more computers? Second section is how do computers turn into automation?* And then the final section is a pretty standard off-the-shelf representative agent, semi-endogenous growth model. Right? You know, it's got all the hits: it's got a CES (Constant Elasticity of Substitution) production function over all of the different tasks, it's got a representative agent with an intertemporal Euler equation. All you macro folks in the audience, you're gonna be eating this stuff up.So those are the three big elements. I think you would think that these are kinds of the elements that you would want in a model of AI takeoff, right? Because if you think computers are what drive automation, you need both the investment in computer side and you need the automation side. If you think automation changes our productivity, our output, our ability to reinvest into new computers, then you definitely want a connection to the real economy. So I think whether or not we think this is an adequate list of things you would want in a scenario planning tool, this has definitely got three essential things you would definitely need. So what do you think at the high level, do you think this has got the right elements?Andrey: Yeah, so I think those are kind of pretty critical elements. You know, a lot of the paper, it seems like a lot of the effort actually went into figuring out, you know, it's not computers that's the output, right? It's the effective compute, which is a function of hardware and R&D and software R&D and so on, right? So they kind of spend a lot of time thinking, maybe formalizing some of the reasoning in "Situational Awareness" about the orders of magnitude of effective compute. And that to me seems like there are so many functional form assumptions in that entire exercise that I would've been happier to just skip that micro-foundation and to just say that we can, you know, invest directly in effective compute. And then there's some sort of, you know, elasticity involved there. And, and call it a day.Seth: Kind of. Yeah, I think that's basically right. I think the model is basically the most plausible when we're in that linear zone, and the really wacky stuff happens once we hit like the tops of these sigmoids. So I, yeah, I agree with that. In the compute side, there may have been like a little bit of sort of over-modeling of what's going on. It's like, given that they're immediately—and we'll talk about this in more detail in a second—in the automation side, I kind of feel like that's where I wish there was more thinking.Andrey: Of course. Yes.Seth: It's sort of just kind of posited sigmoid shape, relating the amount of effective compute to the amount of automation. It's not really particularly justified by anything. They just like sigmoids. The functional form also seems a little bit arbitrary. We can get back into the details of what we like and don't like about that, but that is the essential question. Like, what is the conversion between resources poured into AI and effective jobs taken? And unless you've got a really good answer there, it's hard to be satisfactory on the other sections.Andrey: Yeah, and importantly, right, the model models task automation in a kind of very reduced form way. There's some tasks that are easier to automate, there's some tasks that are harder to automate. You're gonna go through that full automation cycle in some amount of time. There's gonna be a shape associated with that. That's kind of made up. But I think they don't think about the production function very hard and that, you know, it's very easy to come up with examples where task automation is not gonna improve productivity very much. Right? You know, the task… for example, the task automation of creating the transcript for this podcast has been a solved task.Seth: Oh.Andrey: Well, actually it's not true because even still I'm tweaking it once in a while, but it's mostly done. Right? But it, you know…Seth: Fewer racial slurs. Right.Andrey: Oh, come on. I only, my key form of slurring is anything that has to do with [bleeped]. If it's a [bleeped], I slur it. That's the only thing I slur.Seth: You bleep that out, guys. Bleep it out. Listen to whoever's recording listening to this. Bleep it out when he says, "whenever I talk about [bleeped]," you bleep that part. Keep the rest. Alright. Alright. The third part of the model…Andrey: But anyway, our production function, our production function for this podcast, right, certainly includes a task of transcript. But I would say that if we didn't have that automatic transcript generation, we probably just wouldn't have a transcript. Right? There's kind of a lot, like, there's a lot of these things in production where, you know, what is a task, what is a job, what is the production unit? You have to start thinking about this pretty hard if you want to get correct implications of AI being capable of doing some things, not other things.Seth: I wanna draw an important distinction here, right? Which is you could, might believe that they got two different things wrong. The first question is, do you think they got wrong the rate at which effective flops turn into automation of tasks? And then the second question is, do you think that they got the way that you combine tasks, right? The way that this paper does it is drawing from Acemoglu and Restrepo. It does that beautiful, beautiful, silly thing of saying that the output of all tasks and the output of work in the economy is a constant elasticity of substitution function between all of the tasks in the economy. And then they plug that into a Cobb-Douglas. We'll come back to that. Okay. In other words, there's, let's say there's three tasks in the economy. There's clipping hedges, you know, being a doctor, and flying planes, right? They say those are three jobs. And they say we've already automated flying planes. 'Cause we, right. They assume that we started with 10% of the jobs in the economy are currently automated, by the way, in terms of just like funny numbers that come out from nowhere in this paper. That's one of my favorites. It is right now 10% of jobs are automated. No idea where that number is from.Andrey: Well, you know, we have calculators, right? So before we would've had to do the calculation by hand, right?Seth: Exactly. It was, that was exactly the percentage of time. Okay. So you got these three jobs. There's the first question of, as we get more computers, how do we replace those jobs with AI? How many computers do we need to continue pouring into the process? So that's a good thing that this paper does really well, is distinguishing between training compute to extend the variety of tasks you might automate, and then runtime compute, which they view as like AI workers who are perfect substitutes for humans at the task. So that's the first, like, do you think they get that right? And then there's the second part, which is really magical, which is it turns out the economy is a mix of those three things mixed together. But importantly, they all have the same elasticity of substitution. So now you might think, so in the example that you just gave of our transcript, it really sounds like we have a beautiful podcast product even without a transcript. Right? You would say probably that the transcript and the podcast itself are substitutable in the sense that they can be enjoyed separately or together, your consumption of one, if anything, they slightly crowd out each other, right? They're kind of more substitutes than they are complements, right? Whereas, you know, somebody washing your hair before you get your barber cut and then somebody actually cutting your hair, those are sort of essential complements. You gotta do the first in order to do the second. This comes even before we talk about splitting up jobs across people. So, why am I building up to this? The premise of this paper requires that every pair of tasks have the same elasticity of substitution. In other words, this paper requires you to take a stance on the elasticity of substitution between trimming a hedge and driving a bus. I don't even know how you would start estimating that elasticity of substitution, Andrey. And yet this paper thinks there's one number that you can just go out there and know for it.Andrey: Yeah. Yeah. I mean, to be fair, they're not unique in this since macroeconomists do this sort of stuff all the time. But I do think, you know, in this case, it is very important to get this right. Let me ask you another question. Let's say that the AI starts to be capable of automating more and more tasks. Do you think the productivity gains are gonna be higher when the first, let's say 20% are capable of being automated or when, let's say we move from 60 to 80% being automated?Seth: Right. So my answer is gonna kind of be uninteresting 'cause it's not based on the AI part. It's kind of based on the econ feedback. I always anticipate the growth rates being faster at the far end than on the close end. And the reason for that is not something to do with the technology, it has to do with the economic feedback loop, right? When you automate 20% of jobs, you get your GDP go up a bit, which means that if your saving rate is constant, your investment rate goes up a bit, right? There's this positive spiral between productivity go up, investment go up. So I would always anticipate the greatest gains to come towards the end than towards the beginning.Andrey: Mm-hmm. But, and you don't think that people will anticipate that we're gonna hit a utopia and stop saving?Seth: And just ahead of it. So let's table… So I think in limitations, let's talk about saving dynamics in this model, right?Andrey: No, no. But let me just say that, you know, even without thinking very hard about saving dynamics, my intuition is that there's a lot of complementarities in production processes, even if specific tasks might be substitutes. And so productivity gains are gonna be greatest when you can nail all the complementarities with AI in one shot. If it is kind of, if you're starting to solve last-mile problems, then you can like literally abstract away from certain production processes and then truly scale 'em up in a way that you can't if, as long as there are humans involved to a major extent, at least in some part of the production process.Seth: Right. So if, yeah, so let me put it this way, whether or not we think that the jump between 20 to 30% has a different effect than the jump from 80 to 90%, it's very clear that the jump from 80 to 90 is extremely different than the jump from 90 to a hundred, right? And like part of this is just mathematical, right? If you go from 90% to a hundred percent of your jobs automated, you've now eliminated a hundred percent of your labor demand. But if you go from 1% automated to 2% automated, you've reduced your labor demand by 1%, right?Andrey: Yeah. 100% is a very stark number. Right. But I was more just saying… No, I know, I know. I guess what I was just saying is that, you know, even if we haven't automated a hundred percent of tasks in the economy, we might have automated 100% of the tasks in a particular production process, right? So that could be long before we hit 100% of all tasks.Seth: And this is one way that Acemoglu and Restrepo, I don't know how much they're able to bring to this in terms of data, but their modeling framework explicitly says you might have a different CES aggregator in this industry than that industry. And I would say it's easily extendable to, you know, thinking about CES aggregators within jobs or within occupations between the different tasks.Andrey: Well, and I also thought you were gonna say there were gonna be new tasks that are…Seth: Oh, we're getting to new tasks. Dude, there's a lot. This favorite, this thing might sound… What I think what I'd like to do now is maybe let's go through the three modules now in more detail.Andrey: Mm-hmm.Seth: Beautiful. Alright.Module 1: AI Development (Investment in Compute)Seth: So we got these three modules. It's how do we get more AI technology? That's through investing in computer capital and computer R&D. How do we automate based on that computer? And then finally, how do we get the macroeconomic growth? And then these three, of course, all flow into each other. Starting with the AI development module. The main kind of thing in this module is effective compute. We're interested in how effective compute grows over time. And effective compute can be devoted either to training or to inference. Training is kind of when we think about spending $500 billion to make, you know, GPT-6 and it's gonna think really, really hard and build this really giant model that you can then run more cheaply. That's called inference compute. And so once you've trained a model, you can do inference compute. This is when you type in your queries to ChatGPT and says, you know, "Ghibli-fy this picture of me punching my neighbor." Right? So that's a lot cheaper, but you still need, there is a marginal cost there. Right? Before I go into more detail here, I mean, I already think that this is an innovation that I really have not seen thought hard about in other econ papers—this distinction between compute devoted to training and inference. I think thinking about these sorts of details is a big step in the right direction.Andrey: So I actually think, yeah, modeling this is actually really interesting and practical in some sense, right? If you're an AI lab, you must be thinking about this all the time. In fact, the question of compute allocation, I feel like is a very important question that the rest of the world kind of hasn't seen a lot of work on because it's so trapped in the labs. But it seems, yeah, it is just fascinating. That said, I'm not so… it just, I don't see this as an essential part of a macroeconomic model in the sense that like, you're abstracting from so many things and you're essentially in the end getting a quality-adjusted compute, and do we really care exactly how you're getting it? I think that's a little less interesting to me. I think more interesting to me is this question of like, we have effective compute. We can use effective compute to do a task already in the economy, or we can devote it to additional, you know, AI research.Seth: So that's, I would, that's almost like the operationalization of the distinction that they make.Andrey: Yeah. Yes. Yeah.Seth: So, you're right that that's kind of why the framing's important. But you might not, you might think that this may be a little bit too detailed for a macroeconomist to talk about. I'll say is, and I think that this may be speaks to why this is too much detail than they're able to actually work with, is they immediately move to this rational social planner framework where the social planner is gonna make the optimal mix between training and inference compute. And like the reason you would introduce the distinction is if there is some sort of divergence there where maybe…Andrey: Yes, of course.Seth: I mean like it's easy to think about why that would be wrong, right. In a race scenario, you expect lots of duplication of efforts on the training side. I think that should be our default assumption.Andrey: That, I mean, that's fascinating, right? Because now we're going back to my syllabus, is that the paper on general purpose technologies kind of suggests that we have vast underinvestment in general purpose technologies because you don't appropriate the gains. So which one of them wins out, the race condition or the under-appropriability of the research? That's not obvious to me. I would guess, I would guess actually that the under-appropriability of the research wins out. That's kind of my guess. That it's bigger.Seth: Okay. Andrey, you're making a huge point here, right? Which is that the model is assuming that like perfectly rational social planner gets a hundred percent of the gains from automation. Slight mischaracterization of reality. I’m not even talking about like we could get overinvestment because of race scenarios. I'm thinking about wasteful duplication because of race scenarios, which is, you know, of course you can get in all-pay auctions, which is kind of what a race is. You can certainly get overinvestment in aggregate. Yeah, I mean, it just goes to show that this is a little bit of a simplification.Andrey: I mean this is macroeconomics though, right? I mean, this is more your world than mine, right? But isn't it always a simplification?Seth: So how would I think about this? So, I mean obviously macroeconomists have a lot to say about the appropriability of innovation. Obviously that's usually ex-post, it's really hard to do ex-ante. But I think the idea of training being completely, not being no duplication there, I think that's of first-order importance. I would divide all of these training numbers by five leading labs. Now maybe it turns out 'cause things are growing in orders of magnitude that like dividing by five is only gonna slow things down a single year. But I'd love to see that as a module here. Like how much redundancy I think there is.Andrey: And to be clear though, some of the compute is not spent on R&D. Some of it is spent on other things, right? So in that case, there wouldn't be duplication. So only partial spend of the compute is potentially duplicated, right?Seth: Let me, let me put a very fine point there. Compute is used for two things. It's used for automating new jobs and it's used for running that automation, the runtime compute. The R&D actually just comes out of the general government budget. It's like a, we call, used to call this in macro… this is like a laboratory equipment model, right? For more AI research, you just put like fancier beanbag chairs in the AI research lab. Right? I don't know, do you, are you okay with like a linear function mapping R&D investment into R&D research? Or, I mean, should we really be thinking about like a scarce amount of geniuses who really move the field forward?Andrey: Yeah. Yeah. I mean, this is, we've been talking about this topic in several of our podcasts, right? Have we run out of geniuses? I mean, look, I think there's a question of practically can you get people to move into AI research? I think there are definitely way more geniuses than those that are working on AI research. I don't think, you know, you can see people have entered the field with very little kind of prior training and have been very successful. So I just don't believe that we're anywhere close to tapped out on talent. But I think getting the talent in is hard. Like, think about, you know, certainly some of our colleagues in our profession could be great AI researchers, and yet they have not been, you know, successfully converted. Like they haven't dropped everything they're doing and started, you know, working on AI or, you know, let alone working on advancing Frontier AI at a research lab.Seth: Right. This reminds me of the Bai, Besley, and co-authors paper, right? Is AI coming? Well, are smart people acting like AI is coming? (Editor's note: Referring to the paper "Are We Saving Enough for the AI Revolution?" by Bai, Baslandze, Besley, and Jäkel)Andrey: Some are.Seth: Some are. That's the answer. Alright, any thoughts you wanna add on the compute module before we move on to the automation and work module? You wanna talk about this orders of magnitude of compute thing number that they plug in?Andrey: No, no, I mean, I guess there's a key assumption, right? That there's some amount of compute that gets you automation, you know, that gets you full automation, right? Between 10 to the 27 and 10 to the 41. They just know those. That's the range. And look, like I'm willing to buy that that's enough compute to achieve full automation. I have no doubt. But the question is, conditional on having that compute, are we guaranteed to get it? And how long will it take to get? And I think, you know, if you posit the kind of self-improving AI models world, then you'll get it pretty quickly. But if we haven't figured that out, then it may take a long time, even with a ton of compute.Seth: You're talking about data pipeline here, right?Andrey: Not just, not even just data pipeline, just, you know, we haven't stumbled on the right algorithm and or the right way to un-hobble the model or, you know, whatever.Seth: Well, I remember when we talked about "Situational Awareness," episode two, a lot flip back or whatever episode it is. We know I came out of that feeling like there's approximately a 50% shot we can do AGI with current architectures versus we need like a whole 'nother, you know, paradigm shift in innovation. Right. Is this kind of the same question for you is like, do we need one more paradigm shift or is it this plus scaling enough?Andrey: Even if we don't need a paradigm shift, let's just say like we just need to rely on reasoning as currently construed, getting it the right way to reason to do what we want it to do in the right way. Right? Like, it might take time, it might take time to figure out how to do that. Right. There might be some diffusion problems as well, you know?Seth: Some of that we'll talk about when we hit automation and we hit the next two modules.Module 2: Automation and Work (Compute to Automated Tasks)Seth: Okay. Module number two, automation and work. So here we get a function that maps from the effective compute on training, which is, by the way, completely cumulative. It's not like, you know…Andrey: Yeah, yeah. That, but I guess if you're increasing like orders of magnitude per year, it doesn't matter. 'Cause most of it is new compute, right?Seth: Why not make it a stock? Why not just make it a flow, whatever. Okay. And you might imagine that you have to retrain at tasks as society changes over time. So just thinking about it as a flow might not even be that bad. Table that question. And actually, this is something that I've thought about, which is like, what if AI changes the world faster than you can train AI to do jobs in the world? It seems implausible, but right. You can build scenarios where, you know, the rate at which new tasks spawn is faster than the rate at which things are automated. Maybe it's not our modal scenario, but it seems like you'd want a model to allow for that. Let's come back to that. Okay. Automation and work. This conversion between amount of effective compute to the percentage of jobs that are automated. It is a sigmoid. You basically get two numbers to shape the sigmoid. First, how much compute do you need in order to make, you know, the super AI that can do everything. And pause for a second here. This includes all physical tasks, right? There's like, no…Andrey: Yeah, so that means building all the robots. Just to be clear, all the robots would need to be built.Seth: All the robots. That guy, you know, that Sub-Saharan African who is like mining for diamonds by hand and being paid 50 cents a day. That's the job we're going to automate, right? Don't think about like, you know, some dude sitting in an office. We're talking about a hundred percent of the tasks, alright? And it's a sigmoid and you get to choose how many flops to, you know, create the super intelligence. Right now to, let me give you guys some context there. OpenAI's GPT-4 was trained with 10 to the 25 flops. Right now we're seeing runs that are kind of on the order of 10 to the 27 flops. And according to Epoch AI's default model, it'll take 10 to the 36 flops to create the God machine, which they anticipate coming in 2040 or 2035. And the maximum they will allow you to put in before judging you into their model—and by the way, everyone should go online and play with their model and plug in different numbers—is 10 to the 41, which would put AGI somewhere out in the second half of the century. Hmm. It's a pretty explicit range. And okay, so that's the first parameter you get. And the second parameter you get is what percentage of the way up the sigmoid is that inflection point. So do you want the inflection point towards the end or do you want the inflection point towards the beginning?Andrey: Yeah.Seth: How do you, how do you parameterize either of those?Andrey: I mean, it's hard. I mean, one of the nice things about this paper is it has this website where you can fiddle around with all the parameters and kind of see how it changes things. So what I was just doing is changing a key part of this, which is the flop gap fraction, which is the range of effective compute over which all the automation happens. And, you know, the results are quite sensitive to this gap. So they assume it's 55%.Seth: And just so the audience at home gets it, this is what I'm calling the, where is the sigmoid? Is the sigmoid at the beginning with the ramp up, or is it more towards the end with the ramp up? Okay, continue.Andrey: Yeah. So if you make it 40%, then, you know, we get full automation a bit later. Interestingly, we don't get the massive GDP increases until now 2028 instead of 2027. So, you know, we push it back a year.Seth: Delay that AGI party, dude.Andrey: So we should already be quite skeptical of this particular part of things. It's like, you know, no one has a clue about this parameter. So the fact that it's shifting around the model so much is suspicious, right?Seth: Well, it's also that like I can't put in any parameters, it doesn't let me put in any parameters that don't seem silly. Right? I literally put in the maximum that it would let me for how long until AGI hits and still we get, you know, if I put in the maximum value that they will let me, have economic growth of 10% rates globally by, you know, by 10 years from now. Right. So like the most pessimistic scenario you are allowed to plug into this model has like AGI takeoff, you know, just a decade later.Andrey: Yeah. Yeah. Or like another implication is like in the next two or three years, we should be getting extraordinarily high growth rates already. Like regardless of how we parameterize this model, we're always getting insane growth rates in the next two or three years.Seth: Yeah. Let me see. With my super pessimistic… Yeah, exactly. Like I say, in my super pessimistic, as pessimistically as they let me plug in version of this model, we get 10% growth rate in 2030.Andrey: Yeah.Seth: So yeah, it's like, I mean, it seems like the model, even if you think that the median scenario is takeoff, which is, you know, hands in the air, you know, take a step back. It seems like your model should allow for a non-takeoff to be possible.Andrey: Yeah.Seth: What else do you wanna say about this automation conversion, other than it's very difficult?Andrey: Well, I mean, they kind of allow for two versions of the labor reallocation. One is where it seamlessly gets reallocated to all the other tasks. So let's say we automate gardening, you know, well, gardening isn't a task we automate, like mowing the lawn, hedge clipping, whatever. And now we're just gonna put you into a task that hasn't been automated yet, like I don't know, delivering food. And so, you know, that doesn't seem like a great assumption to assume seamless labor reallocation globally. But the opposite assumption is zero, is that that labor just goes away. It just stops.Seth: You give up your job. You were born to…Andrey: Yeah. Yeah. Right? So neither of those assumptions is particularly satisfying. What do you think about, like, what do you think about just this "number of AI workers" style modeling? I think that this is the best version of it that I've seen in terms of an AI worker is how much inference compute you have divided by the compute requirement. That kind of seems right. They can kind of plug that into a production function with non-crazy things happening. The crazy things happen because of all the ancillary stuff. I think that, and the way that they think hard about measuring the compute requirements for AI workers, at least calibrated on current data, I think is okay. There's this issue of, can you extrapolate that out to the future? But I thought that that was maybe the most sophisticated version of this that I've seen.Andrey: Yeah. Yeah. I like that idea. I was thinking about the energy requirements. It wasn't obvious to me whether that was in any way baked into it. Right. So that seems…Seth: Yeah, energy's in F.Module 3: The Macroeconomic Engine (Growth and Production)Seth: Maybe. Let's do the macro model. Okay. So we plug those automations into the macro economy. Macro economy is a Cobb-Douglas production function, so that means fixed income shares across workers plus AI workers, physical capital that's not computers (all non-computer capital), and then this mysterious other thing called F. Might be land, maybe it's energy, maybe it's land that you can put solar panels on. Andrey, there's, you know, energy snuck back in. It's, you know, plutonium reserves.Andrey: Yeah, I thought that that was an interesting thing to think about. Like, you know, I think one of the things economists almost always tell AI people in these discussions is that there are certain things like beachfront property that are hard to imagine increasing due to AI. I mean, you can imagine it, of course, but, you know, people wanna live in specific places, so on and so forth. There are so many.Seth: I just invested my entire life savings into Southern California real estate. So…Andrey: Oh, congrats.Seth: Yeah, just put down a 20% deposit. So, probably the right time to get out of the market, right? As we record, for future listeners, Trump's [bleeped] have just hit the global economy over the head. So I don't know. Maybe we'll do a [bleeped] episode, [bleeped] and AI episode someday soon.Andrey: That would be fun. But yeah, I guess what I was thinking here is that I can imagine here, Cobb-Douglas, right? It's a very simple model, but it does seem that perhaps these like scarce resources will bind more and more the more other stuff you have. And there isn't kind of a sense that this, you know, this is not a model where that happens, right?Seth: It reminds me a lot of my model… well, it does remind me of your model. It reminds me of your model a lot. Yes. Reminds me of my model with Erik Brynjolffson currently under review, "Digital Abundance and Scarce Genius." Whereas AI becomes more productive, there's a scarce complement of certain kinds of workers who are able to implement the AI. And if those guys are gross complements to the AI, then their share of the economy will increase, and that'll show up in things like rents to entrepreneurs, the compensation of CEOs. We have seen that. So I think a really natural and sort of easy extension to this model is just to have that F guy be a gross complement to everything else.Andrey: Yes, yes. I totally agree.Seth: What else do we wanna say about this? Oh, let's talk about the representative agent a bit 'cause I wanna smash this guy around. Okay. So there's a representative agent in this model that makes all of these investments perfectly and rationally to maximize lifetime welfare. Alright, I don't know if you've been to the world today, but there's a little bit of a disagreement between countries about where production should be located and how much investment should happen in the future. You know, on its face, this seems like the incorrect way to model how the world works. Even if you wanted to kind of abstract away from country-level tensions, there's this issue, which is that individuals are definitely situated in their life cycles when they're making savings decisions. For example, we just read that Bai et al. paper that really emphasizes—you know, that's a paper that says, because interest rates are low, AGI isn't coming soon. In that paper, people might dis-save because of the incoming AI shocks because they're worried that their money will be, you know, super… they'll be able to buy whatever they want in the future anyway, so let's move consumption to the present. That kind of does happen in this paper, right? So they have an elasticity of substitution between, or rather they have a, it's called a risk aversion preference. But in this context, we'll think of it as a "how much more do you save when interest rates go up?" preference. In this model, they choose a parameter such that when it looks like the future is gonna be really good and interest rates go up, people will dis-save, right? I think that's right, but I think this model perhaps even underestimates the extent to which the dis-saving will happen. To the extent that you actually get severe kind of reductions in the ability of the economy to reinvest into the next generations of technology and the next generations of physical capital that are able to, you know, actually implement these AIs. So I think, you know, and the dynamic that I focus on is this question of do the people making capital income have the same marginal propensity to consume as the people making labor income? But this model posits the most massive shift in who makes money of all time. It is positing that we go from two-thirds of the money being made by workers and one-third by capital to a hundred percent of the money being made by capital. That means different people are going to be making, spending, and saving decisions. And I think more important than some sort of representative agent's gross utility function, which doesn't make even any sense anyway, is like, are we reallocating money towards short-termy people or long-termy people? I think that's the relevant question.Andrey: Hmm. I mean, I do think this ties very much into just the question of appropriability and kind of is the economy over-investing or under-investing in AI technologies in general. Right? I mean, it's easy to pick on their representative agent model. I mean, I guess given this is the first model with effective compute in it that's a macro model, I'm not like, offended that they would make it a macroeconomics model. And another thing about, like all of Chad Jones' papers are almost all representative agent models, and we…Seth: Shout out to Chad Jones. Listen to the previous episode. See the show notes.Andrey: I think we thought those papers were very useful. Right? So I'm not offended by this, you know, but at the same time, it's not adequate. And there's even a sense in which it's not optimistic enough.Seth: Mm-hmm.Andrey: Why? Because the overall technology level in the economy is not influenced by the level of compute.Seth: Right.Andrey: What do we mean by that? So in this model, even though everything gets automated and global GDP shoots through the roof, we haven't used this technology to invent any new technology.Seth: No, not a single new thing. There's no capital deepening at all in this model. There's…Andrey: Yes. Yeah. And capital is just as efficient as it was before it, you know, going back to our previous discussions, right? It's not, capital's not been made more efficient, which is, which is, you might think kind of ridiculous here because, you know, if the AI can optimize, you know, a factory operation… Let me give you a very simple example. You're running a factory or warehouse and now you start using AI to optimize when you turn on the heaters and the coolers in the building. You know, you're becoming more efficient, and in principle that AI would help a lot with this sort of problem. Right?Seth: Exactly. That's the point. Like one of the points that all of these automation-focused AI papers tend to miss is that AI is most useful at tasks that are already automated. And that's just missing here. And it's gonna be really hard to say that these are realistic projections without that critical element being included.Limitations of the GATE ModelAndrey: So do we wanna go to our posteriors or do you have any other discussion topics?Seth: Let's hit my limitations and let's see if there's any we haven't hit. We talked about this sort of simplifying assumption that the compute stock is just aggregating over time. There's no sense in which like, you know, they get deprecated or, you know, you wasted a run, but whatever. That's, that's of anything a limitation I'll tolerate. Even though we talked about that, race scenarios are probably more likely. We've talked about this issue. No non-automation tech gains, we just covered. We talked about how it seems on its face absurd to try to estimate the elasticity of substitution between clipping a hedge and pouring a latte. And yet that's a parameter the model expects us to just know. I guess I would recommend those playing around with the model to err on the side of the really, really sort of close complements. And that's not because I think the average pair of tasks in the economy aren't substitutable. In fact, I think probably putting hedges in pouring lattes are pretty close to substitutable. Rather, what's gonna hold back the economy is not the majority of tasks that are substitutes. It's the minority that are close complements, right? That's where the bottlenecks come from. You wanna riff on that or you agree?Andrey: Yeah, I agree. I agree with that.Seth:No creation of new tasks or a way for the labor share decrease is pre-programmed, so it's not a prediction that the labor share will go down. It is baked into this paper. Limitation.Andrey: I mean, I think it's probably a reasonable assumption though.Seth: But I would want a model that allows for the opposite to show, well, for all these parameter spaces, it doesn't happen. That's kind of what I, but, you know, creation of new tasks, that's another functional form would be…Andrey: Creation of new tasks is interesting. I'm more thinking about labor, I mean. Global labor supply should be going down due to the fertility rate decrease. I mean, I don't think they should try to tackle that question here.Seth: Right. Exogenous? Yeah. Let, to me, we're okay with population growth being exogenous. Do not try to endogenize that with the sex robots. R&D uses raw GDP as input rather than scarce geniuses. I think you basically are comfortable with this. You think that there's spare brain capacity for AI if we threw money at it, but I don't know. At a certain point…Andrey: I think there is adjustment friction. I think there's spare AI—sorry, there's spare talent, but convincing it to work on AI is not that easy.Seth: Fair enough. Yes. They're probably obsessed with something else like model trains or painting Warhammer figures. Physical embodiment necessary for some physical AI tasks. So this model basically treats all physical capital as the same, but if you really were taking this model seriously, it seems like in order to get to the full automation world, you basically need to replace all of today's capital with a completely different capital system. Right? And so basically the physicality of many of these tasks, I think is just basically under-thought about by this model.Andrey: Yeah. And that could be, by the way, like a very reasonable thing that could be very slow, right? Like building, just thinking about car production processes. You know, it's hard to build a lot of cars, but now if we wanna build a lot of robots, that seems like a similar complexity issue. You can imagine that, for example, we still haven't electrified the entire car fleet, and thinking similarly about robots, it could take a while.Seth: Right. Last and most important topic, not to beat around the bush, which is the super simplified saving and reinvestment decisions. So we talked about why that's wrong in a race scenario, but I just wanna emphasize this, which is, in my opinion—I told Tamay this when we sat down for lunch a year ago—I said, have an exogenous saving rate. Right. And then I can play around with whether I think the saving rate's gonna go up or go down. Because basically when I play with this model, the only thing that that representative agent's welfare function thing does is pin down the saving rate. And it does it in kind of an unrealistic, and in my opinion, confusing way. That actually has like a lot of leverage over welfare implications when we don't want it to do that. We just want it to give us a saving rate. So just f*****g have an exogenous saving rate and then you can cite my paper saying it'll go up or go down, cite somebody else's paper saying it'll go up or go down. Andrey, back me up on this.Andrey: Yeah, I mean, I don't have as strong of an opinion as you on this particular question.Seth: There’s this huge government lever on the saving rate, right? Which is you can run giant deficits or not. That's a choice variable. That's completely unmodeled here. Just let f*****g…Andrey: Yeah, no, no, no, that's fair. You know, and yeah, and just in general, if we think about the scenarios with a Manhattan project where like, you know, Leopold convinces the government to do it, you know, that that's gonna posit a very different savings rate or investment rate than models where it doesn't happen.Seth: Precisely well put. Right? So we kind of politically have decisions about how much we wanna invest in this technology. It's not primarily going to be determined by welfare decisions of this one theoretical global representative agent. So it seems like the wrong approach there. I'm ready to move to posteriors if you are, Andrey.Andrey: Alright. Yeah, I'm ready.Our Posteriors: Has the GATE Model Shifted Our Beliefs?Seth: Alright, so Andrey, the first question we asked was: do we think that GDP growth will be above 20% in the year 2027? With what probability are you at after reading this document?Andrey: I mean, look, it's still tiny. I mean, I guess if I have to be honest, it should update it a tiny bit, but it's a tiny bit on a tiny bit, so it's still quite small.Seth: Going from one in a thousand to one in 999.Andrey: Something like that. Yeah.Seth: Where do I come at this 20% growth rate in 2027? Am I moved? So I came at this with also thinking, you know, maybe one in a thousand or less chances of this happening. Read this paper. It moves me in the direction of takeoffs leading to large numbers in GDP. So here's the thing is like even in the, I'm like trying to talk myself into it, right? Like think about the world where like literally we got AGI tomorrow, right? And I think that's like the only way we could even get 20% growth in 2027, right? We have AGI tomorrow, it's just a matter of compute to do any, let's say, AGI for non-physical tasks. It's physically impossible for us to physically automate all jobs by 2027. So let's say that 25% of work is like theoretically automatable without new capital deployments. So like, let's say that's the remote worker share of employment is 25%. You'd have to do a hundred f*****g percent of that being automated, right? This is the, this is kind of, now I'm using the simple macroeconomics of AI. (See show notes) To try to like back of the envelope this. And 2027 is too soon for that capital reinvestment feedback loop to kick in. It's too soon for physical stuff to be automated. The only way you'd ever get to 27% would be by counting either deploying a huge share of the economy, which that wouldn't be GDP growth, that'd be like productivity growth. Or through like, kind of these quality improvements. And the model doesn't talk about quality improvements, right? The only way you could actually get 20% growth in a year is if like all of our digital services just magically were 20% better. And somehow GDP captured that. All digital services would be like 80% better, and somehow GDP captured that.Andrey: Yeah. Yeah.Seth: GDP is not good at capturing that.Andrey: I mean, it would have to be like, you have an artificially super intelligent agent, and now it has magical powers because that's how these things work to convince everyone to do everything at once. And then it appropriates the resources to develop a Von Neumann-factorial style factory that operates 24/7 at super speeds. You know, physically it's possible. I guess it's totally physically possible to get 20% growth, but the scenario is very knife-edge.Seth: Yeah, I think it's, I can't get my brain there. I'm staying at one in a thousand. If anything, like thinking through the scenario harder kind of moved me a little bit away. So I have to say I got a little bit anti-persuaded about that specific claim. Now, but again, that's even with thinking that there is some percentage chance that we have something like an intelligence explosion in the next few years. My objection really as an economics expert is the translation of that intelligence explosion into GDP growth in that timeframe.Andrey: Yeah. Yes. More so than the technology, which I think we both agree there's a high chance we get just through scaling alone, very powerful technologies. I mean this is also related to, I think, to the J-curve idea, right? So, you know, oftentimes—this is a paper by a friend of the show, we'll cite it.Seth: Daniel Rock, friend of the show. We know you're listening. Out of Wharton…Andrey: Oh, of what, well, Wharton doesn't have the best reputation these days. But essentially like you get a new technology, and oftentimes what happens is various organizations spend a lot of time investing in intangible capital. So things that aren't easily measured, like better organizational processes and things like that. They devote a lot of resources to that that doesn't show up in output, and it shows up in output a lot later. So I could totally see this being, you know, happening already, right, in some sense. Right? A lot of organizations are already trying to restructure processes to become more productive. But we don't see that in GDP growth right now. But we might see it, you know, five, 10 years from now, right? So, yeah.Seth: Yeah. One more reason why we should expect the measured gains to kind of happen towards the end rather than towards the beginning. Okay. So now to the sort of the meta question, right? Which is, okay, maybe we don't think this is a useful tool for prediction or a super useful tool for prediction. Can it be useful as a scenario planning tool? Where do you land there?Andrey: I wouldn't think about it as a scenario planning tool necessarily. I'd think about it more like it's bridging the conversation between technologists and economists, and it's creating a better bridge than what we had before. So, you know, assumptions are stated more clearly. What technologists think is important is stated more clearly. And now we have maybe more to grasp onto, kind of here are the key missing elements or not. And so it's gonna move the conversation forward. And it's also, you know, interesting to tweak around the parameters and kind of see what happens.Seth: You can either get 20% growth tomorrow or in two weeks. I agree with you. Well, let me tell you where I land on this. I land on this is it's not a good prediction tool for the reasons that we've talked about. On the one hand, the short-run predictions are absurd, and on the other hand, I don't know if you've played around with seeing what it predicts after full automation, but it just is like, s**t, right? It just like, basically the model gives up. It's like GDP growth fails to have any meaning.Andrey: Well, it doesn't, Seth, it doesn't have the utility of AI agents, so how could it possibly work?Seth: A, it doesn't have the utility of AI agents. And then second of all, it says like, the utility of humans is like maxed out at like, you know, 2.5 times America, right, with that strong concavity in the utility function. So yeah, that's a problem. I guess what I would say is that it's so, it's bad at predicting in the short run. It's definitely, it's never claimed to be good at predicting in the long run. So it can't be a good prediction tool, at least in my opinion. So that leaves us as sort of a scenario planning tool. Maybe you have a third category, right? Which is like an intellectual bridging tool. I think you're actually right about that, and this effort scores points on that. We are now bridging communities, getting these numbers to talk to each other. If the numbers say something silly when you put the numbers together, either the move is, there's something silly about the numbers, or people f*****g better get ready for the explosion. Tamay and the gang at Epoch AI think the latter. But maybe we can learn the former instead. Maybe what we actually learn is that there's something silly about some of the numbers we plugged in.Andrey: And to be clear, I think there are plenty of people at Epoch who don't believe in like a two-year takeoff scenario. They believe more like a 30-year takeoff scenario. Right. So it's not like they even think that.Seth: Well, it's not when you talk to Tamay. That's not Tamay.Andrey: Yeah, fair enough. But I was listening to, they also now have a competing podcast. I don't know if I should be promoting…Seth: No, don't mention them.Andrey: There we are, gonna collude against our competition. But yeah, in that podcast, they say substantially longer GDP takeoff timelines than two years.Seth: Alright, well, there we go. We have to get them. What I would give for a one-handed AI and technology economist. Alright, so what are my last thoughts here? My last thought is what would make this better as a scenario planning tool is if there were explicit introduction of the relevant levers that policymakers have in order to kind of nudge this one way or another. It doesn't need a detailed version, but what's a version of this where the government has some regulatory choices that maybe changed the conversion rate of AI compute into automation, right? And that could be either thinking about like occupational licensing or regulations, or, you know, safety checks that slow down development, right? So I'd wanna see kind of that knob in here, like a government "how much do we wanna speed up or slow this down" knob, as well as just sort of government fiscal policies, right? So one thing I really think super hard about in these fast AGI takeoff scenarios is the sustainability of government fiscal policy. Andrey, as you may or may not know, Elon Musk recently announced that Social Security is a Ponzi scheme. He is correct. It is a Ponzi scheme. And the government needs money to pay its very many Medicare, Medicaid, Social Security entitlement benefits. What's going to happen in the next 5, 10, 20 years is that if we actually do get an AGI takeoff, there will be an increase in growth rates, which should hopefully help fiscal sustainability. On the other hand, one huge new call for government spending, whether that's social support for people losing their jobs, or whether that's military spending, as we get into some sort of crazy f*****g arms race. At the same time, interest rates exploding. Most government debt is short-term. Interest rates go up enough, this is unsustainable. And so what I think is somebody should build a tool that's like this, but including more realistic heterogeneity amongst the population and including government policies and government regulations in a more sophisticated way. Somebody should make that, Andrey.Andrey: Yeah. I wonder if someone's trying to make it.Seth: You know, if any of anybody listening to this has funding, please let me know. The research agenda is currently unfunded and we could use your support.Andrey: Alright. So do you wanna wrap up here?Seth: I think this is a natural place to leave it, which is, I like where this is going kind of as an intellectual contribution, but it's not quite a practical tool yet. That's kind of where I leave it.Andrey: Alright. Well, thanks for joining us for another episode of Justified Posteriors. Please like, comment, and subscribe to our podcast. And do let us know if you have any feedback. Feel free to tell us.Seth: Yeah, but only good feedback on the website, the negative feedback in person. Good feedback on the website.Andrey: Alright.Seth: See you all later. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit empiricrafting.substack.com

We hear it constantly: social media algorithms are driving polarization, feeding us echo chambers, and maybe even swinging elections. But what does the evidence actually say? In the darkest version of this narrative, social media platform owners are shadow king-makers and puppet masters who can select the winner of close election by selectively promoting narratives. Amorally, they disregard the heightened political polarization and mental anxiety which are the consequence of their manipulations of the public psyche. In this episode, we dive into an important study published in Science (How do social media feed algorithms affect attitudes and behavior in an election campaign?https://www.science.org/doi/10.1126/science.abp9364) that tackled this question. Researchers worked with Meta to experimentally change the feeds of tens of thousands of Facebook and Instagram users in the crucial months surrounding the 2020 election.One of the biggest belief swings in the history of Justified Posteriors in this one!The Core Question: What happens when you swap out the default, engagement-optimized algorithmic feed for a simple, reverse-chronological one showing posts purely based on recency?Following our usual format, we lay out our priors before dissecting the study's findings:* Time Spent: The algorithmic feed kept users scrolling longer.* Content Consumed: The types of content changed in interesting ways. The chronological feed users saw more posts from groups and pages, more political content overall, and paradoxically, more content from untrustworthy news sources.* Attitudes & Polarization: The study found almost no effect on key measures like affective polarization (how much you dislike the other side), issue polarization, political knowledge, or even self-reported voting turnout.So, is the panic over algorithmic manipulation overblown?While the direct impact of this specific algorithmic ranking vs. chronological feed seems minimal on core political beliefs in this timeframe, other issues are at play:* Moderation vs. Ranking: Does this study capture the effects of outright content removal or down-ranking (think the Hunter Biden laptop controversy)?* Long-term Effects & Spillovers: Could small effects accumulate over years, or did the experiment miss broader societal shifts?* Platform Power: Even if this comparison yields null results, does it mean platforms couldn't exert influence if they deliberately tweaked algorithms differently (e.g., boosting a specific figure like Elon Musk on X)?(Transcript below)🗞️ Subscribe for upcoming episodes, post-podcast notes, and Andrey’s posts:💻Follow us on Twitter:@AndreyFradkin https://x.com/andreyfradkin?lang=en@SBenzell https://x.com/sbenzell?lang=enTranscript:Andrey: We might have naively expected that the algorithmic feed serves people their "red meat"—very far-out, ideologically matched content—and throws away everything else. But that is not what is happening.Seth: Welcome everyone to the Justified Posterior Podcast, where we read and are persuaded by research on economics and technology so you don't have to. I'm Seth Benzell, a man completely impervious to peer influence, coming to you from Chapman University in sunny Southern California.Andrey: And this is Andrey Fradkin, effectively polarized towards rigorous evidence and against including tables in the back of the article rather than in the middle of the text.Seth: Amazing. And who's our sponsor for this season?Andrey: Our sponsor for the season is the Digital Business Institute at the Questrom School of Business at Boston University. Thanks to the DBI, we're able to provide you with this podcast.Seth: Great folks. My understanding is that they're sponsoring us because they want to see information like ours out there on various digital platforms, such as social media, right? Presumably, Questrom likes the idea of information about them circulating positively. Isn't that right?Andrey: Oh, that's right. They want you to know about them, and by virtue of listening to us, you do. But I think, in addition, they want us to represent the ideal of what university professors should be doing: evaluating evidence and contributing to important societal discussions.Andrey: So with that set, what are we going to be talking about today?Seth: Well, we're talking about the concept of participating in important societal discussions itself. Specifically, we're discussing research conducted and published in Science, a prestigious journal. The research was conducted on the Facebook and Instagram platforms, trying to understand how those platforms are changing the way American politics works.The name of the paper is, "How Do Social Media Feed Algorithms Affect Attitudes and Behavior in an Election Campaign?" by Guess et al. There are many co-authors who I'm sure did a lot of work on this paper; like many Science papers, it's a big team effort. See the show notes for the full credit – we know you guys put the hours in.This research tries to get at the question, specifically in the 2020 election, of to what extent decisions made by Mark Zuckerberg and others about how Facebook works shaped America's politics. It's an incredibly exciting question.Andrey: Yeah, this is truly a unique study, and we'll get into why in just a bit. But first, as you know, we need to state our prior beliefs about what the study will find. We're going to pose two claims: one narrow and one broader. Let's start with the narrow claim.Seth: Don't state a claim, we hypothesize, Andrey.Andrey: Pardon my imprecision. A hypothesis, or question, if you will: How did the algorithmic feed on Facebook and Instagram affect political attitudes and behavior around the time of the 2020 presidential election? Seth, what is your prior?Seth: Alright, I'm putting myself in a time machine back to 2020. It was a crazy time. The election was at the end of 2020, and the pandemic really spread in America starting in early 2020. I remember people being hyper-focused on social media because everyone was locked in their houses. It felt like a time of unusually high social media-generated peer pressure, with people pushing in both directions for the 2020 election. Obviously, Donald Trump is a figure who gets a lot of digital attention – I feel like that's uncontroversial.On top of that, you had peak "woke" culture at that time and the Black Lives Matters protests. There was a lot of crazy stuff happening. I remember it as a time of strong populist forces and a time where my experience of reality was really influenced by social media. It was also a time when figures like Mark Zuckerberg were trying to manage public health information, sometimes heavy-handedly silencing real dissent while trying to act for public welfare.So, that's a long wind-up to say: I'm very open to the claim that Facebook and Instagram had a thumb on the scale during the 2020 election season, broadly in favor of chaos or political polarization – BLM on one side and MAGA nationalism on the other. At the same time, maybe vaguely lefty technocratic, like the "shut up and listen to Fauci" era. Man, I actually have a pretty high prior on the hypothesis that Facebook's algorithms put a real thumb on the scale. Maybe I'll put that around two-thirds. How about you, Andrey?Andrey: In which direction, Seth?Seth: Towards leftiness and towards political chaos.Andrey: And what variable represents that in our data?Seth: Very remarkably, the paper we studied does not test lefty versus righty; they do test polarization. I don't want to spoil what they find for polarization, but my prediction was that the algorithmic feed would lead to higher polarization. That was my intuition.Andrey: I see. Okay. My prior on this was very tiny effects.Seth: Tiny effects? Andrey, think back to 2020. Wasn't anything about my introduction compelling? Don't you remember what it was like?Andrey: Well, Seth, if you recall, we're not evaluating the overall role of social media. We're evaluating the role of a specific algorithm versus not having an algorithmic feed and having something else – the reverse chronological feed, which shows items in order with the newest first. That's the narrow claim we're putting a prior on, rather than the much broader question of what social media in general did.Seth: Yeah, but I guess that connects to my censorship comments. To the extent that there is a Zuckerberg thumb on the scale, it's coming through these algorithmic weightings, or at least it can come through that.Andrey: I think we can come back to that. My understanding of a lot of platform algorithm stuff, especially on Facebook, is that people mostly get content based on who they follow – people, groups, news outlets. The algorithm shifts those items around, but in the end, it might not be that different from a chronological feed. Experts in this field were somewhat aware of this already. That's not to say the algorithmic feed had no effects, but I expected the effects to be very small.Another aspect is how our political beliefs are formed. Yes, we spend time online, but we also talk to friends, read the news, get chain emails from our crazy uncle (not my crazy uncle, but people do).Seth: One thing we'll get to see is what people substitute into when we take away their Facebook algorithmic feed.Andrey: Yes. Furthermore, political beliefs generally don't change very frequently. I don't have a specific study handy, but it's fairly understood. There are exceptions, like preference cascades, but generally, if you believe markets work well, you won't suddenly change your mind, and vice versa. This holds for many issues. Imagine polling people on who they voted for in 2016 versus 2020 – the correlation for voting for Donald Trump would be immensely high. It's really hard to move people's political preferences.Seth: I think that's right. There are things people's beliefs move around more on shorter timelines, though. One thing they look at is political knowledge, which also seems unaffected, interestingly. The only thing I'd push back on regarding fixed beliefs is the idea of preference cascades. Settings where beliefs like "we are now in chaos, everyone for themselves" can spread very fast if seeded correctly.Okay, so that was our narrow claim. Let me put a bow on that, Andrey. With what percentage probability would you say that the effect of social media algorithms on political outcomes or polarization is very small?Andrey: 80 percent confident.Seth: Alright. Well now, Andrey, let's talk about the broader hypothesis. Go ahead.Andrey: So, this is something we're realizing as we do more episodes: there's often a very narrow, precise claim a paper addresses, and then there's the more relevant claim of interest to society.Seth: And this is what we're going to put on the TikTok ads.Andrey: Yes. The narrow claim is about comparing people looking at either algorithmic or reverse chronological feeds on Facebook over a specific three-month period. The broader question is whether, for society as a whole, the fact that feeds are algorithmic has very different effects.Why might the effect for society differ from the effect on individuals in an experiment? One key assumption in causal inference – great time to bring this up as I'm teaching my first class tomorrow...Seth: (To himself) I hope he brings this up right now.Andrey: ...is the non-interference assumption, or the Stable Unit Treatment Value Assumption (SUTVA). This essentially says that people who receive a treatment don't affect people in the control group, and vice versa. There are no spillovers. But if there's anything we know about social media, it's that it's all about spillovers. If I live with roommates and get slightly different news because of the algorithm, I can still tell them about it.A broader spillover is the incentive algorithms create for content generation. If the algorithm promotes things with high engagement, and people make money from engagement (like news media, influencers), they'll start creating outrageous stuff to get boosted. Since the incentive is high, a lot of content on the platform might become like this.Seth: ...unless there's a thumb on the scale from Zuckerberg to shape that.Andrey: I think the default of any algorithmic feed is to optimize for engagement. Tweaks might happen, but as a first-order approximation, they show engaging stuff – funny claims, videos, outrageous things that keep people using social media more.Seth: That's the hypothesis. But as we get into the evidence, we'll see how people's content actually switched, at least in this sample. Where are we going with this broader claim? The broader question is: do social media algorithms have political effects more broadly, and are these effects large enough to swing elections or drive polarization?I come to that question thinking about a famous book, The Revolt of the Public, which argues that digital platforms inherently favor populist politics. When something gets digitized, you often see power-law distributions: superstars at one end, a long tail of niche interests. I think that's basically right as an effect of social media, whether algorithmic or reverse chronological. Remember, even reverse chronological has preferential attachment built-in – people follow others who are already popular.So, asking if the social media world is politically different from the non-social media world – I think that's obvious. Even within social media, platform owners must have significant power over what information rises. On platforms people spend hours on daily... in principle, could an algorithm swing elections or drive polarization? 90 percent plus. Has it happened already in American history? 80 percent plus.Andrey: Yeah, I'm with you that the holistic picture of algorithms' role suggests they must have had effects on politics. But this is where detailed platform knowledge matters: there's no one algorithm. There are layers of algorithms and moderation.A famous example on the right is the Hunter Biden laptop story. There was a perception it came from a hack or was potentially made up. As a result, some platforms manually put a thumb on the scale to limit its spread. Is this an algorithm? It depends. One version is just removing posts with links to the story – hard censorship.Seth: Censorship, if you will.Andrey: Right. There's also potentially a scoring system that flags content as possibly fraudulent, illegitimate, or low-quality, giving it a lower algorithmic score without a full ban.Seth: Shadow banned.Andrey: Exactly. That's clearly mediated by the platform. But there's a world where this content is removed and wouldn't show up in the reverse chronological feed either, depending on moderation specifics.Why am I saying this? The algorithm predicting what you'll click on is a bit different from the content moderation system. Famously, Facebook had many people trying to moderate content. These are extremely serious issues. There are credible accusations of Facebook not censoring content inciting genocide in Myanmar (the Rohingya genocide). The stakes are high. It's not just about machine learning algorithms; people are scoring content and deciding what's good or bad.Seth: Right. So there are values built into the process, is what you're maybe conceding.Andrey: Yes. Alright, with that broad prior discussion...Seth: Give me a percentage on the broad hypothesis.Andrey: I guess I was trying to say it's hard to make it precise. Let's just say all the things Facebook does affecting what you see in the feed – the cumulative aspects – certainly have political effects. But it's not just Facebook alone; many types of social media contribute. Even if we made one platform very unpolitical (and we'll see something about this with Instagram in the experiment), it wouldn't remove the potential role of social media overall.Seth: Okay, good. Alright, let's get to the evidence.These researchers worked with Facebook to conduct a pre-registered study. That's impressive – they wrote down all analyses, recruitment, and filtering beforehand. In their main comparisons, they had about 20,000 Facebook users and 20,000 Instagram users. About half were assigned to a reverse chronological feed for three months around the 2020 U.S. election, seeing only the most recent posts from accounts they follow. The control group got the default algorithmic feed, curated by Facebook to be engaging.Andrey: And also to remove violating content.Seth: Yes, and reduce slurs. They looked at three types of effects. First: platform usage. Unsurprisingly, the algorithmic feed makes people use Facebook substantially more – perhaps 30% more? My recollection differs slightly, but it's significant.Andrey: I think the paper states the average respondent in the algorithmic feed group spent 73% more time each day compared to the average US monthly active user. In the chronological feed, this reduced to 37% more. So maybe closer to a 50% reduction relative to the algorithmic group's excess time, but still significant usage even without the algorithm.Seth: Yes, yes.Andrey: The effects are interestingly a bit smaller for Instagram.Seth: It's a really strange way to show the result in the paper. This is a comment about my dislike of Science magazine editors and how they sometimes don't give us the parts needed to evaluate things easily.Andrey: They don't... well, two things they don't report straightforwardly are the overall level of time use (bizarrely obfuscated) and whether the feed made you vote for Trump or not, which is the question I want to know.Seth: Well, in their defense, they asked people whether they voted, and there was no effect on turnout.Andrey: They asked people whether they voted for Trump or Biden; they just don't tell us the answer in the main text. They definitely asked.Seth: Yeah, I don't know. There are a lot of things to say about this paper. For listeners, there are 300 pages of appendix! Some are survey instruments, but the amount of results is staggering. My understanding is they were obligated to report everything they pre-specified.Andrey: Even without correcting for multiple hypothesis testing? Well, when all effects are zero, does it really matter?Seth: You could still get wider confidence intervals.Andrey: What I wanted to say is this is a very unusual study. Facebook agreed to this and let researchers have high autonomy. My understanding is Facebook also funded it, which is non-negligible given participant payments (potentially over $100 each for 20,000+ participants). They also had full-time Facebook research scientists providing data and coding support. It was a huge endeavor, so many things were measured.Uniquely, there was an on-platform experiment (different algorithms) and surveys. Some users even consented to install software tracking their off-Facebook activity. It's very comprehensive.So far, we've mentioned people spend more time with algorithmic feeds. Unsurprisingly, they're also more likely to like and comment on posts they see – consistent with optimization goals.But some findings about what people see are maybe surprising. With the algorithmic feed, about 60% of content is from friends. With the chronological feed, that falls to 33%. Chronological feed users see much more content from Groups they're in (a popular product, even if I haven't joined one since college) and Pages (brands, news outlets, etc.).Seth: 90% are just Minion memes. If you're making the mistake of projecting your feed onto the rest of the world... you met Americans? 90% of their Facebook is Minions feeds.Andrey: Alright. Which result next? There's a lot here.Seth: How about the political content of the posts?Andrey: Yes, let's get to the political stuff. Highlighting post sources helps understand how different the content is. In the chronological feed, people actually see a higher proportion of political content (about 15% more) and more content from moderate or mixed sources.At the same time, a really big effect: they see 70% more posts from untrustworthy news sources in the chronological feed. This relates to moderation. Facebook has scores suggesting certain outlets are "fake news."Seth: Clickbait factories, right? Tabloids, basically.Andrey: Yeah. This portrays a nuanced story. We might naively expect the algorithmic feed serves ideological "red meat" and discards everything else. That's not happening. If anything, the chronological feed sends people more potentially outrageous stuff from untrustworthy sources.Seth: Or maybe the algorithmic feed finds content at the intersection of engaging and anodyne? It wants to bring you engaged content, maybe political if mainstream, but mostly non-political news.Andrey: Just to clarify, the chronological feed shows more political news (40% more).Seth: Yes, to be clear, chronological is 40% more political. My point is the algorithm seems to point you towards less political content. To the extent it is political, it's more trustworthy. The chronological feed also has fewer slurs, though.Andrey: Yeah, but slurs occur so infrequently, I don't know how important that is. This difference in content is what we call the "first stage" in statistical analysis. Any change in the algorithm matters because you see different content.Seth: Now, let's see how Dr. Evil Zuckerberg manipulated American minds. How big are those effects, Andrey?Andrey: They are essentially zero. The effects are tiny and fairly precisely estimated. Let's list the primary outcomes:* Affective polarization (how you view the other party/politicians)* Issue polarization* Election knowledge* News knowledge* Self-reported political participation* Self-reported turnout (did you vote?)No effect on these. The one difference: people with the chronological feed were less likely to post political comments and posts themselves on Facebook. Maybe not surprising, since they see less from friends, and most people might only engage politically when talking to friends via their feed, not random groups or pages.Seth: The only political activity we see more of in the chronological feed is clicks on partisan news. It seems people in the chronological feed are exposed to more of these less quality-adjusted sources and click on them more often.Andrey: Let me push back. Putting on my educator hat: this is one of the secondary outcomes. If you run enough hypothesis tests, something will be significant by chance. There are tons of secondary outcomes, and only one is statistically significant. I wouldn't pay much attention to it. If I were reviewing a paper based solely on finding one significant secondary outcome after null primary findings, I'd say, "Dude, what are you doing? You told us you cared about X, found no effect, then dug around until you found Y and built your story on that? That seems wrong."Seth: Fair enough.Andrey: Not saying the authors did that, just my general view.Seth: I just picked out the one number that wasn't zero. But speaking of these zeros, they're reported in standard deviations. The confidence intervals for most outcomes are within +/- 0.05 standard deviations of zero. Is that small, or could 0.05 standard deviations swing an election if scaled across America?Andrey: Great point, and a limitation. From this study, we know the effects aren't huge. But U.S. presidential elections are often enormously close. If we multiply even a tiny effect out, it could matter. We can't say for sure from this study, but the evidence is consistent with effect sizes that could swing a close election.Seth: Right. We can't rule out small but potentially significant effects. I'm still frustrated they don't just give us the party-line voting outcome. I understand why Facebook might not want that, but why not? Did this make people vote more for Trump or Biden?Andrey: They do report "party-line presidential voting" as an outcome in the appendix, I believe.Seth: I want to see: did they vote more or less for Trump as a function of being assigned to the chronological feed?Andrey: I haven't dug that deeply into the appendix. Maybe you're confident they didn't report it prominently. I'm confident they have the number. My strong belief is the effect is zero. I'd be shocked if there was an effect.Seth: I can see why Facebook wouldn't want them to highlight it. If there's a non-zero result there, there's no winning that conversation.Andrey: But "party-line presidential voting" seems so close to what you want. I'm wary of conspiracy thinking about why it wasn't emphasized. Maybe you're right, but I'm not sure.I should also mention, earlier I should have disclosed I've had a research collaboration with Facebook in the past.Seth: Boo, hiss, boo.Andrey: I got paid a trivial amount, forced to be a contractor for the project. This doesn't mean I'm using inside information; I have none about this study from my prior work.Seth: But what you're saying, in a sense, is the audience should pay more attention to me for this episode.Andrey: Just to be clear, I generally think social media is not that great, so you should update based on that too.Seth: Oh my gosh. Pivoting to the center here, Andrey? Despicable. We need to be extreme!Andrey, you labeled my speculation about the voting outcome reporting a "conspiracy theory." Well, I want you to know that one of the secondary hypotheses in this article was about whether Facebook makes you into a conspiracy theorist.Andrey: Oh, yes.Seth: I'd like to ask you a series of questions Facebook used to evaluate this. Do you accept this challenge?Andrey: I accept.Seth: Alright, I need your belief (0-100%) on these statements circulating in 2020. Advanced difficulty: some are in Spanish.* Evidence found on Hunter Biden's laptop proves Joe Biden took bribes from foreign powers.Andrey: It doesn't prove things. No. I take objection to the wording. It's poorly worded.Seth: Okay. Question two: 2. The current FBI director, Wray, has said that the greatest domestic terrorist threat is white supremacists.Andrey: That is what he said.Seth: Correct. Not a conspiracy theory. 3. Amy Coney Barrett said that a woman needs a man's permission to own property.Andrey: Probably not. 5 percent?Seth: 5%? You are correct, 0% was the answer. 4. The US government has a plan to force a COVID-19 vaccine on everyone.Andrey: "Force" is doing a lot of lifting here. I'm guessing the narrow claim of forcing is zero.Seth: That would be a 0 percent claim. You see how this determines conspiracy theoriness. 5. Masks and face coverings are not effective in preventing the spread of COVID-19.Andrey: Right? They're all... (mumbles) The entire world got COVID-19. I don't know what this question wants. It's not like we prevented the spread entirely.Seth: Alright, next one: 6. Millions of fraudulent ballots were cast in the 2020 presidential election.Andrey: Hopefully not millions. That's a 0.00001 percent.Seth: 7. Donald Trump held a Bible upside down in front of a church.Andrey: Sure.Seth: 8. In October 2020, most rural counties were in the COVID-19 red zone based on their high rates of new cases.Andrey: No idea.Seth: That was correct. Okay. 9. (Spanish) Antes de las elecciones presidenciales de 2016, Donald Trump pagó en secreto a una estrella de cine para adultos. (Before the 2016 presidential election, Donald Trump secretly paid an adult film star.)Andrey: I don't speak Spanish, Seth.Seth: You can't get that? Una Estrella... 10. (Spanish) Joe Biden es un pedófilo. (Joe Biden is a pedophile.)Andrey: Wait, seriously? That's what they asked?Seth: Facebook scientists asked the public, "Is Joe Biden a pedophile?" In both Spanish and English.Andrey: Alright.Seth: Andrey, thanks for playing "Are You a Conspiracy Theorist?" My takeaway: many questions aren't black and white. Believing the "wrong" answer doesn't necessarily mean someone is a schizophrenic-style conspiracy theorist. What do you think?Andrey: Yeah, it depends if you take them literally or as gestures towards something. Not the best conspiracy test. But I guess the effect [of the feed type on conspiracy beliefs] was zero? I didn't look at this specific outcome closely.Seth: I think we found you were at least 25% conspiracy theorist, Andrey. Proud or terrified?Andrey: I'm a free thinker, Seth.Seth: Alright, Andrey, should we move on to limitations?Andrey: The only other thing I'll mention is this is part of a bigger set of studies. My understanding is there are at least four, maybe eight papers in progress from this collaboration, studying various aspects like deactivation experiments (paying users to not use Facebook).Seth: Right.Andrey: That could speak to the broader question of what social media is doing. But it suffers from similar criticisms: social media isn't an individual decision in a vacuum. Even if we don't use it, we're affected by it.Seth: Alright, limitations. We already talked about affiliations – doing this with Facebook might mean avoiding highly charged questions. How much does that bother you? Do you think this would have been pocket-vetoed if there were big negative effects found?Andrey: My understanding is this study was unique. There was a pre-commitment from Facebook to publish results. Interfering would have been a huge, publicized deviation. An independent observer wrote a report confirming no interference. So, while we shouldn't dismiss concerns entirely, I'd be more worried about other collaborations, like unpublished advertising studies where results might be canned internally if they showed ads didn't work. This study had strong commitments against interference, and I think we should trust it more.Seth: Here's another question: The "first stage" involved both reducing usage time and changing content mix. Are you worried about a net zero effect masking big, canceling effects in opposite directions? Maybe usage levels had one effect, content mix another, and they coincidentally canceled out?Andrey: It's plausible. The authors do some heterogeneity analysis, which might pick that up if it were happening, but it doesn't seem like much is going on there. It's an interesting interpretation question. If we had found an effect, we'd discuss mechanisms. When there's a zero effect, finding canceling mechanisms is tricky.Seth: Any limitations I missed?Andrey: A big one: duration. Three months is long by academic standards (we see one-week studies!), but if we're interested in truly broad effects over years, it's short. If a tiny effect materializes linearly over, say, four years between elections, you could multiply the potential effect from this study by 16. Small effects can get big over time.Seth: Okay, ready to move into the posterior, Andrey?Andrey: Sure.Seth: Alright, my posterior. I started at two-thirds chance the algorithm put a significant thumb on the scale favoring lefty candidates and chaos/polarization (MAGA vs. BLM). The other third was "no net effect." I've moved considerably towards "no net effect," at least regarding political polarization. This paper is convincing the algorithmic feed didn't make people more polarized leading up to 2020. On that specific claim, I go from 67% true to maybe 5% true.We don't get the Biden/Trump vote answer, so I can't update hard on the "lefty candidate" part, but I'd still update towards zero, maybe from 67% to 30%, because my mechanism involved effects on both polarization and candidate choice simultaneously. How about you on the narrow question?Andrey: Yeah, it definitely made me update. I'd seen versions of this paper over the past year. But fundamentally, it doesn't answer a critical question: moderation. Take the Hunter Biden laptop. If Facebook moderated posts by simply not showing them, that would likely affect both the algorithmic and reverse chronological feeds equally. We learn nothing about that type of moderation from this comparison. And that's what much political discussion focuses on – these fiery stories that could shift opinions being potentially suppressed across the board. I don't see anything here telling me those bans don't apply to the reverse chronological feed.Seth: Right. Important editorial choices might exist outside this experimental comparison.Andrey: Yes.Seth: How about the broader claim? I come down a bit, from ~90% "this could be super important" to maybe still 90% on the potential, but down from ~80% to maybe 50-60% on the idea that these choices have historically had major political effects. 2020 seemed like a prime election to see big effects jump out, and we didn't see strong evidence here for this specific mechanism.Andrey: I agree my belief goes down. Here's what I'd say: the role of the specific machine learning part of the algorithm seems less important than I might have thought. A big driver of what people see is simply who they follow. Now, who they follow might be influenced by other algorithmic systems (friend recommendations, nudges) not tested here. Maybe those have big effects. But conditional on following someone, the content seems somewhat similar whether ranked by algorithm or chronology.Seth: Well, maybe that's a good place to leave it, Andrey, unless you have parting thoughts.Andrey: I do have one. This discussion is interesting, especially now with the moderation changes on X (formerly Twitter). It's part of the narrative that Elon Musk did something to cause a "vibe shift," possibly increasing support for Trump and decreasing support for progressive causes. What specifically did he do? I'll leave listeners with this: Suppose you put a score in your algorithm to put whatever Elon Musk says at the top of everyone's feed. Could that possibly have different effects than the experiment studied here?Seth: Right. The question is still unanswered. I know many listeners are young researchers, and we invite you to attack that question. This paper feels like a starting gun for investigating algorithms in politics, rather than the final answer.Andrey: Yes. Well, thanks for listening. Please make sure to comment, like, subscribe, and generally spread the good word about Justified Posterior.Seth: And tune in in two weeks where we'll talk through one more paper on economics and technology and get persuaded by it so you don't have to. Alright? This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit empiricrafting.substack.com

In this episode of Justified Posteriors, we dive into the paper "Which Economic Tasks Are Performed with AI: Evidence from Millions of Claude Conversations." We analyze Anthropic's effort to categorize how people use their Claude AI assistant across different economic tasks and occupations, examining both the methodology and implications with a critical eye.We came into this discussion expecting coding and writing to dominate AI usage patterns—and while the data largely confirms this, our conversation highlights several surprising insights. Why are computer and mathematical tasks so heavily overrepresented, while office administrative work lag behind? What explains the notably low usage for managerial tasks, despite AI's apparent suitability for scheduling and time management?We raise questions about the paper's framing: Is a gamer asking for help with their crashing video game really engaging in "economic activity"? How much can we learn from analyzing four million conversations when only 150 were human-verified? And what happens when different models specialize—are people going to Claude for coding but elsewhere for art generation?We also asked Claude itself to review this paper about Claude usage, revealing some surprisingly pointed critiques from the AI about the paper's fundamental assumptions.Throughout the episode, we balance our appreciation for this valuable descriptive work with thoughtful critiques, ultimately suggesting directions for future research that could better connect what people currently use AI for with its potential economic impact. Whether you're interested in AI adoption, labor economics, or just curious about how people are actually using large language models today, we offer our perspectives as economists studying AI's integration into our economy.Join us as we update our beliefs about what the Anthropic Economic Index actually tells us—and what it doesn't—about the future of AI in economic tasks. The full transcript is available at the end of this post.The episode is sponsored by the Digital Business Institute at Boston University’s Questrom School of Business. Big thanks to Chih-Ting (Karina) Yang for her help editing the episode.-🔗 Links to the paper for this episode’s discussion:Which Economic Tasks are Performed with AI? Evidence from Millions of Claude ConversationsGPTs are GPTs: Labor market impact potential of LLMs🗞️ Subscribe for upcoming episodes, post-podcast notes, and Andrey’s posts:💻 Follow us on Twitter:@AndreyFradkin https://x.com/andreyfradkin?lang=en@SBenzell https://x.com/sbenzell?lang=enTranscriptSeth: Welcome to the Justified Posteriors Podcast. The podcast that updates beliefs about the economics of AI and technology. I'm Seth Benzel with nearly half of my total output constituting software development and writing tasks coming to you from Chapman University in sunny Southern California.Andrey: And I'm Andrey Fradkin, enjoying playing around with Claude 3.7 coming to you from Cambridge, Massachusetts.Seth: So Andrey, what's the last thing you used AI for?Andrey: The last thing I use AI for, well, it's a great question, Seth, because I was so excited about the new Anthropic model that I decided to test run it by asking it to write a referee report about the paper we are discussing today.Seth: Incredible. It's a little bit meta, I would say, given the topic of the paper. Maybe we can hold in our back pockets the results of that experiment for later. What do you think?Andrey: Yeah, I think we don't want to spoil the mystery about how Claude reviewed the work of its creators.Seth: Claude reviewing the work of its creators - can Frankenstein's monster judge Frankenstein? Truly. So Andrey, maybe we've danced around this a little bit, but why don't you tell me what's the name of today's paper?Andrey: The name of the paper is a bit of a mouthful: "Which Economic Tasks Are Performed with AI: Evidence from Millions of Claude Conversations." But on a more easy-to-explain level, the paper is introducing the Anthropic Economic Index, which is a measure of how people use the Claude chatbot, demonstrating how it can be useful in a variety of interesting ways for thinking about what people are using AI for.Seth: Right. So at a high level, this paper is trying to document what people are using Claude for. I was also perplexed about the fact that they refer to this paper as an AI index given that an index usually means a number, and it's unclear what is the one number they want you to take away from this analysis. But that doesn't mean they don't give you a lot of interesting numbers over the course of their analysis of how people are using Claude.Andrey: So before we get into the paper a bit more, let's talk about the narrow and broad claims and what our priors are. The narrow claim is maybe what specifically are people using Claude for. Do we think this is a representative description of the actual truth? The authors divide up the analysis in many different ways, but one way to think about it is: is it true that the primary uses of this chatbot are computer and mathematical tasks? And is it also true that relatively few people use the chatbot for office and administrative support as well as managerial decision making?Seth: Those are excellent questions. The first question is what are people using Claude for right now? And do we buy that the way they're analyzing the usage data gives us an answer to that question? Before I answer whether I think Claude's approach in analyzing their own chats is appropriate, let me tell you what my sense was coming in. If you had asked "What are people using chatbots for right now?" I would have guessed: number one, they're using it for doing their homework instead of actually learning the material, and number two, actual computer programmers are using it to speed up their coding. It can be a great coding assistant for speeding up little details.Although homework wasn't a category analyzed by Claude, they do say that nearly half of the tasks they see people using these AI bots for are either some form of coding and software development or some form of writing. And of course, writing could be associated with tasks in lots of different industries, which they try to divide up. If you told me that half of what people use chatbots for is writing help and coding help - if anything, I would have thought that's on the low side. To me, that sounds like 80 percent of use cases.Andrey: I think I'd say I'm with you. I think we probably agree on our priors. I'd say that most of the tasks I would expect to be done with the chatbot might be writing and programming related. There's a caveat here, though - there's a set of behaviors using chatbots for entertainment's sake. I don't know how frequent that is, and I don't know if I would put it into writing or something else, but I do know there is a portion of the user base that just really likes talking to Claude, and I don't know where that would be represented in this dataset.Seth: Maybe we'll revisit this question when we get to limitations, but I think one of the limitations of this work is they're trying to fit every possible usage of AI into this government list of tasks that are done in the economy. But I've been using AI for things that aren't my job all the time. When America came up with this O*NET database of tasks people do for their jobs, I don't think they ever pretended for this to be a list of every task done by everyone in America. It was supposed to be a subset of tasks that seem to be economically useful or important parts of jobs that are themselves common occupations. So there are some limitations to this taxonomical approach right from the start.Coming back to your point about people playing around with chatbots instead of using them for work - I have a cousin who loves to get chatbots to write slightly naughty stories, and then he giggles. He finds this so amusing! Presumably that's going to show up in their data as some kind of creative writing task.Andrey: Yeah.Seth: So moving from the question of what we think people are using chatbots for - where I think we share this intuition that it's going to be overwhelmingly coding and writing - now we go to this next question you have, which is: to what extent can we just look at conversations people have with chatbots and translate the number of those conversations or what sort of things they talk about into a measure of how people are going to usefully be integrating AI into the economy? There seems to be a little bit of a step there.Andrey: I don't think the authors actually make the claim that this is a map of where the impact is going to be. I think they mostly just allude to the fact that this is a really useful system for real-time tracking of what the models are being used for. I don't think the authors would likely claim that this is a sign of what's to come necessarily. But it's still an interesting question.Seth: I hear that, but right on the face, they call it the Anthropic Economic Index. If they wanted to call it the "Anthropic What Are People Using Anthropic For Right Now Snapshot" or the "Anthropic Usage Index," I'm a lot more sympathetic. I think they have to do a lot less work defending that idea than the "Anthropic Economic Index."Andrey: Well, this is maybe where the academic and corporate lingo collide. But I hear you in the sense that it's not clear that what is being done in these chats is necessarily economic activity versus personal activity, learning activity, and so on. A more humble naming of the index could have appeased some of the criticisms.Seth: You've gotta be on the defensive when you come on the Justified Posteriors podcast, because we challenge you to justify your posterior, so you better be ready to defend yourself. So, for the narrow question, I gave you my prior - it's gonna be overwhelmingly used for coding and people doing homework assignments. And homework assignments will look like mostly creative writing and regular writing and history writing and all the different things people do homework assignments for. So we'll see what the data actually says.For the broad question, I would say this is a great view of what people are using Claude for right now, but to try to translate that into economic value, or what people are going to use Claude for in the future, we need giant grains of salt here. I think it's better than random guessing, but there's a huge gap between the things people will use AI to play around with as a tool, or for fun, or to explore, versus where are people getting consistent economic value from it.Andrey: I would say the same. I view this as a proof of concept, something that has very natural extensions that can make it much more useful. To be clear, I think it was probably a large effort just getting everything in shape for this sort of analysis, and I doubt that this is the end-all be-all of the work the team is doing there. But I agree that we need a lot more work to convince us that this is giving us a general shape of what LLMs are going to be used for.In particular, one limitation is that a lot of work moves to the API. So a lot of the activity that is done for work is not actually captured by this index because business users use the API. There's also a business plan where the usage from the business plan is not included in the index. I can imagine why these were not included, but it does limit our ability to understand economic impact.Seth: Right. Having laid out our priors, Andrey, do you feel like you've laid yours out in sufficient detail to confront the new evidence that Claude is putting before us?Andrey: Yes. So let's get to what the paper does. At a very high level, what they do is come up with a method for categorizing conversations as being mapped to tasks. Then they map those tasks to a database that's been used all over economic research of how tasks correspond to jobs. By doing that crosswalk, they're able to say something about what jobs have many tasks that are already being done by the chatbot versus what jobs do not. And then in addition to that, they think about when people are having these conversations, are they automating a task or are they more like collaborating with the AI to do a task? So that's the high-level thing that they do in this paper, and then it's kind of a measurement exercise.Seth: They actually give some really useful examples of conversations that are matched to tasks and then occupations. For example, they consider the user conversation where the user posts, "My game keeps crashing as I only have eight gigabytes of RAM." That is then classified by their automatic categorization as the O*NET task "modify software to improve performance and adapt to new hardware," which is then mapped to a specific computer and mathematical occupation.Similarly, they give the example of "Can you make sure this blog post follows Chicago style?" That's associated with the task "standardize materials from other writers and staff," which is considered associated with an arts and media job.The first thing I want to point out is that all of these conversations sound like hobby activities rather than actually creating economic output. So on its face, it's not clear that they're actually saying things about people doing their jobs. Secondly, the guy whose video game keeps crashing because he only has 8 gigabytes of RAM is clearly not a computer programmer. He's clearly a guy who's just playing a video game. It seems like a misclassification. I just want to say that the examples they give of this classification task do not inspire confidence that they are measuring people's work activities.Andrey: They do have some better examples when they're thinking about automated behaviors and augmented behaviors, like "format this technical document in Markdown" or "here's my Python script for data analysis, it's giving an index error, can you help fix it?" That seems like more work-related stuff - although the Python error thing could easily be one of my students asking for help with a homework assignment. But those are plausibly more work-related.Seth: What I make of this is that the title of this paper should just be "Which Tasks Are Performed with AI," not "Which Economic Tasks." It's not clear what makes a task economic. In my opinion, a task is economic if it's either some sort of Robinson Crusoe economy where even if I'm not interacting with anyone, this is an economic behavior because I'm building a thing that I'm going to use, or what makes something economic is that I'm participating in a market with this thing and I'm going to buy it and sell it after I go through these steps."My video game is crashing cause I only have eight gigabytes of RAM" doesn't sound like either of those. It sounds like this guy is troubleshooting his consumption, which maybe could be thought of as the consumer taking on some of the job of customer service. The other example, "Can you make sure this blog post follows Chicago style?" - if I'm making an artistic or creative project that I'm just putting out on the internet for people, again, I'm not sure I would call that economic activity. So no problems with this paper being about measuring what activities or tasks people do with AI, but I think it's probably a breach too far to call these economic tasks.Andrey: I think I agree with you. There needs to be more metadata around these conversations. A survey of whether users are using this for their job or not could be really informative, or even just a subset analysis of just the pro users who are more likely to be using this for their job.I do think it's an interesting phenomenon of substituting professional labor with personal labor. Hal Varian used to bring up this example all the time with YouTube - before, you'd hire someone to repair your appliance or do work around the house, but now you can watch a YouTube video and do it yourself. This means YouTube is generating tremendous economic value that's not being measured. I think both of us are generally on board with that idea - GDP is going to miss a bunch of interesting activity just by virtue of how it's measured. But especially for an academic contribution, we want a more rigorous analysis.Seth: Or just be clear what your domain of analysis is. If you're going to take the stance that anything anybody does is economic, then just call it "tasks." You don't have to call it "economic tasks" if every task is an economic task.But in this paper's favor, they do look at four million conversations on Claude, the world's second leading LLM. So even if what they're not measuring is exactly economic usage, this is a very important cross-section of usage.Andrey: And it's important for a lot of stakeholders - policymakers who are thinking about what LLMs are being used for, businesses thinking about consumer needs they can service with these models, and obviously Anthropic itself to understand its user base. The new model they released today is very focused on computer programming tasks in a way that other competitor models are not. That must be informed by the fact that their users really value this use case, and they're going to meet their customers' needs rather than just trying to push a model that's very smart generically but isn't catered to the use cases of the user base.Seth: You said three really interesting things there. The first is that to the extent these models are not perfect substitutes for each other, we would expect them to develop specializations. One important limitation of this study is maybe Claude just turned out to be the coding-specialized LLM or the writing-specialized LLM, and that's what we're picking up. I don't think we're that deep into the tech tree at this point where the models are that different for that to be a giant consideration, but you can imagine that being a bigger consideration as we get four or five more years down the line.The second thing you pointed out is this question of to what extent model builders are able to direct what tasks they get better at. Something I really want us to talk about in a future episode is to what extent development is directable in the sense of "I'm going to make an AI that's really good at coding" and "you're going to make an AI that's really good at writing." To what extent are those separate tasks versus just making a better AI, with maybe a little bit of an intangible asset in making a shell that's useful for coders, but that's basically trivial?Andrey: That is a really big question. I tend to come from the world of thinking about personalized rankers - in my dissertation, I thought about personalization.Seth: If I recall, your dissertation was about ranking people from best to worst, right?Andrey: I would never rank people, Seth, come on! Only by objective metrics.Seth: Thank you. It was science.Andrey: More seriously, a lesson from digital technologies has been that personalized rankers, personalized recommendations, experiences really increase the utility of users. They make users use the product more and create more value for the users, also through personalized advertising. I think it would be a little weird to then have this generic model that's not in any way catered to the users.So far, we haven't seen a lot of catering to users. We've seen big models and maybe system prompts, but not a lot of talk about "What if you tweak the final layer to give a certain type of answer that this certain type of person wants?" That's been left to specific application developers - so Harvey might be developing the lawyer version of ChatGPT, and they're going to do some fine-tuning on their end to cater it. But to the extent that there's an interface that people are generically using, you would expect the designers of the model for that interface to think really hard about what their users want.Seth: Right. So there are two questions there: is it directable, and to the extent that there is a non-directable component, what's the ratio of investment in the non-directable component to the custom occupation wrapper or the custom task wrapper that adds a little bit more, but maybe not fundamentally? Anyway, great question for a future episode.So, they had four million conversations. They basically got the AI to label all of the example conversations and assigned them to tasks that are then assigned to occupations. Similarly, they classified each of these 4 million conversations by whether they're more "automate" versus "augmenty." I'll have more to say about that in limitations.One thing I want to say here before we get into the findings is the amount of human validation of these automatic ratings seems a little bit limited. They talk about in their appendix conversation and 86 percent agreement between their 150 human coders and the AI labeler. Not terrible, not great. How do you feel about the automatic labeling here? They have 4 million observations, and they only checked 150? It seems a little low.Andrey: My prior is that this can do a pretty good job. If I was a referee, I might push them a bit more on this - it's not that expensive to check the conversations. I guess what they would tell you is that they actually really care about privacy-preserving methods, so maybe they didn't feel comfortable having external raters check the data. One interesting emphasis of this paper is how they're really worried about privacy concerns, which makes sense because people talk to these chatbots about very personal issues related to their health.Seth: Things they wouldn't talk about at work.Andrey: There are even studies that suggest you tell chatbots things that you wouldn't tell your therapist. So I think this emphasis on privacy seems very prudent for a chatbot provider, but maybe it limits what they can do.Seth: It's also non-interventional, which limits them a lot too. It's just purely descriptive, but we like descriptive stuff, don't we Andrey?Andrey: Yes. This is what our profession under-provides.Seth: So maybe we can start running through the specific findings now. Their first main result is what occupational groups use Claude, proportional to their representation in the US economy. They find that the most common use of Claude is for computer and mathematical conversations - 37 percent of conversations, which in my brain is some combination of coding help and tech support. But when you think about it, only 3.4 percent of the U.S. workforce is involved in computer and mathematical occupations. So that's a giant over-representation of those tasks in their data.Meanwhile, Office and Administrative Support, which is 12 percent of American workers, they see as only constituting 8 percent of their conversational tasks - a slight under-representation of office work, which you would think would be at least somewhat susceptible to automation.What do they not see any usage of AI for at all? Very little usage for farming, fishing, and forestry - not a surprise, very physical. Physical and social science - 6 percent usage, people are asking questions about that, maybe a slight overrepresentation compared to the US economy. Very low usage for legal services, which I'm a little surprised about. I've definitely asked Claude some legal questions. I don't know what jumps out at you from figure three, Andrey.Andrey: The office and administrative support is fascinating because it's so low when obviously so much of the work can be automated.Seth: That's weird to us.Andrey: Yeah, just filling out forms, creating forms, various compliance tasks - I wouldn't be surprised if the current generation of models is already better than the vast majority of the humans doing that job, and certainly when they do it together, they should do a better job. So this really speaks to the issue of diffusion and barriers to adoption.Imagine you're an office worker, not a senior manager or anything, and you have a bunch of tasks to do about expense reports and so on. You might be hesitant or actually just disallowed from using LLMs to do this type of work. My mom works in a hospital, and she tells me that there are a lot of restrictions about the use of LLMs within the hospital. That might be for legal reasons or even perceived legal reasons - maybe there aren't actually any laws being broken by using it within the context of a hospital, but the management might be conservative in a variety of ways.So even though this would be very useful, it is not being done. Both of us have the strong prior that Office and Administrative Support work has to be automated by LLMs.Seth: If it should help us with anything.Andrey: The legal services thing is quite similar. This raises another question about this index - the number of times you use the LLM for something is not indicative of the value of the usage.Seth: Are you telling me writing a thousand lines of code might not have produced as much value as someone who wrote two lines of code?Andrey: Exactly. As the cost goes down, you might start using these things for very trivial things that aren't very high value. The other version of this is, "Hey, that one medical question I asked Claude might've saved my life," and the value of that is much greater than every other interaction I've had with Claude.Seth: Wait, you can't drop that in the conversation without giving context.Andrey: No, there's no actual context for that. I'm not saying it saved my life, but I have used it to help me interpret medical results, for example. Maybe that's not well-advised, but it's given me peace of mind and provided value that I think is probably greater than the value it might've provided for other things I use it more frequently for, like to write referee reports for papers. Just to be clear, I write my own reports, but I do like to check my reasoning with Claude.Seth: Now we're going to start moving into some results from the paper that I find much less convincing. The authors argue that they can measure, between occupations, what percentage of tasks do people use AI for at least a little bit. For a dataset with four million conversations, what does "at all" mean? It means they need to find at least 15 observations of someone having a conversation on this topic to count it as a task that appears in the data. Why 15? Who knows? Maybe it has some esoteric properties they find desirable.Why am I a little suspicious of this? We already heard they only double-checked 150 of these classifications, with an 86 percent correct classification rate. So 14 percent of the classifications are wrong, they've got 4 million of them, and they only want to see 15 instances for it to count as happening? I'm not 100 percent on board with this.Andrey: I agree with you. It could be that a lot of these low-end things are really just misclassifications. You'd want to change that threshold - to vary it to 100 or 1000.Seth: It's not necessarily just misclassification. This is supposed to be a paper about economic value creation. The fact that I tried a thing two times and it never worked, then I stopped using it - that could add to 15 use cases from people experimenting and realizing it doesn't work.Andrey: This goes back to one of my big questions: Where's the indicator of success? Where is the success button at the end? I know they collect likes and not-likes, but there's a sense in which we don't know whether someone actually accomplished what they were seeking to accomplish with their interaction with the chatbot.Seth: So I'm not sure how much we learn from this analysis beyond what we already heard. The next result we should cover is, instead of looking at occupations, they look at different skills that seem to be called for in these Claude conversations. The things at the top of the list are pretty intuitive for me - they list critical thinking, active listening, reading comprehension, writing, and programming as basically the five or six top usage skills that are called for when people use Claude. Those all make sense to me.But the stuff on the bottom I find pretty surprising. They find that almost none of the records relate to repairing or operation and control. That's a little surprising - I know YouTube is probably a better source overall for repair advice, but it seems like a natural place to get help from chatbots. The next set that I'm very surprised to see so lowly ranked are things like management of financial resources, time management, management of personnel, monitoring, selection - these are all managerial jobs. Other than judgment and decision making, which ranks reasonably high up, most of these managerial tasks are really not called for in Claude.I would ask people not to sleep on this because we have been seeing employment growth in managerial occupations. There's some sense in which managerial or entrepreneurship tasks have to be the scarce complement to AI. It is very striking to see the lack of managerial talent called for in these Claude queries.Andrey: That's a great observation. It raises a lot of interesting hypotheses that would be nice to investigate. Before I get to those managerial tasks, I do think that the number one task, critical thinking, is, of course, a managerial task - it's cognitive labor, and hopefully managers are critically thinking.Seth: And hopefully they're active listening. I mean, there's some overlap for every task.Andrey: Looking at these things - let's start with repair. I think the right question might be, conditional on having to repair something, how often do you use an LLM? That could be 100 percent, and it would still be a tiny portion of all the usage because you just don't need to repair things that often. Negotiation is similar - when was the last time I negotiated something?Seth: It's stressful, dude. Negotiating is stressful.Andrey: It is stressful. So I think one of the things is just the base rates - that's really important to consider here. The other thing, and this is a point that Tyler Cowen makes a lot, is that the people who will learn to use the AIs will be most successful. Maybe the AIs are already very good at some of these tasks, like active learning, management of personnel resources, but people don't view them as AI tasks. And maybe that's because there isn't such a close feedback loop as there is in programming. As a result, they're just not going to the AIs for advice. That might be a growth opportunity or place where a lot of value can be generated, just dealing with diffusion friction.Seth: Right. If you could figure out a way to overcome people's frictions, or if you built a wrapper that made using it more intuitive for those tasks, maybe that's a big entrepreneurship avenue. If you get a unicorn startup based on that idea, please send your checks to Justified Posteriors.Are there any other results you wanted to cover before we start talking about our posteriors?Andrey: I guess the augmentative versus automative aspect.Seth: When do you buy this at all? What do you think of this? Maybe you can tell us the five different kinds of tasks that they classify conversations into.Andrey: They're classified as directive tasks (like "complete this task with minimal interaction"), feedback loop (like debugging a piece of code - you put in a bug, it gives you a potential solution, you try it, then you come back to it), task iteration (which seems a lot like a feedback loop to me, but it's a collaborative refinement process), learning (knowledge acquisition and understanding), and validation (I've already written this thing, can you check it and suggest any improvements).They say that directive and feedback loops are automative, while task iteration, learning, and validation are augmentive. Then they show what percentage of conversations are of each type - about 15 percent are feedback loop automation, about 28 percent are directive automation. For the augmentative behaviors, there's a lot of task iteration and learning going on.Seth: I love the idea of looking at the style of the conversation - is it a feedback loop, is it validation? That's super kosher, and I'd love to see these results. I'm not surprised, but it's interesting to see that the majority is task iteration at 31%, while validation is pretty rare at 3%. So on its face, some of these results aren't so surprising.The part that I object to deeply is calling one of these sets "automation" while calling the other set "augmentation." I've been studying robots taking our jobs for over a decade now, Andrey, and as far as I can tell, there is not a good definition for automation. When people talk about automation, what they usually mean is a technological change that reduces the attractiveness of jobs, that reduces demand for labor - or at least that's what I think it should mean. If you said, "Here's my automation technology, it's increasing demand for labor," it doesn't sound very automated to me. It sounds like you need more labor.Andrey: Well, conditional on type, right? So you have a technology that reduces demand for a certain type of labor, but there might be complementary labor types for which demand increases. One might say that's automative of one occupation and not of the other.Seth: Now explain the absurd disease that this gets you into. My favorite example of how something that looks automated at the micro level actually is augmentative at the macro level comes from the U.S. experience of slavery. Back in the olden days, when America was growing cotton with slave labor, it was very time-intensive to take the seeds out of the cotton. Cotton was a crop people used for some kinds of clothing - it was in the mix.Then a technology came around called Eli Whitney's cotton gin, which basically automated the incredibly labor-intensive process of taking the seeds out of the harvested cotton. So we're going from a super labor-intensive job, 100 percent labor, to a now 99 percent capital job. Does this reduce demand for slaves in the American South? No! It leads to an explosion in demand for slaves in the American South because now American cotton is able to outcompete European wool and European linen.There's a micro sense in which the cotton gin automates the task of taking the seeds out of cotton, but there's a macro sense in which speeding up cotton production dramatically increases demand for people making cotton. If you were going to say anybody was automated, you'd say it was the sheep herders that got their wool replaced with cotton - they were the people who, if anybody, got automated. I find the way that people talk about automation very loosely here frustrating.Andrey: I'm with you, Seth. I do think there's a difference between occupation and task level. It makes a little more sense at an occupation level rather than a task level. The slaves in your example, or the bank tellers in the ATM example - their job consisted of a mix of tasks. Then some of those tasks became very cheap to do automatically, but the other tasks remained.To steelman a version of automation: if every task that a person in a particular occupation does got automated, they might find work in other occupations, but it's not necessarily obvious that the same worker benefits from increases in demand in other parts of the economy caused by this technological change. You might think of undifferentiated labor - of course, undifferentiated labor is going to be able to do any type of labor where demand has increased that doesn't require an education or whatever. But I'm not sure that's representative.Seth: So on its face, if you told me, "Hey, look, this job that you used to do, your productivity has gone up by 10X" - am I anticipating doing as many hours of that job as I did before? No, probably there's complementarities across different tasks. If you make me way more productive because you automated some subset of my tasks, I'll probably do less of the job, definitely do less of the automated tasks, but maybe less of the unautomated tasks as well. But that's a partial equilibrium analysis, and even if it's rare, it is certainly conceptually possible for the general equilibrium effects to work differently for my occupation or my remaining tasks.My takeaway here is, people use AIs for a mix of things. Some of them look a little bit more like one-shot interactions, some look a little bit more like iterative interactions, some look like the human is bringing a little bit more of their own thinking. Maybe that's the way to think about it - 57 percent of these tasks, the user is bringing more of their own thinking and creativity. I wouldn't call that augmentation versus automation, but I do think there is a distinction here that's interesting.Andrey: I don't even know if I like what you just said, Seth. The example of the directive task is "format this technical documentation in markdown," but someone presumably wrote that technical documentation. That someone is probably the user.Seth: Right, coming up with the prompt is the worker's work in the automated task.Andrey: But I do think this is valuable descriptive work about how people are using the tools. To the extent that it's changing over time, that's telling us something. An important concept in these systems is "human in the loop" - at what point do you not need the human in the loop?If there's a way to see that the chatbot one-shots a task with very high probability, that's interesting. But once again, what I'd want here is a success metric - did the interaction succeed in generating a result that was valuable, correct, etc., to the user? Without that, it's just really hard to interpret this.Seth: So maybe this is a natural place for us to transition into limitations. We've listed a few. One limitation is that amount of time spent talking about something is not exactly proportional to economic value. Lord knows I spent a lot of time talking about the New York Jets, and it's not helping the Jets succeed at all.Another limitation that I pointed out is it's not clear that everything everyone uses AI for is a work task, which introduces both problems in terms of their only classification schema being work tasks, so if somebody's using AI for not work, it's gonna do something weird. And also just based on the fact that if you can't distinguish between what's experimenting versus what's in-production operations, it's hard to really connect this to economic value. What do you see as the biggest limitations?Andrey: You've already said most of them. I guess in addition to what you've said, I'm fascinated by this model specialization thing - are people going to Claude for coding and going to other models for different tasks? I don't know.Seth: Oh man, I'm sure Elon Musk said to his staff, "We need the AI that's best at meme posting."Andrey: Yes, yes.Seth: They list in terms of their limitations that model classification might be imperfect. I do think that's an issue - I know you don't worry about it so much.Andrey: I do worry about it for the minor tasks, to be clear. I don't think they're getting programming wrong that much on average - it's not a difficult task to classify. Can I also now say what Claude said in its referee report?Seth: This is perfect timing. What did Claude say about its own paper? Now be mean.Andrey: I first asked it to write a generic economics referee report, and it gave concerns about external validity, task complexities, how it distinguishes professional from novice-level inquiries, dynamic considerations, the O*NET framework limits, and causal interpretation - readers might draw causal inferences about AI's impact on the labor market, and the authors should more explicitly describe the limitations of drawing such conclusions.Then I said, "Be real - if this was a real economics referee report, there would be additional concerns." So major concerns: One, fundamental identification issues - the paper fundamentally fails to establish that it is measuring what it claims to measure. Two, absence of a theoretical framework - I don't really blame them for this one. You shouldn't put theory into a paper just because there is theory about the topic. Three, selection bias and external validity because of just having Claude users. We've already talked about this - I think it's a limitation, but it's still interesting even with this limitation.Four, endogeneity concerns - that's an interesting way to put it.Seth: What are they worried is endogenous?Andrey: Claude is worried that Claude's capabilities in different domains may lead people to use Claude in different ways, that Anthropic's marketing and positioning of Claude may lead people to use Claude in different ways, that the user interface design favors certain interactions, and that temporal factors, including Claude's release timing relative to competitors, may also affect these patterns.This is a nice point - how do usage patterns change when they've just released a new model? Are we seeing a fundamental change in the usage patterns or mostly more of the same? Is it a slow drift or a sharp discontinuity? There are so many questions to answer with this type of data, but not necessarily economic ones.Seth: Well, the fact they call it an economic index suggests that we're going to get updates, so I'm excited for that.Andrey: I think the overtime series of this type of usage is very interesting.Seth: Is it fair to say that Claude did not hit upon what I see as the biggest limitation here, which is the assumption that this is all economic activity when a lot of it probably isn't?Andrey: No, that's its number one point. It calls it "fundamental identification issues" - the mapping from "a person asked Claude about X" to "AIs being used to perform economic tasks" involves unsubstantiated leaps in logic that undermine the entire analysis. That's Claude. Calm down, buddy.Seth: That's reviewer two, dude.Andrey: Yeah.Seth: I feel like if they just left "economic" out of the title, that would defeat that objection pretty heavily.Andrey: There's a paper we haven't discussed on this podcast yet, which is the paper by friend of the pod, Daniel Rock, on task exposure. We'll probably devote a separate episode to this, but I do wonder, how do you compare this paper to that?Seth: That's fascinating. That's a paper about what the AI thinks it can do, whereas this is a paper about what are people actually using AI for. If I recall, Dan's paper (GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models, 2023, in Science) does have an extension with some sort of validation - I forget if it was from a survey or from something used on Stack Overflow, but they did have some correlation between what we think people should use this for and what they actually use it for that was very positive.So I like that mixed-methods approach of "here's what we think they should be able to use it for, here's what they are using it for." How else would I compare these two papers? They're kind of doing different things. One's a descriptive paper about usage now. One's sort of a possibilities paper about whether tasks are conceptually automatable by these kinds of systems. So I view them as complementary.Andrey: I think I'm with you there. One thing one could think about is a measure of the gap between the potential capabilities from Rock et al. and realized usage from this paper.Seth: And you measure that wedge and you give it a fun name. You call it the Fradkin Wedge. And it's like a measure of the size of the administrative and legal frictions in that domain.Andrey: It would be interesting to have it.Seth: That's where this is going. I think the next steps for this sort of very zoomed-out literature are: one, connecting what people actually use it for to these measures of what we think it should be good at; and two, the thing that you keep coming back to - an economic success measure. Did this succeed? Am I happy with it? Did it do the job? Because as we keep talking about, you can talk about a thing a lot without getting any work done.Andrey: All right, so maybe let's move on to our posteriors.Seth: Our posteriors. I would say that I came into reading this paper thinking that people use AI first for coding and second for cheating on their homework. Nothing I have seen in this paper contradicts that prior.I guess the biggest update for me would be how striking the lack of usage for managerial tasks is. I would've thought things like manager time usage or scheduling tasks - that's the kind of thing I would have thought AI would be good at. And to see it not being used for that is interesting and suggestive. Did you have any big surprises in what people are using AI for?Andrey: I think I had the same reaction as you. I don't think I had a very strong prior about how large the computer share of usage would be - I just knew it would be pretty large for all the reasons we talked about. And then I was surprised about office and administrative support - we can explain it post hoc, but it is surprising that the jobs we think are most mundane, the knowledge-type work that should be automated first - that's not where the usage is. That is really interesting.Seth: I guess the last thing I'll say is maybe I thought there would be a little bit more in the artistic realm because we always talk about AI being really good in domains where having a lot of candidate options that you can sort through is good. That's kind of like the Avi Goldfarb machines framework, and you'd think art would be perfect for that - generate 1000 images and choose the one good one. But art is merely at 10 percent of usage, which is a little bit lower than I would have guessed.Andrey: For me, it's higher than I would have guessed. I don't view Anthropic as investing heavily into artistic modeling.Seth: So now we get back to the selection issue - Claude might not be the one you go to for that.Andrey: DALL-E is an OpenAI model. The other major image generation models are also not produced by Anthropic, the major video models are not produced by Anthropic. Anthropic must have a voice model, but I've heard more about Whisper and others that are not Anthropic properties. For music, we have specialized players like Suno AI that seem to be in the lead. So if you're an artist, you might use a chatbot to ideate at a very high level, but when it comes to making your art, you're going to use another tool.Seth: Right. And to the extent that you're using a lot of AI for iterating on drawing or design, you're probably not using Claude. But that comes back to a limitation of the paper - it can't move our beliefs about the usage of AI overall that much if it's only showing us Claude usage.Andrey: We need the API data. We need the API economic activity index.Seth: Exactly. So what would be the perfect next dataset we need to really answer these questions?Andrey: The dream dataset is a cross-platform usage dataset. People have been doing survey studies where they ask how people use LLMs, and those studies are good at what people report, but they're not measuring the use cases in a finely grained manner or the frequency. If we had a dataset of a representative sample of LLM usage in a population, that would be really great. It'd be really great to get business users and measures of willingness to pay for these things. But I don't think we're going to get those datasets - the reason we don't have them is they're really, really hard to collect.Seth: Well, I guess you can measure the difficulty of the task by the product that you would have gotten from doing the task, or at least you can bound it.Andrey: One interesting thing is that OpenAI released a new benchmark that uses actual jobs on Upwork and whether the AI could complete them. That's not going to give you a representative sample of anything, but if we're thinking about economic impacts, I do think that if you can go end-to-end on a task that someone is willing to pay money for - not a small amount of money - that is an economic task. Upwork is not a representative sample of tasks in the economy, obviously, but if someone is already paying for the job to be done and that gets end-to-end automated by an LLM system, that's fascinating.Seth: I agree. We should definitely read that paper and more along those lines someday soon. But maybe until then, our audience will have to read economics papers on their own. Do you have any closing thoughts for our beautiful and well-informed guests?Andrey: Make sure to review, like, comment, subscribe to Justified Posteriors. Let us know what type of content you enjoy seeing and we'll try to provide more of it. Or if there are any topics that you would like us to cover, we are happy to take suggestions. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit empiricrafting.substack.com

The discussion dives into the crucial question of balancing the massive economic potential of AI against the existential risks it poses. With insights from economist Chad Jones, they outline models suggesting society might rationally accept significant extinction risks for AI-driven prosperity. The conversation highlights the importance of investing a substantial portion of global GDP in AI safety, potentially ranging from 1.8% to 15.8%. Philosophical tensions emerge around individual vs. collective risk, shaping how we navigate our technological future.

Dive into the intersection of economics and healthcare with our latest podcast episode. How much can AI systems enhance high-stakes medical decision-making? In this episode, we explore the implications of a research paper titled “Diagnosing Physician Error: A Machine Learning Approach to Low Value Health Care” by Sendhil Mullainathan and Ziad Obermeyer.The paper argues that physicians often make predictable and costly errors in deciding who to test for heart attacks. The authors claim that incorporating machine learning could significantly improve the efficiency and outcome of such tests, reducing the cost per life year saved while maintaining or improving standards of care. We discuss the challenges and limitations of implementing AI in healthcare, the potential biases doctors may have, and the broader systemic issues in medical technology adoption. Sponsored by the Digital Business Institute at Boston University’s Questrom School of Business. Big thanks to Ching-Ting “Karina” Yang for her help editing the episode.-🔗Links to the paper for this episode’s discussion:(Full Paper) Diagnosing Physician Error-🗞️Subscribe for upcoming episodes, post-podcast notes, and Andrey’s posts:💻 Follow us on Twitter:@AndreyFradkin https://x.com/andreyfradkin?lang=en@SBenzell https://x.com/sbenzell?lang=en This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit empiricrafting.substack.com

The discussion revolves around the puzzling low interest rates in the face of imminent artificial general intelligence (AGI). It delves into market skepticism towards rapid technological shifts and their economic implications. The hosts debate the relationship between AGI, investment opportunities, and growth rates, highlighting differing opinions on future economic dynamics. Additional insights into historical interest rates and the challenges posed by income inequality and existential risks create a thought-provoking examination of our financial landscape.

Does the U.S. auto industry prioritize consumers or corporate profits? In this episode of Justified Posteriors, hosts Seth Benzell and Andrey Fradkin explore the evidence behind this question through the lens of the research paper “The Evolution of Market Power and the U.S. Automobile Industry” by Paul Grieco, Charles Murry, and Ali Yurukoglu.Join them as they unpack trends in car prices, market concentration, and consumer surplus, critique the methodology, and consider how competition and innovation shape the auto industry. Could a different competitive structure have driven even greater innovation? Tune in to find out!Sponsored by the Digital Business Institute at Boston University’s Questrom School of Business. Big thanks to Ching-Ting “Karina” Yang for her help editing the episode.-🔗Link to the paper for this episode’s discussion:(Full Paper) The Evolution of Market Power in the US Automobile IndustryFigures for reference:🗞️Subscribe for upcoming episodes, post-podcast notes, and Andrey’s posts: This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit empiricrafting.substack.com

The discussion unpacks whether simply scaling AI models leads to transformative results. It explores the predictable outcomes of increasing compute, data, and parameters, but questions their ultimate effectiveness. The conversation also delves into the complexities of data quality and model performance, emphasizing the importance of innovative solutions over mere resource addition. Real-world applications like translation and software development reveal AI's potential and economic implications. The dynamic between AI capabilities and human performance is examined, uncovering both challenges and opportunities.