The Nonlinear Library EA - How Long Do Policy Changes Matter? New Paper by zdgroff
Nov 2, 2023
08:58
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How Long Do Policy Changes Matter? New Paper, published by zdgroff on November 2, 2023 on The Effective Altruism Forum.
A key question for many interventions' impact is how long the intervention changes some output counterfactually, or how long the intervention washes out. This is often the case for work to change policy: the cost-effectiveness of efforts to pass
animal welfare ballot initiatives
,
nuclear non-proliferation policy
,
climate policy
, and
voting reform
, for example, will depend on (a) whether those policies get repealed and (b) whether they would pass anyway. Often there is an explicit assumption, e.g., that passing a policy is equivalent to speeding up when it would have gone into place anyway by
X
years.
[1]
[2]
As people routinely note when making these assumptions, it is very unclear what assumption would be appropriate.
In a
new paper
(my economics "job market paper"), I address this question, focusing on U.S. referendums but with some data on other policymaking processes:
Policy choices sometimes appear stubbornly persistent, even when they become politically unpopular or economically damaging. This paper offers the first systematic empirical evidence of how persistent policy choices are, defined as whether an electorate's or legislature's decisions affect whether a policy is in place decades later. I create a new dataset that tracks the historical record of more than 800 state policies that were the subjects of close referendums in U.S. states since 1900. In a regression discontinuity design, I estimate that passing a referendum increases the chance a policy is operative 20, 40, or even 100 years later by over 40 percentage points. I collect additional data on U.S. Congressional legislation and international referendums and use existing data on state legislation to document similar policy persistence for a range of institutional environments, cultures, and topics. I develop a theoretical model to distinguish between possible causes of persistence and present evidence that persistence arises because policies' salience declines in the aftermath of referendums. The results indicate that many policies are persistently in
place - or not - for reasons unrelated to the electorate's current preferences.
Below I'll pull out some key takeaways that I think are relevant to the EA community and in some cases did not make it into the paper.
Overview of Results and Methods
My strategy in the paper involves comparing how many policies that barely passed or barely failed in U.S. state-level referendums are in place over time. I collect data on all referendums whose vote outcome is within 2.5 percentage points of the threshold for passage (typically 50%) since 1900 in a subset of U.S. states. I then do what's called a regression discontinuity design, which allows me to estimate the effect of passing a referendum on whether it is in place later on.
The headline result from the paper is below. Many referendums that barely fail eventually pass in the first few years or decades afterward, and then this levels off. At 100 years later, just under 80% of the barely passed ones are in place compared to just under 40% of the barely failed ones. Importantly, the hazard rate - the rate at which this effect declines over time - is much lower in the later years, meaning that if you were to extrapolate this out beyond 100 years, the effect at 200 years would be expected to be significantly more than 40% * 40%.
Something relevant to EAs that I don't focus on in the paper is how to think about the effect of campaigning for a policy given that I focus on the effect of passing one conditional on its being proposed. It turns out there's a method (
Cellini et al. 2010
) for backing this out if we assume that the effect of passing a referendum on whether the policy is in place lat...
A key question for many interventions' impact is how long the intervention changes some output counterfactually, or how long the intervention washes out. This is often the case for work to change policy: the cost-effectiveness of efforts to pass
animal welfare ballot initiatives
,
nuclear non-proliferation policy
,
climate policy
, and
voting reform
, for example, will depend on (a) whether those policies get repealed and (b) whether they would pass anyway. Often there is an explicit assumption, e.g., that passing a policy is equivalent to speeding up when it would have gone into place anyway by
X
years.
[1]
[2]
As people routinely note when making these assumptions, it is very unclear what assumption would be appropriate.
In a
new paper
(my economics "job market paper"), I address this question, focusing on U.S. referendums but with some data on other policymaking processes:
Policy choices sometimes appear stubbornly persistent, even when they become politically unpopular or economically damaging. This paper offers the first systematic empirical evidence of how persistent policy choices are, defined as whether an electorate's or legislature's decisions affect whether a policy is in place decades later. I create a new dataset that tracks the historical record of more than 800 state policies that were the subjects of close referendums in U.S. states since 1900. In a regression discontinuity design, I estimate that passing a referendum increases the chance a policy is operative 20, 40, or even 100 years later by over 40 percentage points. I collect additional data on U.S. Congressional legislation and international referendums and use existing data on state legislation to document similar policy persistence for a range of institutional environments, cultures, and topics. I develop a theoretical model to distinguish between possible causes of persistence and present evidence that persistence arises because policies' salience declines in the aftermath of referendums. The results indicate that many policies are persistently in
place - or not - for reasons unrelated to the electorate's current preferences.
Below I'll pull out some key takeaways that I think are relevant to the EA community and in some cases did not make it into the paper.
Overview of Results and Methods
My strategy in the paper involves comparing how many policies that barely passed or barely failed in U.S. state-level referendums are in place over time. I collect data on all referendums whose vote outcome is within 2.5 percentage points of the threshold for passage (typically 50%) since 1900 in a subset of U.S. states. I then do what's called a regression discontinuity design, which allows me to estimate the effect of passing a referendum on whether it is in place later on.
The headline result from the paper is below. Many referendums that barely fail eventually pass in the first few years or decades afterward, and then this levels off. At 100 years later, just under 80% of the barely passed ones are in place compared to just under 40% of the barely failed ones. Importantly, the hazard rate - the rate at which this effect declines over time - is much lower in the later years, meaning that if you were to extrapolate this out beyond 100 years, the effect at 200 years would be expected to be significantly more than 40% * 40%.
Something relevant to EAs that I don't focus on in the paper is how to think about the effect of campaigning for a policy given that I focus on the effect of passing one conditional on its being proposed. It turns out there's a method (
Cellini et al. 2010
) for backing this out if we assume that the effect of passing a referendum on whether the policy is in place lat...