Improving research reliability beyond peer review

With 698 inmates per 100,000 citizens, the U.S. is by far the leader among large wealthy nations in incarceration. But what effect does imprisonment actually have on crime?

According to David Roodman, Senior Advisor to the Open Philanthropy Project, the marginal effect is zero.

* 80,000 HOURS IMPACT SURVEY - Let me know how this show has helped you with your career.
* ROB'S AUDIOBOOK RECOMMENDATIONS

This stunning rebuke to the American criminal justice system comes from the man Holden Karnofsky’s called "the gold standard for in-depth quantitative research", whose other investigations include the risk of geomagnetic storms, whether deworming improves health and test scores, and the development impacts of microfinance.

Links to learn more, summary and full transcript.

The effects of crime can be split into three categories; before, during, and after.

Does having tougher sentences deter people from committing crime?

After reviewing studies on gun laws and ‘three strikes’ in California, David concluded that the effect of deterrence is zero.

Does imprisoning more people reduce crime by incapacitating potential offenders?

Here he says yes, noting that crimes like motor vehicle theft have gone up in a way that seems pretty clearly connected with recent Californian criminal justice reforms (though the effect on violent crime is far lower).

Finally, do the after-effects of prison make you more or less likely to commit future crimes?

This one is more complicated.

Concerned that he was biased towards a comfortable position against incarceration, David did a cost-benefit analysis using both his favored reading of the evidence and the devil's advocate view; that there is deterrence and that the after-effects are beneficial.

For the devil’s advocate position David used the highest assessment of the harm caused by crime, which suggests a year of prison prevents about $92,000 in crime. But weighed against a lost year of liberty, valued at $50,000, plus the cost of operating prisons, the numbers came out exactly the same.

So even using the least-favorable cost-benefit valuation of the least favorable reading of the evidence -- it just breaks even.

The argument for incarceration melts further when you consider the significant crime that occurs within prisons, de-emphasised because of a lack of data and a perceived lack of compassion for inmates.

In today’s episode we discuss how to conduct such impactful research, and how to proceed having reached strong conclusions.

We also cover:

* How do you become a world class researcher? What kinds of character traits are important?
* Are academics aware of following perverse incentives?
* What’s involved in data replication? How often do papers replicate?
* The politics of large orgs vs. small orgs
* Geomagnetic storms as a potential cause area
* How much does David rely on interviews with experts?
* The effects of deworming on child health and test scores
* Should we have more ‘data vigilantes’?
* What are David’s critiques of effective altruism?
* What are the pros and cons of starting your career in the think tank world?

Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type 80,000 Hours into your podcasting app. Or read the transcript below.

The 80,000 Hours Podcast is produced by Keiran Harris.