Randomistas and Radical Research: Uncovering What Works and What Doesn’t feat. Andrew Leigh

Oct 18, 2021

Ask episode

Chapters

Transcript

Episode notes

Experimental research has been a perennial practice in the natural sciences, but social scientists have taken it up in recent decades as well. Randomized trials have been used to design policies aimed at increasing educational attainment, lowering crime rates, enhancing employment rates, and improving living standards among the poor. In his book, Randomistas, Andrew Leigh, an economist turned politician, tells the stories of radical researchers who uncovered what works and what doesn’t using experiments.

From finding a cure for scurvy to improving social policies, Andrew talks about how randomistas are changing the world. In this episode, he shares fascinating accounts of randomized trials and studies from across the globe, and the challenges of gaining acceptance for their findings.

Learn from Andrew and Greg as they draw out key lessons from the book and their thoughts on applying these principles to real-life trials.

Episode Quotes:

Why does measuring the effects of social programs help address our society’s problems?

I don't take any sense of pessimism about our ability to change the world. Well, I think we ought to regard tackling problems such as long-term joblessness. As being as difficult as tackling health challenges like cancer, HIV. And we need to approach them not with blind ideology, but with scientific rigor. Discarding theories but not losing any of our passion for solving these big social problems.

How do randomistas use data from randomized trials?

So, I think the best randomistas are now moving away from whether a particular tweak works or not, to how do we understand the world? And the great thing about randomized trials, unlike natural experiments, is that you can design the experiment very precisely to test the theory.

What can companies and organizations learn from Toms Shoes in terms of having a scientific method for its CSR programs, and do people support causes that take this approach?

One of the firms that's most impressive in that space is Toms Shoes. Toms was founded with the notion that if you bought a pair of shoes in an advanced country, somebody in a poor country would get a pair of shoes as well. So, this wasn't corporate philanthropy bolted on, it was part of the model. And then, after that had been going for about a decade, they asked a team of researchers led by Bruce Wydick to evaluate using a randomized trial. What happened when a community received Toms shoes? And they found that, in general, those getting the shoes had other shoes beforehand. So, they were upgrading the shoes rather than getting their first pair of shoes. It didn't improve school attendance, and that it did increase the sense of dependency on outsiders. And Bruce has a lovely response to it where he says, “Many companies would have looked to bury that result, but Toms didn't.” They saw the result; they adapted, they moved from loafers to sneakers. They looked at ways of giving the shoes through the parents and the community and as incentives for school attendance, looking to improve their program rather than attack the researchers. If Tom's can do that for an evaluation which really struck hard at the very heart of their model, then I think anybody else should be able to do the same with the result that makes them uncomfortable.

What have you learned from Obama’s campaign in terms of connecting better with your constituents?

There are all kinds of little ways in which you can just tweak what you're doing. And for me, that's a part of staying fresh. We should always be looking to learn. I'm always asking colleagues about their ideas on better connecting with constituents because we're in this world, Greg, of declining trust in politicians. And so, it's incumbent on all of us in elected office to be thinking about, how do we do better at connecting with the people who we represent? What are the platforms we can reach out through? What are the ways in which we can connect with people?

Thoughts on experimentation, conversations, and reducing political conflicts

There are good amounts of evidence that encouraging people to have those cross-party conversations can make a difference. I'm surprised as to how many people think they're engaging in politics when they're speaking only to people who voted the same way as they did in the last election. If you want to change the next election result, you've got to find someone who voted one way from the last election and persuade them to do something different. That involves having a conversation with someone who might have different views than you. And guess what? That's the best way in which we've always done politics.

Time Code Guide:

00:01:19 How did he become interested in field experiments

00:02:30 Thoughts on progress, discrediting bad policies and ideas, and discovery of new impactful policies

00:04:40 Resistance to the use of experimentation in policies

00:08:26 Why narrowly focusing on what works and what doesn't means missing opportunities to discover underlying mechanisms

00:09:45 The Highest Paid Person’s Opinion (H.I.P.P.O. Effect)

00:10:33 Why testing something obvious is still important

00:12:31 In randomized trials, where have we seen the biggest improvements in scenario policy?

00:14:37 How do you respond to people who are not comfortable becoming experimental subjects?

00:16:53 The cost of randomizing and not randomizing studies

00:19:49 Thoughts on governments and universities having a centralized or explicit approach to experimentation

00:21:39 When trying something new, should our existing practices and protocols be forced to go through a test as well?

00:24:05 What have you learned in the areas of education and crime prevention that helped form your approach to making policies?

00:26:13 Philosophical approach as one of the most effective approaches to policymaking

00:27:11 Passion-driven in comparison to data, and result-driven policies

00:28:50 Development as an area where a lot of this experimentation is happening

00:35:03 High standards in politics and resistance to greater use of experimentation in the world of policymaking

Show Links:

Guest Profile

His Work

Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.