The Nonlinear Library EA - Some heuristics I use for deciding how much I trust scientific results by Nathan Barnard
Jan 18, 2024
07:20
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some heuristics I use for deciding how much I trust scientific results, published by Nathan Barnard on January 18, 2024 on The Effective Altruism Forum.
I've done nothing to test these heuristics and have no empirical evidence for how well they work for forecasting replications or anything else. I'm going to write them anyway. The heuristics I'm listing are roughly in order of how important I think they are. My training is as an economist (although I have substantial exposure to political science) and lots of this is going to be written from an econometrics perspective.
How much does the result rely on experimental evidence vs causal inference from observational evidence?
I basically believe without question every result that mainstream chemists and condensed matter physicists say is true. I think a big part of this is that in these fields it's really easy to experimentally test hypotheses, and to really precisely test differences in hypotheses experimentally. This seems great.
On the other hand, when relying on observational evidence to get reliable causal inference you have to control for
confounders while not controlling for
colliders. This is really hard! It generally requires finding a natural experiment that introduces randomisation or having very good reason to think that you've controlled for all confounders.
We also make quite big updates on which methods effectively do this. For instance, until last year we thought that
two-way fixed effects did a pretty good job of this before we realised that actually
heterogeneous treatment effects are a really big deal for two-way fixed effects estimators.
What's more, in areas that use primarily observational data there's a really big gap between fields in how often papers even try to use causal inference methods and how hard they work to show that their
identifying assumptions hold. I generally think that modern microeconomics papers are the best on this and nutrition science the worst.
I'm slightly oversimplifying by using a strict division between experimental and observational data. All data is observational and what matters is how credibly you think you've observed
what would happen counterfactually without some change. But in practice, this is much easier in settings where we think that we can change the thing we're interested in without other things changing.
There are some difficult questions around scientific realism here that I'm going to ignore because I'm mostly interested in how much we can trust a result in typical use cases. The notable area where I think this actually bites is thinking about the implications of basic physics for
longtermism where it does seem like basic physics actually changes quite a lot over time with important implications for questions like how
large we expect the future to be.
Are there practitioners using this result and how strong is the selection pressure on the result
If a result is being used a lot and there would be easily noticeable and punishable consequences if the result was wrong, I'm way more likely to believe that the result is at least roughly right if it's relied on a lot.
For instance, this means I'm actually really confident that important results in
auction design hold. Auction design is used all the time by both
government and
private sector actors in ways that earn these actors billions of dollars and, in the private sector case at least, are iterated on regularly.
Auction theory is an interesting case because it comes out of pretty abstract microeconomic theory and wasn't developed really based on laboratory experiments, but I'm still pretty confident in it because of how widely it's used by practitioners and is subject to strong selection pressure.
On the other hand, I'm much less confident in lots of political science research. It seems like places like hedg...
I've done nothing to test these heuristics and have no empirical evidence for how well they work for forecasting replications or anything else. I'm going to write them anyway. The heuristics I'm listing are roughly in order of how important I think they are. My training is as an economist (although I have substantial exposure to political science) and lots of this is going to be written from an econometrics perspective.
How much does the result rely on experimental evidence vs causal inference from observational evidence?
I basically believe without question every result that mainstream chemists and condensed matter physicists say is true. I think a big part of this is that in these fields it's really easy to experimentally test hypotheses, and to really precisely test differences in hypotheses experimentally. This seems great.
On the other hand, when relying on observational evidence to get reliable causal inference you have to control for
confounders while not controlling for
colliders. This is really hard! It generally requires finding a natural experiment that introduces randomisation or having very good reason to think that you've controlled for all confounders.
We also make quite big updates on which methods effectively do this. For instance, until last year we thought that
two-way fixed effects did a pretty good job of this before we realised that actually
heterogeneous treatment effects are a really big deal for two-way fixed effects estimators.
What's more, in areas that use primarily observational data there's a really big gap between fields in how often papers even try to use causal inference methods and how hard they work to show that their
identifying assumptions hold. I generally think that modern microeconomics papers are the best on this and nutrition science the worst.
I'm slightly oversimplifying by using a strict division between experimental and observational data. All data is observational and what matters is how credibly you think you've observed
what would happen counterfactually without some change. But in practice, this is much easier in settings where we think that we can change the thing we're interested in without other things changing.
There are some difficult questions around scientific realism here that I'm going to ignore because I'm mostly interested in how much we can trust a result in typical use cases. The notable area where I think this actually bites is thinking about the implications of basic physics for
longtermism where it does seem like basic physics actually changes quite a lot over time with important implications for questions like how
large we expect the future to be.
Are there practitioners using this result and how strong is the selection pressure on the result
If a result is being used a lot and there would be easily noticeable and punishable consequences if the result was wrong, I'm way more likely to believe that the result is at least roughly right if it's relied on a lot.
For instance, this means I'm actually really confident that important results in
auction design hold. Auction design is used all the time by both
government and
private sector actors in ways that earn these actors billions of dollars and, in the private sector case at least, are iterated on regularly.
Auction theory is an interesting case because it comes out of pretty abstract microeconomic theory and wasn't developed really based on laboratory experiments, but I'm still pretty confident in it because of how widely it's used by practitioners and is subject to strong selection pressure.
On the other hand, I'm much less confident in lots of political science research. It seems like places like hedg...