Understanding Noise: What Affects Human Judgement feat. Olivier Sibony

Researchers and academics tend to focus on what people have in common, instead of looking at individual differences and nuances, which often drive decisions and judgements. In his latest book, Olivier Sibony, professor of strategy and award-winning author, showed the detrimental effects of noise in many fields, like judicial systems, hospitals, recruitment, human resource departments, and machine learning. 

Olivier emphasizes that wherever there is judgement, there is noise. Yet, individuals and organizations are generally unaware of it. In this episode, learn how we can reduce both noise and bias, so we can make better decisions.

Episode Quotes:

How do noise and biases affect errors in decision-making?

Having no bias is better than having a bias, but noise and bias are completely separate, independent sources of error, and you actually need to reduce both.

How difficult is it to detect and eliminate noise when making decisions?

It is very hard to see noise. But in a system — in anything that makes repeated decisions— in—say the judicial system or the insurance company that prices a lot of insurance policies. Or in a hiring organization that makes a lot of hiring decisions. It is actually difficult to establish the presence of bias because you need to know where the truth is, as you pointed out. But it's actually quite easy to do a test of noise, what we call a noise audit. Which means to give the same case to a number of different judges, a number of different people, and to check how different their judgements are. Whenever you do that, in our experience, you find that, of course, there is a difference.

What if idiosyncratic biases and extraneous factors are applied in decision-making?

Our minds are instruments of judgements. Those instruments are not completely stable. They are not completely consistent. We're not completely consistent with ourselves all the time. We think we are, especially when we're making only one judgement. We can't imagine that at a different time, in a different mood, in a different temperature, or if our favorite football team had lost the game yesterday instead of winning it, we would make a different decision. Yet, all those things are true.

Time Code Guide:

00:02:44 Why does noise in data receive such little attention in most fields, except for data science?

00:06:29 Do biases cancel each other out and ultimately cause you to make bad decisions?

00:16:45 Different kinds of noise, an earlier version of auditing noise and sentencing guidelines

00:24:16 How can individuals learn and improve how to avoid bias?

00:30:34 Structuring decisions using a reflectively designed framework and how it applies to companies and individuals

00:34:40 Thoughts on algorithmic decision-making, leaving human judgment out, and understanding essential characteristics of judgment

00:44:56 Elimination of noise, equal protection, and consideration of individual idiosyncrasies in the judicial system

00:51:02 Auditing noise and cleaning out processes that doesn't help eliminate noise

Show Links:

Guest Profile

• Academic Profile at HEC Paris
• Faculty Profile at the University of Oxford
• Professional Profile from His Official Website
• Olivier Sibony on Twitter

His Work

• Olivier Sibony on Google Scholar
• Noise: A Flaw in Human Judgment
• You're About to Make a Terrible Mistake: How Biases Distort Decision-Making and What You Can Do to Fight Them
• Cracked it!: How to solve big problems and sell solutions like top strategy consultants

Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.