Interviewing for Incident Analysis w/special guest John Allspaw

The new website is live! thisisfinepod.com

You can find John Allspaw at Adaptive Capacity Labs: https://www.adaptivecapacitylabs.com

Mike McGill, the skateboarder: https://en.wikipedia.org/wiki/Mike_McGill

Annie Duke’s Thinking in Bets, referenced by our question-asker is a great one: https://bookshop.org/p/books/thinking-in-bets-making-smarter-decisions-when-you-don-t-have-all-the-facts-annie-duke/31466984521c3d8a?ean=9780735216372&next=t

Naturalistic Decision Making has its own association, which has a ton of resources (and a conference!) - https://naturalisticdecisionmaking.org/

They also have a podcast! https://naturalisticdecisionmaking.org/new-podcast/

Gary Klein is the NDM guy - https://bookshop.org/p/books/seeing-what-others-don-t-the-remarkable-ways-we-gain-insights-chief-scientist-gary-klein/c4ae5e017fe005ff?ean=9781610393829&next=t

We contrast him and his style of approaching cognition and decision making with Kahneman and Tversky.

Kahneman and Tversky wrote a lot, but Judgement Under Uncertainty is probably the most famous? https://www.science.org/doi/abs/10.1126/science.185.4157.1124

And Kahneman wrote Thinking Fast and Slow: https://bookshop.org/p/books/thinking-fast-and-slow-daniel-kahneman-phd/83a544fe6f98df87?ean=9780606275644&next=t

It has been zero episodes since we’ve mentioned Lisanne Bainbridge’s Ironies of Automation: https://ckrybus.com/static/papers/Bainbridge_1983_Automatica.pdf

But also she has Verbal Reports as evidence of the process operator’s knowledge: https://www.sciencedirect.com/science/article/abs/pii/S1071581979603075?via%3Dihub

And the Etsy Debriefing Guide is super great: https://extfiles.etsy.com/DebriefingFacilitationGuide.pdf

Sidney Dekker and The Field Guide are foundational: https://bookshop.org/p/books/the-field-guide-to-understanding-human-error-sidney-dekker/3a4209dfc8b3a721?ean=9781472439055&next=t

From Dekker’s field guide (pg 47) there is a list referencing Gary Klein’s questions for an incident investigation:

Cues: 

What were you seeing?

What were you focusing on?

What were you expecting to happen?

Interpretation: 

If you had to describe the situation to your colleague at that point, what would you have told?

Errors: 

What mistakes (for example in interpretation) were likely at this point?

Previous experience/knowledge:

Were you reminded of any previous experience?

Did this situation fit a standard scenario?

Were you trained to deal with this situation?

Were there any rules that applied clearly here?

Did any other sources of knowledge suggest what to do?

Goals:

What were you trying to achieve?

Were there multiple goals at the same time?

Was there time pressure or other limitations on what you could do?

Taking action:

How did you judge you could influence the course of events?

Did you discuss or mentally imagine a number of options or did you know straight away what to do?

Outcome:

Did the outcome fit your expectation?

Did you have to update your assessment of the situation?

John mentioned Uptime Labs, who do staged worlds for software incidents: https://uptimelabs.io/

Facets of Complexity in Situated Work is here: https://www.researchgate.net/publication/345523195_Facets_of_Complexity_in_Situated_Work

On the Jamie Zawinski quote: https://regex.info/blog/2006-09-15/247

If you don’t know the parable of the blind men and the elephant: https://en.wikipedia.org/wiki/Blind_men_and_an_elephant