The Nonlinear Library LW - The 6D effect: When companies take risks, one email can be very powerful. by scasper
Nov 4, 2023
04:31
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The 6D effect: When companies take risks, one email can be very powerful., published by scasper on November 4, 2023 on LessWrong.
Recently, I have been learning about industry norms, legal discovery proceedings, and incentive structures related to companies building risky systems. I wanted to share some findings in this post because they may be important for the frontier AI community to understand well.
TL;DR
Documented communications of risks (especially by employees) make companies much more likely to be held liable in court when bad things happen. The resulting Duty to Due Diligence from Discoverable Documentation of Dangers (the 6D effect) can make companies much more cautious if even a single email is sent to them communicating a risk.
Companies tend to avoid talking about risk through documented media.
Companies often intentionally avoid discussing the risks of what they are doing through permanent media such as email. For example, this
article
gives some very shady advice on how companies can avoid liability by using "safe communication" practices to avoid the creation of incriminating "bad documents".
Often the drafters of these documents tend to believe that they are providing the company with some value to the business. For example, an engineer notices a potential liability in a design so he informs his supervisor through an email. However, the engineer's lack of legal knowledge and misuse of legal vocabulary in the communication may later implicate the company with notice of the problem when a lawsuit arises.
I personally enjoyed the use of "when" and not "if" in the excerpt.
This is a perverse consequence of how it is relatively hard for companies to be held liable for risks when it cannot be proven they knew about them, even if they did. When an incident happens and a company is sued, evidence about its role in the problem is gathered during what is known as the "
discovery
" phase of a lawsuit (emails are usually discoverable). When records showing that a company had knowledge of the problem are found in discovery, they are much more likely to be found liable.
One email can have a lot of power.
The unfortunate consequence of how discovery works is that companies strategically avoid communicating risks via documented media. But there is a silver lining. The threat of liability due to documented communications of risks can have a lot of influence over how cautious a company is. One discoverable record of a risk can be very impactful.
I like to call this the 6D effect - the Duty to Due Diligence from Discoverable Documentation of Dangers.
A few examples
Here are some notable examples of companies being held liable for damages because they ignored documented communication of risks (but there are many throughout legal history).
In
Grimshaw v. Ford Motor Company, 1981
, Ford was held liable for damages involving a fatal crash with a Ford Pinto because it was shown that leadership within the company ignored warnings about problems with the vehicle's fuel system.
In April of this year,
a large settlement was reached
after the 2017 Grenfell Tower fire in London, which killed 72 people. A big factor in the lawsuit was that the company managing the tower had
ignored
numerous fire safety warnings which were found in discovery.
Last year, the Hardwick v. 3M case
ended
. It was a class action lawsuit from 2018 about the presence of harmful "forever chemicals" (PFAS) in consumer products. The company behind these chemicals was found to have
known about risks
since the 1970s but
was knowingly negligent,
which led to a ruling against them.
Miscellaneous notes
The 6D effect can result from any discoverable communication, but it is especially powerful when the warning comes from an employee of the company itself.
If you communicate a risk, it is important to speak up and ...
Recently, I have been learning about industry norms, legal discovery proceedings, and incentive structures related to companies building risky systems. I wanted to share some findings in this post because they may be important for the frontier AI community to understand well.
TL;DR
Documented communications of risks (especially by employees) make companies much more likely to be held liable in court when bad things happen. The resulting Duty to Due Diligence from Discoverable Documentation of Dangers (the 6D effect) can make companies much more cautious if even a single email is sent to them communicating a risk.
Companies tend to avoid talking about risk through documented media.
Companies often intentionally avoid discussing the risks of what they are doing through permanent media such as email. For example, this
article
gives some very shady advice on how companies can avoid liability by using "safe communication" practices to avoid the creation of incriminating "bad documents".
Often the drafters of these documents tend to believe that they are providing the company with some value to the business. For example, an engineer notices a potential liability in a design so he informs his supervisor through an email. However, the engineer's lack of legal knowledge and misuse of legal vocabulary in the communication may later implicate the company with notice of the problem when a lawsuit arises.
I personally enjoyed the use of "when" and not "if" in the excerpt.
This is a perverse consequence of how it is relatively hard for companies to be held liable for risks when it cannot be proven they knew about them, even if they did. When an incident happens and a company is sued, evidence about its role in the problem is gathered during what is known as the "
discovery
" phase of a lawsuit (emails are usually discoverable). When records showing that a company had knowledge of the problem are found in discovery, they are much more likely to be found liable.
One email can have a lot of power.
The unfortunate consequence of how discovery works is that companies strategically avoid communicating risks via documented media. But there is a silver lining. The threat of liability due to documented communications of risks can have a lot of influence over how cautious a company is. One discoverable record of a risk can be very impactful.
I like to call this the 6D effect - the Duty to Due Diligence from Discoverable Documentation of Dangers.
A few examples
Here are some notable examples of companies being held liable for damages because they ignored documented communication of risks (but there are many throughout legal history).
In
Grimshaw v. Ford Motor Company, 1981
, Ford was held liable for damages involving a fatal crash with a Ford Pinto because it was shown that leadership within the company ignored warnings about problems with the vehicle's fuel system.
In April of this year,
a large settlement was reached
after the 2017 Grenfell Tower fire in London, which killed 72 people. A big factor in the lawsuit was that the company managing the tower had
ignored
numerous fire safety warnings which were found in discovery.
Last year, the Hardwick v. 3M case
ended
. It was a class action lawsuit from 2018 about the presence of harmful "forever chemicals" (PFAS) in consumer products. The company behind these chemicals was found to have
known about risks
since the 1970s but
was knowingly negligent,
which led to a ruling against them.
Miscellaneous notes
The 6D effect can result from any discoverable communication, but it is especially powerful when the warning comes from an employee of the company itself.
If you communicate a risk, it is important to speak up and ...