The Nonlinear Library LW - We are already in a persuasion-transformed world and must take precautions by trevor
Nov 4, 2023
11:02
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We are already in a persuasion-transformed world and must take precautions, published by trevor on November 4, 2023 on LessWrong.
"In times of change, learners inherit the earth, while the learned find themselves beautifully equipped to deal with a world that no longer exists"
Summary:
We're already in the timeline where the research and manipulation of the human thought process is widespread; SOTA psychological research systems require massive amounts of human behavior data, which in turn requires massive numbers of unsuspecting test subjects (users) in order to automate the process of analyzing and exploiting human targets. This therefore must happen covertly, and both the US and China have a strong track record of doing things like this. This outcome is a strong attractor state since anyone with enough data can do it, and it naturally follows that powerful organizations would deny others access e.g. via data poisoning. Most people are already being persuaded that this is harmless, even though it is obviously ludicrously dangerous. Therefore, we are probably already in a hazardously transformative world and must take
standard precautions
immediately.
This should not distract people from AI safety. This is valuable because the AI safety community must survive. This problem connects to the AI safety community in the following way:
State survival and war power
==>
already depends on information warfare capabilities.
Information warfare capabilities
==>
already depends on SOTA psychological research systems.
SOTA psychological research systems
==>
already improves and scales mainly from AI capabilities research, diminishing returns on everything else.
[1]
AI capabilities research
==>
already under siege from the AI safety community.
Therefore, the reason why this might be such a big concern is:
State survival and war power
==>
their toes potentially already being stepped on by the AI safety community?
Although it's also important to note that
people with access to SOTA psychological research systems are probably super good at intimidation and bluffing
, it's also the case that the AI safety community needs to get a better handle on the situation if we are in the bad timeline; and
the math indicates that we are already well past that point
.
The Fundamental Problem
If there were intelligent aliens, made of bundles of tentacles or crystals or plants that think incredibly slowly, their minds would also have discoverable exploits/zero days, because any mind that evolved naturally would probably be like the human brain, a
kludge of spaghetti code
that is operating outside of its intended environment.
They would probably not even begin to scratch the surface of finding and labeling those exploits, until, like human civilization today, they began surrounding thousands or millions of their kind with sensors that could record behavior several hours a day and find
webs of correlations
.
In the case of humans, the use of social media as a controlled environment for automated AI-powered experimentation appears to be what created that critical mass of human behavior data. Current 2020s capabilities for psychological research and manipulation vastly exceed the 20th century academic psychology paradigm.
The 20th century academic psychology paradigm still dominates our cultural impression of what it means to research the human mind; but when the effectiveness of psychological research and manipulation starts increasing by an order of magnitude every 4 years, it becomes time to stop mentally living in a world that was stabilized by the fact that manipulation attempts generally failed.
The capabilities of social media to steer human outcomes are not advancing in isolation, they are parallel to a broad acceleration in the understanding and exploitation of the human mind, which itself
is ...
"In times of change, learners inherit the earth, while the learned find themselves beautifully equipped to deal with a world that no longer exists"
Summary:
We're already in the timeline where the research and manipulation of the human thought process is widespread; SOTA psychological research systems require massive amounts of human behavior data, which in turn requires massive numbers of unsuspecting test subjects (users) in order to automate the process of analyzing and exploiting human targets. This therefore must happen covertly, and both the US and China have a strong track record of doing things like this. This outcome is a strong attractor state since anyone with enough data can do it, and it naturally follows that powerful organizations would deny others access e.g. via data poisoning. Most people are already being persuaded that this is harmless, even though it is obviously ludicrously dangerous. Therefore, we are probably already in a hazardously transformative world and must take
standard precautions
immediately.
This should not distract people from AI safety. This is valuable because the AI safety community must survive. This problem connects to the AI safety community in the following way:
State survival and war power
==>
already depends on information warfare capabilities.
Information warfare capabilities
==>
already depends on SOTA psychological research systems.
SOTA psychological research systems
==>
already improves and scales mainly from AI capabilities research, diminishing returns on everything else.
[1]
AI capabilities research
==>
already under siege from the AI safety community.
Therefore, the reason why this might be such a big concern is:
State survival and war power
==>
their toes potentially already being stepped on by the AI safety community?
Although it's also important to note that
people with access to SOTA psychological research systems are probably super good at intimidation and bluffing
, it's also the case that the AI safety community needs to get a better handle on the situation if we are in the bad timeline; and
the math indicates that we are already well past that point
.
The Fundamental Problem
If there were intelligent aliens, made of bundles of tentacles or crystals or plants that think incredibly slowly, their minds would also have discoverable exploits/zero days, because any mind that evolved naturally would probably be like the human brain, a
kludge of spaghetti code
that is operating outside of its intended environment.
They would probably not even begin to scratch the surface of finding and labeling those exploits, until, like human civilization today, they began surrounding thousands or millions of their kind with sensors that could record behavior several hours a day and find
webs of correlations
.
In the case of humans, the use of social media as a controlled environment for automated AI-powered experimentation appears to be what created that critical mass of human behavior data. Current 2020s capabilities for psychological research and manipulation vastly exceed the 20th century academic psychology paradigm.
The 20th century academic psychology paradigm still dominates our cultural impression of what it means to research the human mind; but when the effectiveness of psychological research and manipulation starts increasing by an order of magnitude every 4 years, it becomes time to stop mentally living in a world that was stabilized by the fact that manipulation attempts generally failed.
The capabilities of social media to steer human outcomes are not advancing in isolation, they are parallel to a broad acceleration in the understanding and exploitation of the human mind, which itself
is ...