The Nonlinear Library LW - Don't sleep on Coordination Takeoffs by trevor
Jan 28, 2024
07:46
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Don't sleep on Coordination Takeoffs, published by trevor on January 28, 2024 on LessWrong.
It's important to remember that the culture we grew up in is deeply nihilistic at its core. People expect Moloch, assume Moloch as a given, even defer to Moloch. If you read enough about business and international affairs (not news articles, those don't count, not for international affairs at least, I don't know about business), and then read about dath ilan, it becomes clear that our world is ruled by Moloch cultists who nihilistically optimized for career advancement.
Humans are primates; we instinctively take important concepts and turn them into dominance/status games, including that concept itself; resulting in many people believing that important concepts do not exist at all.
So it makes sense that Moloch would be an intensely prevalent part of our civilization, even ~a century after decision theory took off and ~4 centuries after mass literacy took off.
Some of the first people to try to get together and have a really big movement to enlighten and reform the world was the Counter Culture movement starting in the 60's, which overlapped with the Vietnam Antiwar movement and the Civil Rights movement.
The Counter Culture movement failed because they were mainly a bunch of inept teens and 20-somethings; not just lacking knowledge of decision theory or economics or Sequence-level understanding of heuristics/biases, but also because they lived in a world where social psychology and thinking-about-society were still in infancy. Like the European Enlightenment and the French Revolution before them, they started out profoundly confused about the direction to aim for and the correct moves to make (see Anna Salamon's Humans are not Automatically Strategic).
The Antiwar movement permanently damaged the draft-based American military apparatus, permanently made western culture substantially more cosmopolitan than the conformist 1950s, but their ignorance and ineptitude and blunders were so immense that they shrank the Overton window on people coming together and choosing to change the world for the better.
As soon as lots of people acquired an incredibly primitive version of the understandings now held by the EA, rationalist, and AI safety communities, those people started the Counter Culture movement of the 1960s in order to raise the sanity waterline above the deranged passivity of the 1950s conformist culture. And they botched it so hard, in so many ways, that everyone now cringes at the memory; the Overton window on changing the world was fouled up, perhaps intractably. Major governments and militaries also became predisposed to nip similar movements in the bud, such as the use of AI technology to psychologically disrupt groups of highly motivated people.
Since then, there hasn't been a critical mass behind counter culture or societal reform, other than Black Lives Matter, the Women's March, Occupy Wall Street, and the Jan 6th Riots, which only got that many people due to heavily optimizing for memetic spread among the masses via excessively simple messages, and prevailing on already-popular sentiment such post-2008 anger at banking institutions, and likely only getting that far due to the emergence of the social media paradigm (which governments are incentivized to hijack).
Game theory didn't take off until the 1950s, when it was basically absorbed by the US military, just like how economics was absorbed by the contemporary equivalent of Wall Street (and remains absorbed to this day). I'm pretty sure that the entire 20th century came and went with nearly none of them spending an hour a week thinking about solving the coordination problems facing the human race, so that the world could be better for them and their children.
Even though virtually all of them would prefer to live ...
It's important to remember that the culture we grew up in is deeply nihilistic at its core. People expect Moloch, assume Moloch as a given, even defer to Moloch. If you read enough about business and international affairs (not news articles, those don't count, not for international affairs at least, I don't know about business), and then read about dath ilan, it becomes clear that our world is ruled by Moloch cultists who nihilistically optimized for career advancement.
Humans are primates; we instinctively take important concepts and turn them into dominance/status games, including that concept itself; resulting in many people believing that important concepts do not exist at all.
So it makes sense that Moloch would be an intensely prevalent part of our civilization, even ~a century after decision theory took off and ~4 centuries after mass literacy took off.
Some of the first people to try to get together and have a really big movement to enlighten and reform the world was the Counter Culture movement starting in the 60's, which overlapped with the Vietnam Antiwar movement and the Civil Rights movement.
The Counter Culture movement failed because they were mainly a bunch of inept teens and 20-somethings; not just lacking knowledge of decision theory or economics or Sequence-level understanding of heuristics/biases, but also because they lived in a world where social psychology and thinking-about-society were still in infancy. Like the European Enlightenment and the French Revolution before them, they started out profoundly confused about the direction to aim for and the correct moves to make (see Anna Salamon's Humans are not Automatically Strategic).
The Antiwar movement permanently damaged the draft-based American military apparatus, permanently made western culture substantially more cosmopolitan than the conformist 1950s, but their ignorance and ineptitude and blunders were so immense that they shrank the Overton window on people coming together and choosing to change the world for the better.
As soon as lots of people acquired an incredibly primitive version of the understandings now held by the EA, rationalist, and AI safety communities, those people started the Counter Culture movement of the 1960s in order to raise the sanity waterline above the deranged passivity of the 1950s conformist culture. And they botched it so hard, in so many ways, that everyone now cringes at the memory; the Overton window on changing the world was fouled up, perhaps intractably. Major governments and militaries also became predisposed to nip similar movements in the bud, such as the use of AI technology to psychologically disrupt groups of highly motivated people.
Since then, there hasn't been a critical mass behind counter culture or societal reform, other than Black Lives Matter, the Women's March, Occupy Wall Street, and the Jan 6th Riots, which only got that many people due to heavily optimizing for memetic spread among the masses via excessively simple messages, and prevailing on already-popular sentiment such post-2008 anger at banking institutions, and likely only getting that far due to the emergence of the social media paradigm (which governments are incentivized to hijack).
Game theory didn't take off until the 1950s, when it was basically absorbed by the US military, just like how economics was absorbed by the contemporary equivalent of Wall Street (and remains absorbed to this day). I'm pretty sure that the entire 20th century came and went with nearly none of them spending an hour a week thinking about solving the coordination problems facing the human race, so that the world could be better for them and their children.
Even though virtually all of them would prefer to live ...