The Nonlinear Library LW - The Offense-Defense Balance Rarely Changes by Maxwell Tabarrok
Dec 9, 2023
05:31
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Offense-Defense Balance Rarely Changes, published by Maxwell Tabarrok on December 9, 2023 on LessWrong.
You've probably seen
several conversations on X go something
like this:
Michael Doomer : Advanced AI can help anyone make bioweapons
If this technology spreads it will only take one crazy person to destroy the world!
Edward Acc : I can just ask my AI to make a vaccine
Yann LeCun:
My good AI will take down your rogue AI
The disagreement here hinges on whether a technology will enable offense (bioweapons) more than defense (vaccines). Predictions of the "offense-defense balance" of future technologies, especially AI, are
central in debates about techno-optimism and existential risk.
Most of these predictions rely on intuitions about how technologies like cheap biotech, drones, and digital agents would affect the ease of attacking or protecting resources. It is hard to imagine a world with AI agents searching for software vulnerabilities and autonomous drones attacking military targets without imagining a massive shift the offense defense balance.
But there is little historical evidence for large changes in the offense defense balance, even in response to technological revolutions.
Consider cybersecurity. Moore's law has taken us through
seven orders of magnitude reduction in the cost of compute since the 70s. There were massive changes in the form and economic uses for computer technology along with the increase in raw compute power: Encryption, the internet, e-commerce, social media and smartphones.
The usual offense-defense balance story predicts that big changes to technologies like this should have big effects on the offense defense balance. If you had told people in the 1970s that in 2020 terrorist groups and lone psychopaths could access more computing power than IBM had ever produced at the time from their pocket, what would they have predicted about the offense defense balance of cybersecurity?
Contrary to their likely prediction, the offense-defense balance in cybersecurity seems stable. Cyberattacks have not been snuffed out but neither have they taken over the world. All major nations have defensive and offensive cybersecurity teams but no one has gained a decisive advantage. Computers still sometimes get viruses or ransomware, but they haven't grown to endanger a large percent of the GDP of the internet. The US military budget for cybersecurity has
increased by about 4% a year every year from 1980-2020, which is faster than GDP growth, but in line with GDP growth plus the growing fraction of GDP that's on the internet.
This stability through several previous technological revolutions raises the burden of proof for why the offense defense balance of cybersecurity should be expected to change radically after the next one.
The stability of the offense-defense balance isn't specific to cybersecurity. The graph below shows the per capita rate of death in war from 1400 to 2013. This graph contains all of humanity's major technological revolutions. There is lots of variance from year to year but almost zero long run trend.
Does anyone have a theory of the offense-defense balance which can explain why the per-capita deaths from war should be about the same in 1640 when people are fighting with swords and horses as in 1940 when they are fighting with airstrikes and tanks?
It is very difficult to explain the variation in this graph with variation in technology. Per-capita deaths in conflict is noisy and cyclic while the progress in technology is relatively smooth and monotonic.
No previous technology has changed the frequency or cost of conflict enough to move this metric far beyond the maximum and minimum range that was already set 1400-1650. Again the burden of proof is raised for why we should expect AI to be different.
The cost to sequence a human genome h...
You've probably seen
several conversations on X go something
like this:
Michael Doomer : Advanced AI can help anyone make bioweapons
If this technology spreads it will only take one crazy person to destroy the world!
Edward Acc : I can just ask my AI to make a vaccine
Yann LeCun:
My good AI will take down your rogue AI
The disagreement here hinges on whether a technology will enable offense (bioweapons) more than defense (vaccines). Predictions of the "offense-defense balance" of future technologies, especially AI, are
central in debates about techno-optimism and existential risk.
Most of these predictions rely on intuitions about how technologies like cheap biotech, drones, and digital agents would affect the ease of attacking or protecting resources. It is hard to imagine a world with AI agents searching for software vulnerabilities and autonomous drones attacking military targets without imagining a massive shift the offense defense balance.
But there is little historical evidence for large changes in the offense defense balance, even in response to technological revolutions.
Consider cybersecurity. Moore's law has taken us through
seven orders of magnitude reduction in the cost of compute since the 70s. There were massive changes in the form and economic uses for computer technology along with the increase in raw compute power: Encryption, the internet, e-commerce, social media and smartphones.
The usual offense-defense balance story predicts that big changes to technologies like this should have big effects on the offense defense balance. If you had told people in the 1970s that in 2020 terrorist groups and lone psychopaths could access more computing power than IBM had ever produced at the time from their pocket, what would they have predicted about the offense defense balance of cybersecurity?
Contrary to their likely prediction, the offense-defense balance in cybersecurity seems stable. Cyberattacks have not been snuffed out but neither have they taken over the world. All major nations have defensive and offensive cybersecurity teams but no one has gained a decisive advantage. Computers still sometimes get viruses or ransomware, but they haven't grown to endanger a large percent of the GDP of the internet. The US military budget for cybersecurity has
increased by about 4% a year every year from 1980-2020, which is faster than GDP growth, but in line with GDP growth plus the growing fraction of GDP that's on the internet.
This stability through several previous technological revolutions raises the burden of proof for why the offense defense balance of cybersecurity should be expected to change radically after the next one.
The stability of the offense-defense balance isn't specific to cybersecurity. The graph below shows the per capita rate of death in war from 1400 to 2013. This graph contains all of humanity's major technological revolutions. There is lots of variance from year to year but almost zero long run trend.
Does anyone have a theory of the offense-defense balance which can explain why the per-capita deaths from war should be about the same in 1640 when people are fighting with swords and horses as in 1940 when they are fighting with airstrikes and tanks?
It is very difficult to explain the variation in this graph with variation in technology. Per-capita deaths in conflict is noisy and cyclic while the progress in technology is relatively smooth and monotonic.
No previous technology has changed the frequency or cost of conflict enough to move this metric far beyond the maximum and minimum range that was already set 1400-1650. Again the burden of proof is raised for why we should expect AI to be different.
The cost to sequence a human genome h...