The Nonlinear Library EA - Cooperating with aliens and (distant) AGIs: An ECL explainer by Chi
Feb 25, 2024
45:42
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cooperating with aliens and (distant) AGIs: An ECL explainer, published by Chi on February 25, 2024 on The Effective Altruism Forum.
Summary
Evidential cooperation in large worlds (ECL) is a proposed way of reaping gains - that is, getting more of what we value instantiated - through cooperating with agents across the universe/multiverse. Such cooperation does not involve physical, or causal, interaction. ECL is potentially a
crucial consideration, because we may be able to do more good this way compared to the "standard" (i.e., causal) way of optimizing for our values.
The core idea of ECL can be summarized as:
According to non-causal decision theories, my decisions relevantly "influence" what others who are similar to me do, even if they never observe my behavior (or the causal consequences of my behavior). (
More.)
In particular, if I behave cooperatively towards other value systems, then other agents across the multiverse are more likely to do the same. Hence, at least some fraction of agents can be (acausally) influenced into behaving cooperatively towards my value system. This gives me reason to be cooperative with other value systems. (
More.)
Meanwhile, there are many agents in the universe/multiverse. (
More.) Cooperating with them would unlock a great deal of value due to gains from trade. (
More.) For example, if I care about the well-being of sentient beings everywhere, I can "influence" how faraway agents treat sentient beings in their part of the universe/multiverse.
Introduction
The
observable universe is large. Nonetheless, the full extent of the universe is likely much larger, perhaps infinitely so. This means that most of what's out there is not causally connected to us. Even if we set out now from planet Earth, traveling at the speed of light, we would never reach most locations in the universe. One might assume that this means most of the universe is not our concern.
In this post, we explain why all of the universe - and all of the multiverse, if it exists - may in fact concern us if we take something called evidential cooperation in large worlds (ECL) into account.[1] Given how high the stakes are, on account of how much universe/multiverse might be out there beyond our causal reach, ECL is potentially very important. In our view, ECL is a crucial consideration for the effective altruist project of doing the most good.
In the next section of this post, we explain the theory underlying ECL. Building on that, we outline why we might be able to do ECL ourselves and how it allows us to do more good. We conclude by giving some information on how you can get involved. We will also publish
an FAQ in the near future, which will address some possible objections to ECL.
The twin prisoners' dilemma
Exact copies
Suppose you are in a
prisoner's dilemma with an exact copy of yourself:
You have a choice: You can either press the defect button, which increases your own payoff by $1, or you can press the cooperate button, which increases your copy's payoff by $2.
Your copy faces the same choice (i.e., the situation is symmetric).
Both of you cannot see the other's choice until after you have made your own choice. You and your copy will never interact with each other after this, and nobody else will ever observe what choices you both made.
You only care about your own payoff, not the payoff of your copy.
This situation can be represented with the following
payoff matrix:
Looking at the matrix, you can see that regardless of whether your copy cooperates or defects, you are better off if you defect. "Defect" is the
strictly dominant strategy. Therefore, under standard notions of rational decision making, you should defect. In particular,
causal decision theory - read in the standard way - says to defect (
Lewis, 1979).
However, the other player is an exact copy of y...
Summary
Evidential cooperation in large worlds (ECL) is a proposed way of reaping gains - that is, getting more of what we value instantiated - through cooperating with agents across the universe/multiverse. Such cooperation does not involve physical, or causal, interaction. ECL is potentially a
crucial consideration, because we may be able to do more good this way compared to the "standard" (i.e., causal) way of optimizing for our values.
The core idea of ECL can be summarized as:
According to non-causal decision theories, my decisions relevantly "influence" what others who are similar to me do, even if they never observe my behavior (or the causal consequences of my behavior). (
More.)
In particular, if I behave cooperatively towards other value systems, then other agents across the multiverse are more likely to do the same. Hence, at least some fraction of agents can be (acausally) influenced into behaving cooperatively towards my value system. This gives me reason to be cooperative with other value systems. (
More.)
Meanwhile, there are many agents in the universe/multiverse. (
More.) Cooperating with them would unlock a great deal of value due to gains from trade. (
More.) For example, if I care about the well-being of sentient beings everywhere, I can "influence" how faraway agents treat sentient beings in their part of the universe/multiverse.
Introduction
The
observable universe is large. Nonetheless, the full extent of the universe is likely much larger, perhaps infinitely so. This means that most of what's out there is not causally connected to us. Even if we set out now from planet Earth, traveling at the speed of light, we would never reach most locations in the universe. One might assume that this means most of the universe is not our concern.
In this post, we explain why all of the universe - and all of the multiverse, if it exists - may in fact concern us if we take something called evidential cooperation in large worlds (ECL) into account.[1] Given how high the stakes are, on account of how much universe/multiverse might be out there beyond our causal reach, ECL is potentially very important. In our view, ECL is a crucial consideration for the effective altruist project of doing the most good.
In the next section of this post, we explain the theory underlying ECL. Building on that, we outline why we might be able to do ECL ourselves and how it allows us to do more good. We conclude by giving some information on how you can get involved. We will also publish
an FAQ in the near future, which will address some possible objections to ECL.
The twin prisoners' dilemma
Exact copies
Suppose you are in a
prisoner's dilemma with an exact copy of yourself:
You have a choice: You can either press the defect button, which increases your own payoff by $1, or you can press the cooperate button, which increases your copy's payoff by $2.
Your copy faces the same choice (i.e., the situation is symmetric).
Both of you cannot see the other's choice until after you have made your own choice. You and your copy will never interact with each other after this, and nobody else will ever observe what choices you both made.
You only care about your own payoff, not the payoff of your copy.
This situation can be represented with the following
payoff matrix:
Looking at the matrix, you can see that regardless of whether your copy cooperates or defects, you are better off if you defect. "Defect" is the
strictly dominant strategy. Therefore, under standard notions of rational decision making, you should defect. In particular,
causal decision theory - read in the standard way - says to defect (
Lewis, 1979).
However, the other player is an exact copy of y...