The Nonlinear Library EA - Evidential Cooperation in Large Worlds: Potential Objections & FAQ by Chi
Feb 28, 2024
39:12
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evidential Cooperation in Large Worlds: Potential Objections & FAQ, published by Chi on February 28, 2024 on The Effective Altruism Forum.
What is this post?
This post is a companion piece to
recent
posts on evidential cooperation in large worlds (ECL). We've noticed that in conversations about ECL, the same few initial confusions and objections tend to be brought up. We hope that this post will be useful as the place that lists and discusses these common objections. We invite the reader to advance additional questions or objections of their own.
This FAQ does not need to be read in order. The reader is encouraged to look through the section headings and jump to those they find most interesting.
ECL seems very weird. Are you sure you haven't, like, taken a wrong turn somewhere?
We don't think so.
ECL, at its core, takes two reasonable ideas that by themselves are considered quite plausible by many - albeit not completely uncontroversial - and notices that when you combine them, you get something quite interesting and novel. Specifically, ECL combines "large world" with "noncausal decision theory." Many people believe the universe/multiverse is large, but that it might as well be small because we can only causally influence, or be influenced by, a small, finite part of it.
Meanwhile, many people think you should cooperate in a near twin prisoners' dilemma, but that this is mostly a philosophical issue because near twin prisoners' dilemmas rarely, if ever, happen in real life. Putting the two ideas together: once you consider the noncausal effects of your actions, the world being large is potentially a very big deal.[1]
Do I need to buy evidential decision theory for this to work?
There are some different ways of thinking that take into account acausal influence and explain it in different ways. These include evidential decision theory and functional decision theory, as mentioned in our "
ECL explainer" post.
Updatelessness and
superrationality are two other concepts that might get you all or part of the way to this kind of acausal cooperation.
Evidential decision theory says that what matters is whether your choice gives you evidence about what the other agent will do.
For example, if you are interacting with a near-copy, then the similarity between the two of you is evidence that the two of you make the same choice.
Functional decision theory says that what matters is whether there is a logical connection between you and the other agent's choices.
For example, if you are interacting with a copy, then the similarity between the two of you is reason to believe there is a strong logical connection.
That said, functional decision theory does not have a clear formalization, so it is not clear if and how this logical connection generalizes to dealing with merely near-copies (as opposed to full copies). Our best guess is that proponents of functional decision theory at least want the theory to recommend cooperating in the near twin prisoner's dilemma.[2]
Updatelessness strengthens the case for cooperation. This is because updatelessness arguably increases the game-theoretic symmetry of many kinds of interactions, which is helpful to get agents employing some
types of decision procedures (including evidential decision theory) to cooperate.[3]
Superrationality says that two rational thinkers considering the same problem will arrive at the same correct answer. So, what matters is common rationality.
In game theory situations like the prisoner's dilemma, knowing that the two answers, or choices, will be the same might change the answer itself (e.g., cooperate-cooperate rather than defect-defect).
ECL was in fact originally named "
multiverse-wide superrationality".
We don't take a stance in our "
ECL explainer" piece on which of these decision theories, concepts, or others we d...
What is this post?
This post is a companion piece to
recent
posts on evidential cooperation in large worlds (ECL). We've noticed that in conversations about ECL, the same few initial confusions and objections tend to be brought up. We hope that this post will be useful as the place that lists and discusses these common objections. We invite the reader to advance additional questions or objections of their own.
This FAQ does not need to be read in order. The reader is encouraged to look through the section headings and jump to those they find most interesting.
ECL seems very weird. Are you sure you haven't, like, taken a wrong turn somewhere?
We don't think so.
ECL, at its core, takes two reasonable ideas that by themselves are considered quite plausible by many - albeit not completely uncontroversial - and notices that when you combine them, you get something quite interesting and novel. Specifically, ECL combines "large world" with "noncausal decision theory." Many people believe the universe/multiverse is large, but that it might as well be small because we can only causally influence, or be influenced by, a small, finite part of it.
Meanwhile, many people think you should cooperate in a near twin prisoners' dilemma, but that this is mostly a philosophical issue because near twin prisoners' dilemmas rarely, if ever, happen in real life. Putting the two ideas together: once you consider the noncausal effects of your actions, the world being large is potentially a very big deal.[1]
Do I need to buy evidential decision theory for this to work?
There are some different ways of thinking that take into account acausal influence and explain it in different ways. These include evidential decision theory and functional decision theory, as mentioned in our "
ECL explainer" post.
Updatelessness and
superrationality are two other concepts that might get you all or part of the way to this kind of acausal cooperation.
Evidential decision theory says that what matters is whether your choice gives you evidence about what the other agent will do.
For example, if you are interacting with a near-copy, then the similarity between the two of you is evidence that the two of you make the same choice.
Functional decision theory says that what matters is whether there is a logical connection between you and the other agent's choices.
For example, if you are interacting with a copy, then the similarity between the two of you is reason to believe there is a strong logical connection.
That said, functional decision theory does not have a clear formalization, so it is not clear if and how this logical connection generalizes to dealing with merely near-copies (as opposed to full copies). Our best guess is that proponents of functional decision theory at least want the theory to recommend cooperating in the near twin prisoner's dilemma.[2]
Updatelessness strengthens the case for cooperation. This is because updatelessness arguably increases the game-theoretic symmetry of many kinds of interactions, which is helpful to get agents employing some
types of decision procedures (including evidential decision theory) to cooperate.[3]
Superrationality says that two rational thinkers considering the same problem will arrive at the same correct answer. So, what matters is common rationality.
In game theory situations like the prisoner's dilemma, knowing that the two answers, or choices, will be the same might change the answer itself (e.g., cooperate-cooperate rather than defect-defect).
ECL was in fact originally named "
multiverse-wide superrationality".
We don't take a stance in our "
ECL explainer" piece on which of these decision theories, concepts, or others we d...