The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Scenario Forecasting Workshop: Materials and Learnings, published by elifland on March 8, 2024 on The AI Alignment Forum. Disclaimer: While some participants and organizers of this exercise work in industry, no proprietary info was used to inform these scenarios, and they represent the views of their individual authors alone. Overview In the vein of What 2026 Looks Like and AI Timelines discussion, we recently hosted a scenario forecasting workshop. Participants first wrote a 5-stage scenario forecasting what will happen between now and ASI. Then, they reviewed, discussed, and revised scenarios in groups of 3. The discussion was guided by forecasts like "If I were to observe this person's scenario through stage X, what would my ASI timelines median be?". Instructions for running the workshop including notes on what we would do differently are available here. We've put 6 shared scenarios from our workshop in a publicly viewable folder here. Motivation Writing scenarios may help to: Clarify views, e.g. by realizing an abstract view is hard to concretize, or realizing that two views you hold don't seem very compatible. Surface new considerations, e.g. realizing a subquestion is more important than you thought, or that an actor might behave in a way you hadn't considered. Communicate views to others, e.g. clarifying what you mean by "AGI", "slow takeoff", or the singularity. Register qualitative forecasts, which can then be compared against reality. This has advantages and disadvantages vs. more resolvable forecasts (though scenarios can include some resolvable forecasts as well!). Running the workshop Materials and instructions for running the workshop including notes on what we would do differently are available here. The schedule for the workshop looked like: Session 1 involved writing a 5-staged scenario forecasting what will happen between now and ASI. Session 2 involved reviewing, discussing, and revising scenarios in groups of 3. The discussion was guided by forecasts like "If I were to observe this person's scenario through stage X, what would my ASI timelines median be?". There were analogous questions for p(disempowerment) and p(good future). Session 3 was freeform discussion and revision within groups, then there was a brief session for feedback. Workshop outputs and learnings 6 people (3 anonymous, 3 named) have agreed to share their scenarios. We've put them in a publicly viewable folder here. We received overall positive feedback, with nearly all 23 people who filled out the feedback survey saying it was a good use of their time. In general, people found the writing portion more valuable than the discussion. We've included some ideas on how to improve future similar workshops based on this feedback and a few other pieces of feedback in our instructions for organizers. It's possible that a workshop that is much more focused on the writing relative to the discussion would be more valuable. Speaking for myself (as Eli), I think it was mostly valuable as a forcing function to get people to do an activity they had wanted to do anyway. And scenario writing seems like a good thing for people to spend marginal time on (especially if they find it fun/energizing). It seems worthwhile to experiment with the format (in the ways we suggest above, or other ways people are excited about). It feels like there might be something nearby that is substantially more valuable than our initial pilot. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Forecasting future gains due to post-training enhancements, published by elifland on March 8, 2024 on The AI Alignment Forum. This work has been done in the context of SaferAI's work on risk assessment. Equal contribution by Eli and Joel. I'm sharing this writeup in the form of a Google Doc and reproducing the summary below. Disclaimer: this writeup is context for upcoming experiments, not complete work. As such it contains a lot of (not always well-justified) guess-work and untidy conceptual choices. We are publishing now despite this to get feedback. If you are interested in this work - perhaps as a future collaborator or funder, or because this work could provide helpful input into e.g. risk assessments or RSPs - please get in touch with us at joel@qallys.com and/or simeon@safer-ai.org. Summary A recent report documented how the performance of AI models can be improved after training, via post-training enhancements (PTEs) such as external tools, scaffolding, and fine-tuning. The gain from a PTE is measured in compute-equivalent gains (CEG): the multiplier on training compute required to achieve equivalent performance to a model combined with a PTE. We are interested in understanding the contribution that PTEs make to AI system capabilities over time. This question in turn is motivated by SaferAI's work on quantitative risk assessments of frontier models. In particular, any risk assessment of open-sourcing models or of having closed-source models stolen or leaked should take into account system capabilities, which we might expect to increase over time as PTEs are added to the system built on top of a given base model. We extend a recent analysis of PTEs in order to understand the trend in CEG over time. There are serious limitations in our preliminary analysis, including: problems with the CEG metric, many uninformed parameter estimates, and reliance on an ill-defined "average task". High-priority future work includes running experiments to get more evidence on important uncertainties for our forecasts of capability gains due to PTEs. In particular, we think it will be important to understand how well different PTEs combine, as well as to directly study performance on benchmarks relevant to dangerous capabilities rather than relying on the CEG and average task abstractions. In this write-up, we will: Outline our methodology. ( More.) Present CEG estimates for various PTEs. ( More.) Aggregate total CEG, using subjective estimates of 'composability.' ( More.) Note limitations of our analysis and important future work. ( More.) Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Woods' new preprint on object permanence, published by Steven Byrnes on March 8, 2024 on LessWrong. Quick poorly-researched post, probably only of interest to neuroscientists. The experiment Justin Wood at University of Indiana has, over many years with great effort, developed a system for raising baby chicks such that all the light hitting their retina is experimentally controlled right from when they're an embryo - the chicks are incubated and hatched in darkness, then moved to a room with video screens, head-tracking and so on. For a much better description of how this works and how he got into this line of work, check out his recent appearance on the Brain Inspired podcast. He and collaborators posted a new paper last week: "Object permanence in newborn chicks is robust against opposing evidence" by Wood, Ullman, Wood, Spelke, and Wood. I just read it today. It's really cool! In their paper, they are using the system above to study "object permanence", the idea that things don't disappear when they go out of sight behind an occluder. The headline result is that baby chicks continue to act as if object permanence is true, even if they have seen thousands of examples where it is false and zero where it is true over the course of their short lives. They describe two main experiments. Experiment 1 is the warmup, and Experiment 2 is the headline result I just mentioned. In experiment 1, the chicks are raised in a VR visual world where they never see anything occlude anything, ever. They only see one virtual object move around an otherwise-empty virtual room. The chicks of course imprint on the object. This phase lasts 4 days. Then we move into the test phase. The test initializes when the chick moves towards the virtual object, which starts in the center of the room. Two virtual opaque screens appear on the sides of the room. In the easier variant of the test, the object moves behind one of the screens, and then nothing else happens for a few minutes. The experimenters measure which screen the chick looks at more. The result: all 8 chicks looked more-than-chance at the screen that the virtual object would be behind, than at the other screen, at least for the first 30 seconds or so after the object disappeared from view. In the harder variant, one of the screens moves to the object, occludes the object, then moves back to its starting point. Again, the experiments measure which screen the chick looks at more. Here, 7 of the 8 chicks looked more-than-chance towards the screen that the virtual object would be behind, at least for 15ish seconds. Moving on to experiment 2, the test phase was the same as the easier variant above - the object moved to behind one of the two opaque virtual screens on the sides. But the preceding 4-day training phase was different for these chicks: instead of never seeing any occlusion events, they witnessed thousands of occlusion events, where the object would go behind a virtual opaque screen, and then after a variable amount of time (0-20 seconds), the screens would lower to reveal that the object was where we might expect (for the "natural world" chicks), or had magically teleported to behind the "wrong" screen (the "unnatural world" chicks). (There was no randomization - each chick lived its whole training-phase in either the natural or unnatural world.) Remarkably, all four chicks in the "natural world" and all four chicks in the "unnatural world" spent more time looking at the screen that the object had disappeared behind, rather than the other one, more than chance, at least for the first 15-30 seconds. In fact, remarkably, there was no difference between the natural-world and unnatural-world chicks! How do we make sense of these results? It's always worth asking: maybe the experiment is garbage? I'm far from an expert, but the methodol...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #54: Clauding Along, published by Zvi on March 8, 2024 on LessWrong. The big news this week was of course the release of Claude 3.0 Opus, likely in some ways the best available model right now. Anthropic now has a highly impressive model, impressive enough that it seems as if it breaks at least the spirit of their past commitments on how far they will push the frontier. We will learn more about its ultimate full capabilities over time. We also got quite the conversation about big questions of one's role in events, which I immortalized as Read the Roon. Since publication Roon has responded, which I have edited into the post along with some additional notes. That still leaves plenty of fun for the full roundup. We have spies. We have accusations of covert racism. We have Elon Musk suing OpenAI. We have a new summary of simulator theory. We have NIST, tasked with AI regulation, literally struggling to keep a roof over their head. And more. Table of Contents Introduction. Table of Contents. Language Models Offer Mundane Utility. Predict the future. Language Models Don't Offer Mundane Utility. Provide basic info. LLMs: How Do They Work? Emmett Shear rederives simulators, summarizes. Copyright Confrontation. China finds a copyright violation. Curious. Oh Elon. He sues OpenAI to… force it to change its name? Kind of, yeah. DNA Is All You Need. Was I not sufficiently impressed with Evo last week? GPT-4 Real This Time. A question of intelligence. Fun With Image Generation. Be careful not to have too much fun. Deepfaketown and Botpocalypse Soon. This will not give you a hand. They Took Our Jobs. They gave us a few back. For now, at least. Get Involved. Davidad will have direct report, it could be you. Introducing. An AI-based RPG will never work, until one does. In Other AI News. The fallout continues, also other stuff. More on Self-Awareness. Not the main thing to worry about. Racism Remains a Problem for LLMs. Covert is a generous word for this. Project Maven. Yes, we are putting the AIs in charge of weapon targeting. Quiet Speculations. Claimed portents of various forms of doom. The Quest for Sane Regulation. NIST might need a little help. The Week in Audio. Sergey Brin Q&A. Rhetorical Innovation. It is not progress. We still keep trying. Another Open Letter. Also not really progress. We still keep trying. Aligning a Smarter Than Human Intelligence is Difficult. Recent roundup. Security is Also Difficult. This too is not so covert, it turns out. The Lighter Side. It's me, would you like a fries with that? Language Models Offer Mundane Utility Forecast almost as well, or sometimes better, than the wisdom of crowds using GPT-4? Paper says yes. Prompt they used is here. This does require an intensive process. First, we generate search queries that are used to invoke news APIs to retrieve historical articles. We initially implement a straightforward query expansion prompt (Figure 12a), instructing the model to create queries based on the question and its background. However, we find that this overlooks sub-considerations that often contribute to accurate forecasting. To achieve broader coverage, we prompt the model to decompose the forecasting question into sub-questions and use each to generate a search query (Min et al., 2019); see Figure 12b for the prompt. For instance, when forecasting election outcomes, the first approach searches directly for polling data, while the latter creates sub-questions that cover campaign finances, economic indicators, and geopolitical events. We combine both approaches for comprehensive coverage. Next, the system retrieves articles from news APIs using the LM-generated search queries. We evaluate 5 APIs on the relevance of the articles retrieved and select NewsCatcher1 and Google News (Section E.2). Our initial retrieval provides wide covera...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: MATS AI Safety Strategy Curriculum, published by Ryan Kidd on March 8, 2024 on LessWrong. As part of the MATS Winter 2023-24 Program, scholars were invited to take part in a series of weekly discussion groups on AI safety strategy. Each strategy discussion focused on a specific crux we deemed relevant to prioritizing AI safety interventions and was accompanied by a reading list and suggested discussion questions. The discussion groups were faciliated by several MATS alumni and other AI safety community members and generally ran for 1-1.5 h. As assessed by our alumni reviewers, scholars in our Summer 2023 Program were much better at writing concrete plans for their research than they were at explaining their research's theory of change. We think it is generally important for researchers, even those early in their career, to critically evaluate the impact of their work, to: Choose high-impact research directions and career pathways; Conduct adequate risk analyses to mitigate unnecessary safety hazards and avoid research with a poor safety-capabilities advancement ratio; Discover blindspots and biases in their research strategy. We expect that the majority of improvements to the above areas occur through repeated practice, ideally with high-quality feedback from a mentor or research peers. However, we also think that engaging with some core literature and discussing with peers is beneficial. This is our attempt to create a list of core literature for AI safety strategy appropriate for the average MATS scholar, who should have completed the AISF Alignment Course. We are not confident that the reading lists and discussion questions below are the best possible version of this project, but we thought they were worth publishing anyways. MATS welcomes feedback and suggestions for improvement. Week 1: How will AGI arise? What is AGI? Karnofsky - Forecasting Transformative AI, Part 1: What Kind of AI? (13 min) Metaculus - When will the first general AI system be devised, tested, and publicly announced? (read Resolution Criteria) (5 min) How large will models need to be and when will they be that large? Alexander - Biological Anchors: The Trick that Might or Might Not Work (read Parts I-II) (27 min) Optional: Davidson - What a compute-centric framework says about AI takeoff speeds (20 min) Optional: Habryka et al. - AI Timelines (dialogue between Ajeya Cotra, Daniel Kokotajlo, and Ege Erdil) (61 min) Optional: Halperin, Chow, Mazlish - AGI and the EMH: markets are not expecting aligned or unaligned AI in the next 30 years (31 min) How far can current architectures scale? Patel - Will Scaling Work? (16 min) Epoch - AI Trends (5 min) Optional: Nostalgebraist - Chinchilla's Wild Implications (13 min) Optional: Porby - Why I think strong general AI is coming soon (40 min) What observations might make us update? Ngo - Clarifying and predicting AGI (5 min) Optional: Berglund et al. - Taken out of context: On measuring situational awareness in LLMs (33 min) Optional: Cremer, Whittlestone - Artificial Canaries: Early Warning Signs for Anticipatory and Democratic Governance of AI (34 min) Suggested discussion questions If you look at any of the outside view models linked in "Biological Anchors: The Trick that Might or Might Not Work" (e.g., Ajeya Cotra's and Tom Davidson's models), which of their quantitative estimates do you agree or disagree with? Do your disagreements make your timelines longer or shorter? Do you disagree with the models used to forecast AGI? That is, rather than disagree with their estimates of particular variables, do you disagree with any more fundamental assumptions of the model? How does that change your timelines, if at all? If you had to make a probabilistic model to forecast AGI, what quantitative variables would you use and what fundamental assumptions would ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Simple Kelly betting in prediction markets, published by jessicata on March 7, 2024 on LessWrong. Kelly betting is a strategy for gambling, maximizing one's log(money) every round, by betting a fixed fraction of one's income. I will define Kelly betting a certain class of discrete prediction markets, give a simple Kelly betting rule for these prediction markets, and show it equivalent to the original Kelly formula in a two-outcome case. A prediction market consists of a finite set of outcomes O, and a probability measure Q(O) on these outcomes. Participants may buy, for some outcome o, a contract that pays out $1 if o comes true, for a price of $Q(o). This assumes no transaction fees. Suppose you have m money. You are going to spend all your money on these contracts, with R being a probability measure over O, and R(o) being the portion of money you spend on each type of contract. Note that you can buy some of each contract as an equivalent to holding on to money (e.g. to "hold on" to $2, buy 2 copies of each contract o, costing $2 in total; these contracts combined will always pay out $2). This means it's fine to assume that spending all your money on contracts doesn't compromise optimality. If your subjective probabilities of the outcomes are defined by a probability measure P(O), what is the optimal R(O) that maximizes your log-money at the end of this round? Your money conditional on outcome o is mR(o)/Q(o), since you are spending mR(o) on contracts costing Q(o) each. Therefore your expected log-money is: f(R):=oOP(o)logmR(o)Q(o)=oOP(o)(logm+logR(o)logQ(o)) Note that the log m and log Q(o) terms do not depend on R. We can therefore ignore these terms when taking the partial derivatives with respect to each R(o): f(R)R(o)=(P(o)logR(o))R(o)=P(o)R(o) If any of these partial derivatives are greater than any other, then expected log-money can be increased by moving a small amount of money from the outcome with the lower partial derivative to the one with the higher partial derivative (since f is continuous). Therefore, at the maximum of f, these partial derivatives all equal some constant c, i.e., P(o)/R(o)=c for some c. (Formally proving this might require some additional work, using the fact that f is concave and R(o) has to be positive whenever P(o) is positive; I'll omit this for brevity.) Equivalently, R(o)=P(o)/c. But this must imply c = 1, since R and P are both probability measures; any other c value would result in R not summing to 1. This implies R = P. What this means is that the optimal Kelly betting strategy involves spending a P(o) portion of your money on contracts paying out conditional on each outcome o. Interestingly, this is entirely independent of Q. This can also be seen by noticing that Q only contributes to additive terms in f that do not depend on R, such that the gradient does not depend on Q. Is this equivalent to the original Kelly rule in a two-outcome case? This rule is given by: f=p1pb where f* is the optimal portion of your money to bet, p is the probability of a win, and b is the ratio between how much is gained on a win versus how much is lost on a loss (e.g. on a triple-or-nothing coin toss, b = 2, because twice as much is gained on a win than is lost on a loss). We can set O = {w, l} (w is win, l is loss) and determine Q as a function of b. Specifically, we set Q(w)=1b+1 Q(l)=11b+1=bb+1 These are the implied house odds for b. If you spend x money on contracts paying out conditional on w, these contracts pay out x(b+1), corresponding to a net gain of xb money, whereas if you lose you simply lose x money; this therefore adequately translates b to a prediction market. Our rule says to spend a P(w) = p portion of your money on w contracts, and a 1-p portion of your money on l contracts. Suppose your starting money is m. If you win, your e...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Mud and Despair (Part 4 of "The Sense Of Physical Necessity"), published by LoganStrohl on March 7, 2024 on LessWrong. This is the fourth post in a sequence that demonstrates a complete naturalist study, specifically a study of query hugging (sort of), as described in The Nuts and Bolts of Naturalism. For context on this sequence, see the intro post. "Mud and Despair" is not officially one of the phases of naturalism. Unofficially, though, it's the phase that often happens somewhere between "Getting Your Eyes On" and "Collection". When I look back at my notes from this part of my study (roughly mid September), I am somewhat bewildered. From my current perspective, it seems as though things were exactly on track. I was making excellent progress, focusing ever more closely on the precise experiences that can lead to mastery of the skills that underlie "hug the query". My study was really taking off. And yet, I just felt so lost. I wasn't convinced I was studying anything real, anything that actually existed. I thought that perhaps I had "made it all up", and now the sham was falling apart in my hands. And so, on September 25th, I gave up. "I should study something else right now," claims my log, "and perhaps come back to this after I've remembered how it's supposed to go." A year previously, in " Getting Your Eyes On", I predicted this exact experience. I wrote about it after watching others go through the very same thing, after watching myself go through this over and over again. It's very common, in this stage, to feel a lot of doubt and confusion about what you're trying to study. (...) People sometimes respond to this kind of deep confusion with despair. They don't like feeling more lost than when they started. But in fact, it is usually an excellent sign to feel deeply confused at this point, and here is why. Naturalism is especially likely to be the right approach when you're not exactly wrong about the truth value of some proposition, so much as not even wrong. It's especially useful when you are thinking about things from the wrong direction, asking the wrong questions, using concepts that do not or cannot match the territory. When you're beginning from a place of not even wrong, you will likely find, in your first moments of direct observation, that you cannot make sense of what you are seeing. Why? Because the sense you are accustomed to making is not the sense that the actual world makes. When you look directly for the first time and do not understand what you see, it means that you may well be actually looking instead of just making things up. In this phase, things that seemed obvious and straightforward before often become perplexing. The most useful responses to this are curiosity and patience. If you stick it out, if you just keep observing through the doubt and confusion, you will begin to form new concepts, and this time they'll develop through intimate contact with the territory. Clarity may come later in the procedure, but things may have to get very muddy first. Surely it's not impossible that feeling lost and confused can mean that your project really is hopeless and you should give up, right? No, it's not impossible. It's just that those signals are not at all reliable indicators. Due to the concept-dissolving nature of naturalism, indications that it's time to abandon the project are not "confusion", "frustration", or "despair." All of these tend to be good signs in context, and your odds of eventual success depend a lot on your tolerance for these feelings. If you're wondering whether to give up (temporarily or for good), I recommend looking instead for "not caring anymore", "having new priorities", or "having underestimated the scope of your project, and considering the value incommensurate with the true scope". I've experienced all of these at...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evidential Correlations are Subjective, and it might be a problem, published by Martín Soto on March 7, 2024 on The AI Alignment Forum. I explain (in layman's terms) a realization that might make acausal trade hard or impossible in practice. Summary: We know that if players believe different Evidential Correlations, they might miscoordinate. But clearly they will eventually learn to have the correct Evidential Correlations, right? Not necessarily, because there is no objective notion of correct here (in the way that there is for math or physics). Thus, selection pressures might be much weaker, and different agents might systematically converge on different ways of assigning Evidential Correlations. Epistemic status: Confident that this realization is true, but the quantitative question of exactly how weak the selection pressures are remains open. What are Evidential Correlations, really? Skippable if you know the answer to the question. Alice and Bob are playing a Prisoner's Dilemma, and they know each other's algorithms: Alice.source and Bob.source.[1] Since their algorithms are approximately as complex, each of them can't easily assess what the other will output. Alice might notice something like "hmm, Bob.source seems to default to Defection when it throws an exception, so this should update me slightly in the direction of Bob Defecting". But she doesn't know exactly how often Bob.source throws an exception, or what it does when that doesn't happen. Imagine, though, Alice notices Alice.source and Bob.source are pretty similar in some relevant ways (maybe the overall logical structure seems very close, or the depth of the for loops is the same, or she learns the training algorithm that shaped them is the same one). She's still uncertain about what any of these two algorithms outputs[2], but this updates her in the direction of "both algorithms outputting the same action". If Alice implements/endorses Evidential Decision Theory, she will reason as follows: Conditional on Alice.source outputting Defect, it seems very likely Bob.source also outputs Defect, thus my payoff will be low. But conditional on Alice.source outputting Cooperate, it seems very likely Bob.source also outputs Cooperate, thus my payoff will be high. So I (Alice) should output Cooperate, thus (very probably) obtain a high payoff. To the extent Alice's belief about similarity was justified, it seems like she will perform pretty well on these situations (obtaining high payoffs). When you take this reasoning to the extreme, maybe both Alice and Bob are aware that they both know this kind of cooperation bootstrapping is possible (if they both believe they are similar enough), and thus (even if they are causally disconnected, and just simulating each others' codes) they can coordinate on some pretty complex trades. This is Evidential Cooperation in Large worlds. But wait a second: How could this happen, without them being causally connected? What was this mysterious similarity, this spooky correlation at a distance, that allowed them to create cooperation from thin air? Well, in the words of Daniel Kokotajlo: it's just your credences, bro! The bit required for this to work is that they believe that "it is very likely we both output the same thing". Said another way, they have high probability on the possible worlds "Alice.source = C, Bob.source = C" and "Alice.source = D, Bob.source = D", but low probability on the possible worlds "Alice.source = D, Bob.source = C" and "Alice.source = D, Bob.source = C". This can also be phrased in terms of logical counterfactuals: if Alice.source = C, then it is very likely that Bob.source = C.[3] This is a logical counterfactual: there is, ultimately, a logical fact of the matter about what Alice.source outputs, but since she doesn't know it yet, she entertains what s...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Social status part 1/2: negotiations over object-level preferences, published by Steven Byrnes on March 7, 2024 on LessWrong. 1.1 Summary & contents This is the first of two blog posts where I try to make sense of the whole universe of social-status-related behaviors and phenomena: This post is focused on a special case of two people interacting, where they have different object-level preferences - maybe one wants to order pizza for dinner while the other wants sushi. This gets us into various topics like "leading and following", averaging different people's utility functions, being more or less "pushy", "ask culture versus guess culture", plausible deniability, politeness arms-races, and more. Then the next post, "Social status part 2/2: everything else", will layer on another heap of complexity on top of all that, related to the fact that people also have preferences related to the interaction itself, like "a preference not to be rude". That gets us into topics like dominance, prestige, getting offended, passive-aggressiveness, status, self-deprecation, and more. Some context for how I came to write this: While I often write about neuroscience and brain algorithms, these two posts have essentially none of that. They're just about systematizing everyday behavior and folk psychology, and I hope they will be generally useful as such. As it happens, my own larger project is to understand the neuroscience underlying social status behaviors (as part of this even larger project related to AI alignment). But I have no hope of figuring out the neuroscience underlying social status behaviors, if I don't understand social status behaviors in the first place. Hence these posts. I previously attempted to talk about social status a couple months ago here. I still think I was pointing towards something important and true in that old post, but it was just one little piece of the puzzle, and I described it very poorly because I was confused about the bigger picture. Anyway, I neither expect nor recommend that you read that; these two posts will hopefully be self-contained. This post is organized as follows: Section 1.2 describes the setting and some basic terminology. In particular, I use the word "negotiation" very broadly to include most everyday interactions, including making plans and decisions as a group, requesting favors, divvying up responsibilities, and even things like taking turns speaking and changing conversation topics. Section 1.3 defines two key terms for this post: "leading" and "following". If two people, Alice & Beth, are interacting, and Alice is "mostly leading" while Beth is "mostly following", that means that, when Alice & Beth have conflicting object-level preferences, the group will make decisions that follow Alice's preferences more than Beth's. I then argue that the idea of "leading" and "following" are equally applicable to both "dominance" and "prestige" interactions (in the terminology of dual strategies theory). Section 1.4 offers a toy model for the dynamic above, where Alice & Beth each has a utility function for their object-level preferences, and the group decisions are based on a weighted average of Alice's and Beth's utilities, and more "leading" simply means that your preferences get more weight in the weighted average. Thus, "leading-ness" always sums to 100%: if Alice is "70% leading" within the interaction, then Beth must be "30% leading", and so on. I discuss some insights that we get from this toy model, and also clarify a technical issue related to the incommensurability of different people's desires. Section 1.5 offers another related toy model, where there's an objective scale of "pushiness" - ranging from making strong explicit demands, to subtly hinting at one's own preferences - and where "leading" and "following" correspond respecti...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Movie posters, published by KatjaGrace on March 7, 2024 on LessWrong. Life involves anticipations. Hopes, dreads, lookings forward. Looking forward and hoping seem pretty nice, but people are often wary of them, because hoping and then having your hopes fold can be miserable to the point of offsetting the original hope's sweetness. Even with very minor hopes: he who has harbored an inchoate desire to eat ice cream all day, coming home to find no ice cream in the freezer, may be more miffed than he who never tasted such hopes. And this problem is made worse by that old fact that reality is just never like how you imagined it. If you fantasize, you can safely bet that whatever the future is is not your fantasy. I have never suffered from any of this enough to put me off hoping and dreaming one noticable iota, but the gap between high hopes and reality can still hurt. I sometimes like to think about these valenced imaginings of the future in a different way from that which comes naturally. I think of them as 'movie posters'. When you look fondly on a possible future thing, you have an image of it in your mind, and you like the image. The image isn't the real thing. It's its own thing. It's like a movie poster for the real thing. Looking at a movie poster just isn't like watching the movie. Not just because it's shorter - it's just totally different - in style, in content, in being a still image rather than a two hour video. You can like the movie poster or not totally independently of liking the movie. It's fine to like the movie poster for living in New York and not like the movie. You don't even have to stop liking the poster. It's fine to adore the movie poster for 'marrying Bob' and not want to see the movie. If you thrill at the movie poster for 'starting a startup', it just doesn't tell you much about how the movie will be for you. It doesn't mean you should like it, or that you have to try to do it, or are a failure if you love the movie poster your whole life and never go. (It's like five thousand hours long, after all.) This should happen a lot. A lot of movie posters should look great, and you should decide not to see the movies. A person who looks fondly on the movie poster for 'having children' while being perpetually childless could see themselves as a sad creature reaching in vain for something they may not get. Or they could see themselves as right there with an image that is theirs, that they have and love. And that they can never really have more of, even if they were to see the movie. The poster was evidence about the movie, but there were other considerations, and the movie was a different thing. Perhaps they still then bet their happiness on making it to the movie, or not. But they can make such choices separate from cherishing the poster. This is related to the general point that 'wanting' as an input to your decisions (e.g. 'I feel an urge for x') should be different to 'wanting' as an output (e.g. 'on consideration I'm going to try to get x'). This is obvious in the abstract, but I think people look in their heart to answer the question of what they are on consideration pursuing. Here as in other places, it is important to drive a wedge between them and fit a decision process in there, and not treat one as semi-implying the other. This is also part of a much more general point: it's useful to be able to observe stuff that happens in your mind without its occurrence auto-committing you to anything. Having a thought doesn't mean you have to believe it. Having a feeling doesn't mean you have to change your values or your behavior. Having a persistant positive sentiment toward an imaginary future doesn't mean you have to choose between pursuing it or counting it as a loss. You are allowed to decide what you are going to do, regardless of what you find...