The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Creative video ads significantly increase GWWC's social media engagement and web traffic to pledge page, published by James Odene [User-Friendly] on March 1, 2024 on The Effective Altruism Forum. Overview We recently ran a test to see if utilising creative marketing practices could increase the performance of a brand-led digital campaign, and it did. We are sharing the results to encourage other organisations to consider the quality of their output in areas where this could directly impact engagement, potentially resulting in increased donations and or pledge sign ups in the long-term. We partnered with Giving What We Can and produced a brand awareness video to test the power of creativity - the results prove the worth in investing time in the creative quality of movement marketing overall. Grace Adams, Head of Marketing at GWWC: "We are really happy with the performance of this campaign, and it's given us more confidence to undertake more creative approaches with future campaigns. We're excited to see how the increased awareness will translate into results further down the line." Creative marketing practice refers to content production that employs innovative and imaginative approaches to capture the attention of the target audience and evoke emotional responses, effectively conveying the core message. The aim here was to test a more creative approach comparatively to the existing content across their social media. The quality of the creative concept in an ad is one of the biggest drivers to impact (see Kantar research here), and the results from this campaign indicate that we could see great returns by swapping out low fidelity, simply informative, ration-led content for more distinctive and emotive content. The Campaign You can see a version of the ad here. Objective Increase Giving What We Can brand awareness over Giving Season Target Educated, top 50% earners, median ~30, working professionals with interest in relevant philanthropic topics e.g. climate breakdown Ad Spend $4,899.58 on YouTube & $7,172.41 on Instagram Duration ~8 weeks Channels Videos for Instagram and YouTube. We also created related display ads to direct web traffic back to the GWWC pledge Overall Metrics Reach: 4,554,692 Total Impressions: 7,923,623 CPM: $1.61 Views: 5,357,063 Engagements: 931,100 CPE: US$0.01 Web Traffic: 24,914 new users Pledge Page Visits: 465 From retargeting: 1111 users visited the website for 4 mins or more & 469 users visited for 10 mins or more The Headliners You can see a version of the ad here. 1) 48x more views on Reels, Stories and Feed, than any previous campaign on Instagram. 2) Attributed Instagram profile visits were 249% higher than any other previous campaign. 3) Referrals from Instagram to the GWWC website are typically extremely low (<100 per month) however we produced: 1622 users referred from Instagram visited 4 mins or more 429 users referred from Instagram visited 10 mins or more 4) 96% of the earned likes[1] from the GWWC account lifetime originated from this campaign. This not only signifies the quality of the audience we targeted - individuals new to the content - but also highlights their engagement with the broader content ecosystem of the brand. 5) Retargeting web traffic with campaign-branded ads produced almost 3x more people visiting the pledge page compared to organic traffic visits to the pledge page, despite organic traffic representing twice as many new users overall. 6) Reached over 4.5m people in the UK with an average CPM (cost per 1000 impressions) of $1.61. This is 2.5 times lower than the average CPM for UK video campaigns, especially for YouTube and Instagram. And substantially lower than Giving What We Can's previous CPM average of $6.14. To consider; Organisations are not fully utilising creative marketing practices meaning that for...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Parable Of The Fallen Pendulum - Part 1, published by johnswentworth on March 1, 2024 on LessWrong. One day a physics professor presents the standard physics 101 material on gravity and Newtonian mechanics: g = 9.8 m/s^2, sled on a ramp, pendulum, yada yada. Later that week, the class has a lab session. Based on the standard physics 101 material, they calculate that a certain pendulum will have a period of approximately 3.6 seconds. They then run the experiment: they set up the pendulum, draw it back to the appropriate starting position, and release. Result: the stand holding the pendulum tips over, and the whole thing falls on the floor. Stopwatch in hand, they watch the pendulum sit still on the floor, and time how often it returns to the same position. They conclude that the pendulum has a period of approximately 0.0 seconds. Being avid LessWrong readers, the students reason: "This Newtonian mechanics theory predicted a period of approximately 3.6 seconds. Various factors we ignored (like e.g. friction) mean that we expect that estimate to be somewhat off, but the uncertainty is nowhere near large enough to predict a period of approximately 0.0 seconds. So this is a large Bayesian update against the Newtonian mechanics model. It is clearly flawed." The physics professor replies: "No no, Newtonian mechanics still works just fine! We just didn't account for the possibility of the stand tipping over when predicting what would happen. If we go through the math again accounting for the geometry of the stand, we'll see that Newtonian mechanics predicts it will tip over…" (At this point the professor begins to draw a diagram on the board.) The students intervene: "Hindsight! Look, we all used this 'Newtonian mechanics' theory, and we predicted a period of 3.6 seconds. We did not predict 0.0 seconds, in advance. You did not predict 0.0 seconds, in advance. Theory is supposed to be validated by advance predictions! We're not allowed to go back after-the-fact and revise the theory's supposed prediction. Else how would the theory ever be falsifiable?" The physics professor replies: "But Newtonian mechanics has been verified by massive numbers of experiments over the years! It's enabled great works of engineering! And, while it does fail in some specific regimes, it consistently works on this kind of system - " The students again intervene: "Apparently not. Unless you want to tell us that this pendulum on the floor is in fact moving back-and-forth with a period of approximately 3.6 seconds? That the weight of evidence accumulated by scientists and engineers over the years outweighs what we can clearly see with our own eyes, this pendulum sitting still on the floor?" The physics professor replies: "No, of course not, but clearly we didn't correctly apply the theory to the system at hand-" The students: "Could the long history of Newtonian mechanics 'consistently working' perhaps involve people rationalizing away cases like this pendulum here, after-the-fact? Deciding, whenever there's a surprising result, that they just didn't correctly apply the theory to the system at hand?" At this point the physics professor is somewhat at a loss for words. And now it is your turn! What would you say to the students, or to the professor? Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Approaching Human-Level Forecasting with Language Models, published by Fred Zhang on February 29, 2024 on The AI Alignment Forum. TL;DR: We present a retrieval-augmented LM system that nears the human crowd performance on judgemental forecasting. Paper: https://arxiv.org/abs/2402.18563 (Danny Halawi*, Fred Zhang*, Chen Yueh-Han*, and Jacob Steinhardt) Twitter thread: https://twitter.com/JacobSteinhardt/status/1763243868353622089 Abstract Forecasting future events is important for policy and decision-making. In this work, we study whether language models (LMs) can forecast at the level of competitive human forecasters. Towards this goal, we develop a retrieval-augmented LM system designed to automatically search for relevant information, generate forecasts, and aggregate predictions. To facilitate our study, we collect a large dataset of questions from competitive forecasting platforms. Under a test set published after the knowledge cut-offs of our LMs, we evaluate the end-to-end performance of our system against the aggregates of human forecasts. On average, the system nears the crowd aggregate of competitive forecasters and in some settings, surpasses it. Our work suggests that using LMs to forecast the future could provide accurate predictions at scale and help inform institutional decision-making. For safety motivations on automated forecasting, see Unsolved Problems in ML Safety (2021) for discussions. In the following, we summarize our main research findings. Current LMs are not naturally good at forecasting First, we find that LMs are not naturally good at forecasting when evaluated zero-shot (with no fine-tuning and no retrieval). On 914 test questions that were opened after June 1, 2023 (post the knowledge cut-offs of these models), most LMs get near chance performance. Here, all questions are binary, so random guessing gives a Brier score of 0.25. Averaging across all community predictions over time, the human crowd gets 0.149. We present the score of the best model of each series. Only GPT-4 and Claude-2 series beat random guessing (by a margin of >0.3), though still very far from human aggregates. System building Towards better automated forecasting, we build and optimize a retrieval-augmented LM pipeline for this task. It functions in 3 steps, mimicking the traditional forecasting procedure: Retrieval, which gathers relevant information from news sources. Here, we use LM to generate search queries given a question, use these queries to query a news corpus for articles, filter out irrelevant articles, and summarize the remaining. Reasoning, which weighs available data and makes a forecast. Here, we prompt base and fine-tuned GPT-4 models to generate forecasts and (verbal) reasonings. Aggregation, which ensembles individual forecasts into an aggregated prediction. We use trimmed mean to aggregate all the predictions. We optimize the system's hyperparameters and apply a self-supervised approach to fine-tune a base GPT-4 to obtain the fine-tuned LM. See Section 5 of the full paper for details. Data and models We use GPT-4-1106 and GPT-3.5 in our system, whose knowledge cut-offs are in April 2023 and September 2021. To optimize and evaluate the system, we collect a dataset of forecasting questions from 5 competitive forecasting platforms, including Metaculus, Good Judgment Open, INFER, Polymarket, and Manifold. The test set consists only of questions published after June 1st, 2023. Crucially, this is after the knowledge cut-off date of GPT-4 and GPT-3.5, preventing leakage from pre-training. The train and validation set contains questions before June 1st, 2023, used for hyperparameter search and fine-tuning a GPT-4 base model. Evaluation results For each question, we perform information retrieval at up to 5 different dates during the question's time span and e...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #53: One More Leap, published by Zvi on February 29, 2024 on LessWrong. The main event continues to be the fallout from The Gemini Incident. Everyone is focusing there now, and few are liking what they see. That does not mean other things stop. There were two interviews with Demis Hassabis, with Dwarkesh Patel's being predictably excellent. We got introduced to another set of potentially highly useful AI products. Mistral partnered up with Microsoft the moment Mistral got France to pressure the EU to agree to cripple the regulations that Microsoft wanted crippled. You know. The usual stuff. Table of Contents Introduction. Table of Contents. Language Models Offer Mundane Utility. Copilot++ suggests code edits. Language Models Don't Offer Mundane Utility. Still can't handle email. OpenAI Has a Sales Pitch. How does the sales team think about AGI? The Gemini Incident. CEO Pinchai responds, others respond to that. Political Preference Tests for LLMs. How sensitive to details are the responses? GPT-4 Real This Time. What exactly should count as plagiarized? Fun With Image Generation. MidJourney v7 will have video. Deepfaketown and Botpocalypse Soon. Dead internet coming soon? They Took Our Jobs. Allow our bot to provide you with customer service. Get Involved. UK Head of Protocols. Sounds important. Introducing. Evo, Emo, Genie, Superhuman, Khanmigo, oh my. In Other AI News. 'Amazon AGI' team? Great. Quiet Speculations. Unfounded confidence. Mistral Shows Its True Colors. The long con was on, now the reveal. The Week in Audio. Demis Hassabis on Dwarkesh Patel, plus more. Rhetorical Innovation. Once more, I suppose with feeling. Open Model Weights Are Unsafe and Nothing Can Fix This. Another paper. Aligning a Smarter Than Human Intelligence is Difficult. New visualization. Other People Are Not As Worried About AI Killing Everyone. Worry elsewhere? The Lighter Side. Try not to be too disappointed. Language Models Offer Mundane Utility Take notes for your doctor during your visit. Dan Shipper spent a week with Gemini 1.5 Pro and reports it is fantastic, the large context window has lots of great uses. In particular, Dan focuses on feeding in entire books and code bases. Dan Shipper: Somehow, Google figured out how to build an AI model that can comfortably accept up to 1 million tokens with each prompt. For context, you could fit all of Eliezer Yudkowsky's 1,967-page opus Harry Potter and the Methods of Rationality into every message you send to Gemini. (Why would you want to do this, you ask? For science, of course.) Eliezer Yudkowsky: This is a slightly strange article to read if you happen to be Eliezer Yudkowsky. Just saying. What matters in AI depends so much on what you are trying to do with it. What you try to do with it depends on what you believe it can help you do, and what it makes easy to do. A new subjective benchmark proposal based on human evaluation of practical queries, which does seem like a good idea. Gets sensible results with the usual rank order, but did not evaluate Gemini Advanced or Gemini 1.5. To ensure your query works, raise the stakes? Or is the trick to frame yourself as Hiro Protagonist? Mintone: I'd be interested in seeing a similar analysis but with a slight twist: We use (in production!) a prompt that includes words to the effect of "If you don't get this right then I will be fired and lose my house". It consistently performs remarkably well - we used to use a similar tactic to force JSON output before that was an option, the failure rate was around 3/1000 (although it sometimes varied key names). I'd like to see how the threats/tips to itself balance against exactly the same but for the "user" reply. Linch: Does anybody know why this works??? I understand prompts to mostly be about trying to get the AI to be in the ~right data distributio...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What are the biggest misconceptions about biosecurity and pandemic risk?, published by 80000 Hours on February 29, 2024 on The Effective Altruism Forum. by Anemone Franz and Tessa Alexanian 80,000 Hours ranks preventing catastrophic pandemics as one of the most pressing problems in the world, and we have advised many of our readers to work in biosecurity to have high-impact careers. But biosecurity is a complex field, and while the threat is undoubtedly large, there's a lot of disagreement about how best to conceptualise and mitigate the risks. We wanted to get a better sense of how the people thinking about these threats every day perceive the risks. So we decided to talk to more than a dozen biosecurity experts to better understand their views. To make them feel comfortable speaking candidly, we granted the experts we spoke to anonymity. Sometimes disagreements in this space can get contentious, and certainly many of the experts we spoke to disagree with one another. We don't endorse every position they've articulated below. We think, though, that it's helpful to lay out the range of expert opinions from people who we think are trustworthy and established in the field. We hope this will inform our readers about ongoing debates and issues that are important to understand - and perhaps highlight areas of disagreement that need more attention. The group of experts includes policymakers serving in national governments, grantmakers for foundations, and researchers in both academia and the private sector. Some of them identify as being part of the effective altruism community, while others do not. All the experts are mid-career or at a more senior level. Experts chose to provide their answers either in calls or in written form. Below, we highlight 14 responses from these experts about misconceptions and mistakes that they believe are common in the field of biosecurity, particularly as it relates to people working on global catastrophic risks and in the effective altruism community. Here are some of the areas of disagreement that came up: What lessons should we learn from COVID-19? Is it better to focus on standard approaches to biosecurity or search for the highest-leverage interventions? Should we prioritise preparing for the most likely pandemics or the most destructive pandemics - and is there even a genuine trade-off between these priorities? How big a deal are "information hazards" in biosecurity? How should people most worried about global catastrophic risks engage with the rest of the biosecurity community? How big a threat are bioweapons? For an overview of this area, you can read our problem profile on catastrophic pandemics. (If you're not very familiar with biosecurity, that article may provide helpful context for understanding the experts' opinions below.) Here's what the experts said. Expert 1: Failures of imagination and appeals to authority In discussions around biosecurity, I frequently encounter a failure of imagination. Individuals, particularly those in synthetic biology and public health sectors, tend to rely excessively on historical precedents, making it difficult for them to conceive of novel biological risks or the potential for bad actors within a range of fields. This narrow mindset hinders proactive planning and compromises our ability to adequately prepare for novel threats. Another frequent problem is appeal to authority. Many people tend to suspend their own critical reasoning when a viewpoint is confidently presented by someone they perceive as an authoritative figure. This can stymie deeper reflections on pressing biosecurity issues and becomes especially problematic when compounded by information cascades. In such scenarios, an uncritically accepted idea from an authoritative source can perpetuate as fact, sometimes going unquestioned for...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Bengio's Alignment Proposal: "Towards a Cautious Scientist AI with Convergent Safety Bounds", published by mattmacdermott on February 29, 2024 on LessWrong. Yoshua Bengio recently posted a high-level overview of his alignment research agenda on his blog. I'm pasting the full text below since it's fairly short. What can't we afford with a future superintelligent AI? Among others, confidently wrong predictions about the harm that some actions could yield. Especially catastrophic harm. Especially if these actions could spell the end of humanity. How can we design an AI that will be highly capable and will not harm humans? In my opinion, we need to figure out this question - of controlling AI so that it behaves in really safe ways - before we reach human-level AI, aka AGI; and to be successful, we need all hands on deck. Economic and military pressures to accelerate advances in AI capabilities will continue to push forward even if we have not figured out how to make superintelligent AI safe. And even if some regulations and treaties are put into place to reduce the risks, it is plausible that human greed for power and wealth and the forces propelling competition between humans, corporations and countries, will continue to speed up dangerous technological advances. Right now, science has no clear answer to this question of AI control and how to align its intentions and behavior with democratically chosen values. It is a bit like in the "Don't Look Up" movie. Some scientists have arguments about the plausibility of scenarios (e.g., see "Human Compatible") where a planet-killing asteroid is headed straight towards us and may come close to the atmosphere. In the case of AI there is more uncertainty, first about the probability of different scenarios (including about future public policies) and about the timeline, which could be years or decades according to leading AI researchers. And there are no convincing scientific arguments which contradict these scenarios and reassure us for certain, nor is there any known method to "deflect the asteroid", i.e., avoid catastrophic outcomes from future powerful AI systems. With the survival of humanity at stake, we should invest massively in this scientific problem, to understand this asteroid and discover ways to deflect it. Given the stakes, our responsibility to humanity, our children and grandchildren, and the enormity of the scientific problem, I believe this to be the most pressing challenge in computer science that will dictate our collective wellbeing as a species. Solving it could of course help us greatly with many other challenges, including disease, poverty and climate change, because AI clearly has beneficial uses. In addition to this scientific problem, there is also a political problem that needs attention: how do we make sure that no one triggers a catastrophe or takes over political power when AGI becomes widely available or even as we approach it. See this article of mine in the Journal of Democracy on this topic. In this blog post, I will focus on an approach to the scientific challenge of AI control and alignment. Given the stakes, I find it particularly important to focus on approaches which give us the strongest possible AI safety guarantees. Over the last year, I have been thinking about this and I started writing about it in this May 2023 blog post (also see my December 2023 Alignment Workshop keynote presentation). Here, I will spell out some key thoughts that came out of a maturation of my reflection on this topic and that are driving my current main research focus. I have received funding to explore this research program and I am looking for researchers motivated by existential risk and with expertise in the span of mathematics (especially about probabilistic methods), machine learning (especially about amorti...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Wholesomeness and Effective Altruism, published by Owen Cotton-Barratt on February 29, 2024 on The Effective Altruism Forum. This is the second of a collection of three essays, ' On Wholesomeness'. In the first essay I introduced the idea of wholesomeness as a criterion for choosing actions. This essay will explore the relationship between acting wholesomely and some different conceptions of effective altruism. Tensions Apparent tensions Acting wholesomely feels relatively aligned with traditional commonsense notions of doing good. To the extent that EA is offering a new angle on doing good, shouldn't we expect its priorities to clash with what being wholesome suggests? (It would be a suspicious convergence if not!) Getting more concrete: It feels wholesome to support our local communities, but EA suggests it would be more effective to support others far removed from us. It doesn't feel wholesome to reorient strategies around speculative sci-fi concerns. But this is what a large fraction of EA has done with AI stuff. Surely there are tensions here? Aside: acting wholesomely and commonsense morality Although I've just highlighted that acting wholesomely often feels aligned with commonsense morality, I think it's important to note that it certainly doesn't equal commonsense morality. Wholesome action means attending to the whole of things one can understand, and that may include esoteric considerations which wouldn't get a look in on commonsense morality. The alignment is more one-sided: if commonsense morality doesn't like something, there's usually some reason for the dislike. Wholesomeness will seek not to dismiss these objections out of hand, but rather to avoid such actions unless the objections have been thoroughly understood and felt not to stand up. The shut-up-and-multiply perspective A particular perspective which is often associated with EA is the idea of taking expected value seriously, and choosing our actions on this basis. The catch-phrase of this perspective might be "shut up and multiply!". Taken at face value, this perspective would recommend: We put everything we can into an explicit model We use this to determine what seems like the best option We pursue that option Deep tensions between wholesomeness and straw EA There's a kind of simplistic version of EA which tells you to work out what the most important things are and then focus on maximizing goodness there. This is compatible with using the shut-up-and-multiply perspective to work what's most important, but doesn't require it. I don't think that this simplistic version of EA is the correct version of EA (precisely because it misses the benefits of wholesomeness; or for another angle on its issues see EA is about maximization, and maximization is perilous). But I do think it's a common thing to perceive EA principles as saying, perhaps especially by people who are keen to criticise EA[1]. For this reason I'll label it "straw EA". There is a quite fundamental tension between acting wholesomely and straw EA: Wholesomeness tells you to focus on the whole, and not let actions be dictated by impact on a few parts of things Straw EA tells you to focus on the most important dimensions and maximize there - implicitly telling you to ignore everything else Indeed when EA is introduced it is sometimes emphasised that we shouldn't necessarily focus on helping those close to us, which could sound like an instruction to forget whom we're close to Wholesome EA I don't think that these apparent tensions are necessary. In this section I'll describe a version of effective altruism, which I'll call "wholesome EA", which is deeply grounded in a desire to act wholesomely. Although the articulation is new, I don't think that the thing I'm proposing here is fundamentally novel - I feel like I've seen some version of t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Tips for Empirical Alignment Research, published by Ethan Perez on February 29, 2024 on The AI Alignment Forum. TLDR: I've collected some tips for research that I've given to other people and/or used myself, which have sped things up and helped put people in the right general mindset for empirical AI alignment research. Some of these are opinionated takes, also around what has helped me. Researchers can be successful in different ways, but I still stand by the tips here as a reasonable default. What success generally looks like Here, I've included specific criteria that strong collaborators of mine tend to meet, with rough weightings on the importance, as a rough north star for people who collaborate with me (especially if you're new to research). These criteria are for the specific kind of research I do (highly experimental LLM alignment research, excluding interpretability); some examples of research areas where this applies are e.g. scalable oversight, adversarial robustness, chain-of-thought faithfulness, process-based oversight, and model organisms of misalignment. The exact weighting will also vary heavily depending on what role you're serving on the team/project. E.g., I'd probably upweight criteria where you're differentially strong or differentially contributing on the team, since I generally guide people towards working on things that line up with their skills. For more junior collaborators (e.g., first time doing a research project, where I've scoped out the project), this means I generally weigh execution-focused criteria more than direction-setting criteria (since here I'm often the person doing the direction setting). Also, some of the criteria as outlined below are a really high bar, and e.g. I only recently started to meet them myself after 5 years of doing research and/or I don't meet other criteria myself. This is mainly written to be a north star for targets to aim for. That said, I think most people can get to a good-to-great spot on these criteria with 6-18 months of trying, and I don't currently think that many of these criteria are particularly talent/brains bottlenecked vs. just doing a lot of deliberate practice and working to get better on these criteria (I was actively bad at some of the criteria below like implementation speed even ~6 months into me doing research, but improved a lot since then with practice). With that context, here are the rough success criteria I'd outline: [70%] Getting ideas to work quickly [45%] Implementation speed Able to quickly implement a well-scoped idea. An example of doing really well here is if we talk about an idea one day and decide it's exciting/worth doing, and you tell me the next day whether it worked Able to run a high volume of experiments. You're doing really well here if it's hard for your supervisor to keep up with the volume of the experiments/results you're showing; 30m or even 60m weekly 1:1 meetings should feel like not long enough to discuss all of the results you have, and you have to filter what we discuss in our weekly meetings to just the most important and decision-relevant results. If some experiments take a while to run, you're running a lot of other project-relevant experiments in parallel or implementing the next experiment (Exceptions: the experiments you're running take more than overnight/18h to run and there's no way to design them to be shorter; or the experiments are very implementation-heavy) Able to design a minimal experiment to test a mid/high-level idea. You run experiments in a way such that you're rarely compute or experiment-time bottlenecked (especially early in a project), and your experiments are designed to be easy/quick to implement You trade off code quality and implementation speed in the best way for long-run productivity. You bias heavily towards speed in gener...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Post series on "Liability Law for reducing Existential Risk from AI", published by Nora Ammann on February 29, 2024 on The AI Alignment Forum. Gabriel Weil (Assistant Professor of Law, Touro University Law Center) wrote this post series on the role of Liability Law for reducing Existential Risk from AI. I think this may well be of interest to some people here, so wanted for a linkpost to exist. The first post argues that Tort Law Can Play an Important Role in Mitigating AI Risk. The second post addressed directly How Technical AI Safety Researchers Can Help Implement Punitive Damages to Mitigate Catastrophic AI Risk Here is the full paper. TLDR (from the first post) Legal liability could substantially mitigate AI risk, but current law falls short in two key ways: (1) it requires provable negligence, and (2) it greatly limits the availability of punitive damages. Applying strict liability (a form of liability that does not require provable negligence) and expanding the availability and flexibility of punitive damages is feasible, but will require action by courts or legislatures. Legislatures should also consider acting in advance to create a clear ex ante expectation of liability and imposing liability insurance requirements for the training and deployment of advanced AI systems. The following post is a summary of a law review article. Here is the full draft paper. Dylan Matthews also did an excellent write-up of the core proposal for Vox's Future Perfect vertical. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evidential Cooperation in Large Worlds: Potential Objections & FAQ, published by Chi on February 28, 2024 on The Effective Altruism Forum. What is this post? This post is a companion piece to recent posts on evidential cooperation in large worlds (ECL). We've noticed that in conversations about ECL, the same few initial confusions and objections tend to be brought up. We hope that this post will be useful as the place that lists and discusses these common objections. We invite the reader to advance additional questions or objections of their own. This FAQ does not need to be read in order. The reader is encouraged to look through the section headings and jump to those they find most interesting. ECL seems very weird. Are you sure you haven't, like, taken a wrong turn somewhere? We don't think so. ECL, at its core, takes two reasonable ideas that by themselves are considered quite plausible by many - albeit not completely uncontroversial - and notices that when you combine them, you get something quite interesting and novel. Specifically, ECL combines "large world" with "noncausal decision theory." Many people believe the universe/multiverse is large, but that it might as well be small because we can only causally influence, or be influenced by, a small, finite part of it. Meanwhile, many people think you should cooperate in a near twin prisoners' dilemma, but that this is mostly a philosophical issue because near twin prisoners' dilemmas rarely, if ever, happen in real life. Putting the two ideas together: once you consider the noncausal effects of your actions, the world being large is potentially a very big deal.[1] Do I need to buy evidential decision theory for this to work? There are some different ways of thinking that take into account acausal influence and explain it in different ways. These include evidential decision theory and functional decision theory, as mentioned in our " ECL explainer" post. Updatelessness and superrationality are two other concepts that might get you all or part of the way to this kind of acausal cooperation. Evidential decision theory says that what matters is whether your choice gives you evidence about what the other agent will do. For example, if you are interacting with a near-copy, then the similarity between the two of you is evidence that the two of you make the same choice. Functional decision theory says that what matters is whether there is a logical connection between you and the other agent's choices. For example, if you are interacting with a copy, then the similarity between the two of you is reason to believe there is a strong logical connection. That said, functional decision theory does not have a clear formalization, so it is not clear if and how this logical connection generalizes to dealing with merely near-copies (as opposed to full copies). Our best guess is that proponents of functional decision theory at least want the theory to recommend cooperating in the near twin prisoner's dilemma.[2] Updatelessness strengthens the case for cooperation. This is because updatelessness arguably increases the game-theoretic symmetry of many kinds of interactions, which is helpful to get agents employing some types of decision procedures (including evidential decision theory) to cooperate.[3] Superrationality says that two rational thinkers considering the same problem will arrive at the same correct answer. So, what matters is common rationality. In game theory situations like the prisoner's dilemma, knowing that the two answers, or choices, will be the same might change the answer itself (e.g., cooperate-cooperate rather than defect-defect). ECL was in fact originally named " multiverse-wide superrationality". We don't take a stance in our " ECL explainer" piece on which of these decision theories, concepts, or others we d...