LessWrong (Curated & Popular)

In this podcast, johnswentworth and David Lorell propose a solution to the shutdown problem in AI by using a sub-agent architecture and negotiation between utility-maximizing subagents. They discuss the design of an agent with multiple subagents and the importance of corrugibility. They also explore alignment problems, ontological issues, designing utility functions, and challenges in bridging the theory-practice gap.

People often parse information through an epistemic consensus filter. They do not ask "is this true", they ask "will others be OK with me thinking this is true". This makes them very malleable to brute force manufactured consensus; if every screen they look at says the same thing they will adopt that position because their brain interprets it as everyone in the tribe believing it.- Anon, 4Chan, slightly editedOrdinary people who haven't spent years of their lives thinking about rationality and epistemology don't form beliefs by impartially tallying up evidence like a Bayesian reasoner. Whilst there is a lot of variation, my impression is that the majority of humans we share this Earth with use a completely different algorithm for vetting potential beliefs: they just believe some average of what everyone and everything around them believes, especially what they see on screens, newspapers and "respectable", "mainstream" websites.--- First published: February 3rd, 2024 Source: https://www.lesswrong.com/posts/bMxhrrkJdEormCcLt/brute-force-manufactured-consensus-is-hiding-the-crime-of --- Narrated by TYPE III AUDIO.

Support ongoing human narrations of LessWrong's curated posts:www.patreon.com/LWCuratedSource:https://www.lesswrong.com/posts/GfZfDHZHCuYwrHGCd/without-fundamental-advances-misalignment-and-catastropheNarrated for LessWrong by Perrin Walker.Share feedback on this narration.[Curated Post] ✓[125+ Karma Post] ✓

Background Terminology: Counterfactual Impact vs “Leading The Parade”Y’know how a parade or marching band has a person who walks in front waving a fancy-looking stick up and down? Like this guy: The classic 80's comedy Animal House features a great scene in which a prankster steals the stick, and then leads the marching band off the main road and down a dead-end alley.That is not the guy who's supposed to have that stick.In the context of the movie, it's hilarious. It's also, presumably, not at all how parades actually work these days. If you happen to be “leading” a parade, and you go wandering off down a side alley, then (I claim) those following behind will be briefly confused, then ignore you and continue along the parade route. The parade leader may appear to be “leading”, but they do not have any counterfactual impact on the route [...]--- First published: January 31st, 2024 Source: https://www.lesswrong.com/posts/LKC3XfWxPzZXK7Esd/leading-the-parade --- Narrated by TYPE III AUDIO.

Support ongoing human narrations of LessWrong's curated posts:www.patreon.com/LWCuratedSource:https://www.lesswrong.com/posts/kcKrE9mzEHrdqtDpE/the-case-for-ensuring-that-powerful-ais-are-controlledNarrated for LessWrong by Perrin Walker.Share feedback on this narration.[Curated Post] ✓[125+ Karma Post] ✓

I often encounter some confusion about whether the fact that synapses in the brain typically fire at frequencies of 1-100 Hz while the clock frequency of a state-of-the-art GPU is on the order of 1 GHz means that AIs think "many orders of magnitude faster" than humans. In this short post, I'll argue that this way of thinking about "cognitive speed" is quite misleading.The clock speed of a GPU is indeed meaningful: there is a clock inside the GPU that provides some signal that's periodic at a frequency of ~ 1 GHz. However, the corresponding period of ~ 1 nanosecond does not correspond to the timescale of any useful computations done by the GPU. For instance; in the A100 any read/write access into the L1 cache happens every ~ 30 clock cycles and this number goes up to 200-350 clock cycles for the L2 cache. The result [...]The original text contained 1 footnote which was omitted from this narration. --- First published: January 29th, 2024 Source: https://www.lesswrong.com/posts/adadYCPFAhNqDA5Ye/processor-clock-speeds-are-not-how-fast-ais-think --- Narrated by TYPE III AUDIO.

A pdf version of this report is available here.Summary. In this report we argue that AI systems capable of large scale scientific research will likely pursue unwanted goals and this will lead to catastrophic outcomes. We argue this is the default outcome, even with significant countermeasures, given the current trajectory of AI development.In Section 1 we discuss the tasks which are the focus of this report. We are specifically focusing on AIs which are capable of dramatically speeding up large-scale novel science; on the scale of the Manhattan Project or curing cancer. This type of task requires a lot of work, and will require the AI to overcome many novel and diverse obstacles.In Section 2 we argue that an AI which is capable of doing hard, novel science will be approximately consequentialist; that is, its behavior will be well described as taking actions in order [...]The original text contained 40 footnotes which were omitted from this narration. --- First published: January 26th, 2024 Source: https://www.lesswrong.com/posts/GfZfDHZHCuYwrHGCd/without-fundamental-advances-misalignment-and-catastrophe --- Narrated by TYPE III AUDIO.

This is a linkpost for https://rootsofprogress.org/the-block-funding-model-for-scienceWhen Galileo wanted to study the heavens through his telescope, he got money from those legendary patrons of the Renaissance, the Medici. To win their favor, when he discovered the moons of Jupiter, he named them the Medicean Stars. Other scientists and inventors offered flashy gifts, such as Cornelis Drebbel's perpetuum mobile (a sort of astronomical clock) given to King James, who made Drebbel court engineer in return. The other way to do research in those days was to be independently wealthy: the Victorian model of the gentleman scientist.Galileo demonstrating law of gravity in presence of Giovanni de' Medici, 1839 fresco by Giuseppe Bezzuoli MeisterdruckeEventually we decided that requiring researchers to seek wealthy patrons or have independent means was not the best way to do science. Today, researchers, in their role as “principal investigators” (PIs), apply to science funders for grants. In the [...]--- First published: January 26th, 2024 Source: https://www.lesswrong.com/posts/DKH9Z4DyusEdJmXKB/making-every-researcher-seek-grants-is-a-broken-model Linkpost URL:https://rootsofprogress.org/the-block-funding-model-for-science --- Narrated by TYPE III AUDIO.

Let your every day be full of joy, love the child that holds your hand, let your wife delight in your embrace, for these alone are the concerns of humanity.[1]— Epic of Gilgamesh - Tablet X Say we want to train a scientist AI to help in a precise, narrow field of science (e.g. medicine design) but prevent its power from being applied anywhere else (e.g. chatting with humans, designing bio-weapons, etc.) even if it has these abilities.Here's one safety layer one could implement: Train a scientist AI on a large scientific corpus translated exclusively into Sumerian. Keep it in a secure containment environment.Train a less-smart reporter whose sole ability is to translate from Sumerian to English only if the Sumerian content is about medical research. It refuses to translate other kinds of content.Human operators are only allowed to interact with the scientist AI through [...]The original text contained 2 footnotes which were omitted from this narration. --- First published: January 15th, 2024 Source: https://www.lesswrong.com/posts/PkqGxkm8XRASJ35bF/the-case-for-training-frontier-ais-on-sumerian-only-corpus-1 --- Narrated by TYPE III AUDIO.

We are organising the 9th edition without funds. We have no personal runway left to do this again. We will not run the 10th edition without funding. In a nutshell: Last month, we put out AI Safety Camp's funding case.  A private donor then decided to donate €5K.   Five more donors offered $7K on Manifund.  For that $7K to not be wiped out and returned, another $21K in funding is needed. At that level, we may be able to run a minimal version of AI Safety Camp next year, where we get research leads started in the first 2.5 months, and leave the rest to them.  The current edition is off to a productive start!  A total of 130 participants joined, spread over 26 projects. The projects are diverse – from agent foundations, to mechanistic interpretability, to copyright litigation.  Our personal runways are running out.  If we do [...]--- First published: January 24th, 2024 Source: https://www.lesswrong.com/posts/EAZjXKNN2vgoJGF9Y/this-might-be-the-last-ai-safety-camp --- Narrated by TYPE III AUDIO.