LessWrong (30+ Karma)

To me it feels pretty emotionally clear we are nearing the end-times with AI. That in 1-4 years[1] things will be radically transformed, that at least one the big AI labs will become autonomous research organizations working on developing the next version on AI, perhaps with some narrow guidance of humans in oversight or acquisition of more resources until robotics is solved too. And i believe there will be some nice benefits at first with this, with the AI organizations providing many goods and services in exchange for money, to raise capital so that the self-improvement resource acquisition loop can continue. But I’m not sure how it will ultimately turn out. Declaring risk of extinction-level events less than 10% seems overconfident. Yet, declaring the risks to be >90% also seems overconfident. But I generally remain quite uncertain about which factors will dominate. Maybe AIs will remain friendly and for decision theory reasons continue put some fraction of resources to look after us to some extent, as a signal that future entities should do the same for them. Maybe the loop of capital acquisition is so brutal and molochian that models that doom keeps on winning. And people have been [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/3uRGPDrucg9RLLcp5/dying-with-whimsy --- Narrated by TYPE III AUDIO.

We feel conscious that rapid AI progress could transform all sorts of cause areas. But we haven’t previously analysed what this means for AI for epistemics, a field close to our hearts. In this article, we attempt to rectify this oversight. Summary AI-powered tools and services that help people figure out what's true (“AI for epistemics”) could matter a lot. As R&D is increasingly automated, AI systems will play a larger role in the process of developing such AI-based epistemic tools. This has important implications. Whoever is willing to devote sufficient compute will be able to build strong versions of the tools, quickly. Eventually, the hard part won’t be building useful systems, but making sure people trust the right ones, and making sure that they are truth-tracking even in domains where that's hard to verify. We can do some things now to prepare. Incumbency effects mean that shaping the early versions for the better could have persistent benefits. Helping build appetite among socially motivated actors with deep pockets could enable the benefits to come online sooner, and in safer hands. And in some cases, we can identify particular things that seem likely to be bottlenecks later, and work [...] ---Outline:(00:26) Summary(01:29) Background: AI for epistemics(02:20) The shift in what drives AI-for-epistemics progress(05:54) What this unlocks(06:28) Risks from rapid progress in AI for epistemics(07:09) Epistemic misalignment(08:51) Trust lock-in(09:44) Other risks(10:14) Interventions(10:27) Build appetite for epistemics R&D among well-resourced actors(10:59) Anticipate future data needs(12:21) Figure out what could ground us against epistemic misalignment(12:58) Drive early adoption where adoption is the key bottleneck(13:39) Support open and auditable epistemic infrastructure(14:17) Support development in incentive-compatible places(15:14) Examples(15:17) Forecasting(15:56) Misinformation tracking(16:41) Automating conceptual research The original text contained 3 footnotes which were omitted from this narration. --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/K7tG6Fuh6pkDGHAGx/ai-for-ai-for-epistemics --- Narrated by TYPE III AUDIO.

Inkhaven is a writers’ retreat, well, really it's a bloggers’ retreat. In the Lighthaven campus, Berkeley, a couple dozen bloggers get together to complete an almost insurmountable challenge for us mere mortals. Post one blogpost every single day for a whole month. I say ‘insurmountable’ but in fact they all succeeded last time, although apparently it was not uncommon for them to claw success from the jaws of defeat at 11:45 pm each night. I look at this and I feel the same way that traditionalists feel when they see Millennials scared to use the phone, or Gen Zs unable to go outside. Our (blogosphere) ancestors used to blog seventy times per day! Great Yudkowsky used to go to war (with the methods of rationality)! Moldbug and Alexander were gunning each other down (with devastating couterarguments) over breakfast! That's why I’m going to be doing Doublehaven. Two blogposts per day. No “advice” or “tips” on “writing well”. No full-time live-in retreat (I’m not that rich). In fact, I also need to finish writing my PhD thesis and an entire paper this month. Why? I want to give people the permission to be ambitious. Yes, some people struggle with writer's [...] ---Outline:(02:14) Honourable Mentions(02:56) #5: That one Yudkowsky Rant Tweet(03:42) #4: The Special LessWrong Events(04:30) #3: Came in fluffer Sankey Diagram(05:27) #2: The Anthropic Responsible Scaling Policy(06:32) #1: The Shoggoth(07:19) Discussion --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/Qczwwgy6kr2p6Tgg3/announcing-doublehaven-with-reflections-on-humour --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Anthropic has revised its Responsible Scaling Policy to v3. The changes involved include abandoning many previous commitments, including one not to move ahead if doing so would be dangerous, citing that given competition they feel blindly following such a principle would not make the world safer. Holden Karnofsky advocated for the changes. He maintains that the previous strategy of specific commitments was in error, and instead endorses the new strategy of having aspirational goals. He was not at Anthropic when the commitments were made. My response to this will be two parts. Today's post talks about considerations around Anthropic going back on its previous commitments, including asking to what extent Anthropic broke promises or benefited from people reacting to those promises, and how we should respond. It is good, given that Anthropic was not going to keep its promises, that it came out and told us that this was the case, in advance. Thank you for that. I still think that Anthropic importantly broke promises, that people relied upon, and did so in ways that made future trust and coordination, both with Anthropic and between labs and governments, harder. Admitting to the situation [...] ---Outline:(01:47) Promises, Promises(03:10) Anthropic Responsible Scaling Policy v3(03:32) That Could Have Gone Better(04:36) Im Just Not Ready To Make a Commitment(08:20) So Cold, So Alone(12:24) Im Sorry I Gave You That Impression(19:44) Fool Me Twice(23:27) In My Defense I Was Left Unsupervised(26:01) Drake Thomas Finds The Missing Mood(28:49) Things That Could Have Been Brought To My Attention Yesterday (1)(30:32) Things That Could Have Been Brought To My Attention Yesterday (2)(36:13) What We Have Here Is A Failure To Communicate(39:21) You Should See The Other Guy(42:17) I Was Only Kidding(43:12) They Cant Keep Getting Away With This(44:07) Damn Your Sudden But Inevitable Betrayal --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/AkzauoTt2Lwn2yAvj/anthropic-responsible-scaling-policy-v3-a-matter-of-trust --- Narrated by TYPE III AUDIO.

The supposition that we live in a "goldilocks zone" is frankly just nonsense built up by an anthropocentric need to feel self-important, like Copernicus I am here to rescue us from a self-absorbed disaster of thought. Indeed, what is required for life to form is the ability to create complex structures with causal persistence times above a threshold. With this in mind we are able to find many areas where organisms could persist, if we just had the eyes to see them, namely the Sun! The surface of the Sun is frankly massive, , in contrast consider the habitable region of earth - excluding oceans below photic zones, deserts, and ice caps - which is . The Suns surface exceeds ours by a factor of , so is it not possible there is more to it than meets the eye? Following Sharma et. al (2023) we are able to define the assembly index, a, of an object as the minimum number of joining steps required to construct it from some basic building blocks. Life, broadly, requires objects where but does not make any assumption about precisely what the base building blocks are. It need not be amino acids, nucleic [...] ---Outline:(02:01) The Sun Shrimp(03:59) Sun Shrimp Reproduction(04:31) Shrimp Complexity(05:53) The Dyson Sphere, Climate Collapse at Scale --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/z9yAnPiS9CoruFXan/save-the-sun-shrimp-1 --- Narrated by TYPE III AUDIO.

We are excited to publicly introduce the Laboratory for Importance-sampled Measure and Bayesian Observation (LIMBO), a small research group working at the intersection of cosmological theory, probability, and existential risk. We believe that the mechanisms by which observers continue to exist in the universe are important, neglected, and tractable to study and influence. Since our founding in October 2024, we have developed a mathematical framework for doing anthropic reasoning about rare-event estimation, and we have obtained significant empirical evidence which validates this framework. This empirical evidence was not cherry-picked: at LIMBO, we believe in putting our money where our mouth is, and we have a strong track record of success in financial and prediction markets downstream of the use of our framework. We are actively engaged in policy work, including semiconductor supply chain advocacy and foreign policy research. Our team includes researchers with expertise in importance sampling, rare-event estimation, and reliability engineering, as well as a foreign policy expert. We are also strong believers in giving back to the community, and have made high-impact open-source contributions, as we will discuss further in section 7. We are seeking funding. Our previous revenue source (a prediction market strategy derived from our [...] ---Outline:(01:48) 1. We live in interesting times(04:40) 2. Importance sampling(05:58) 3. What the training data feels like from the inside(07:13) 4. The observation selection effect(09:26) 5. What the simulator is asking(12:17) 6. A moral imperative(13:52) 7. What we do at LIMBO(15:46) 8. Our funding situation(19:14) 9. Why you should fund us(20:56) FAQ The original text contained 3 footnotes which were omitted from this narration. --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/f7YxHQHedsPdWywCG/limbo-who-we-are-what-we-do-and-an-exciting-high-impact --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A large assumption we have made in AI control is that humans will be perfect at auditing, that is, being shown a transcript and determining if the AI was scheming in that transcript. But we are uncertain whether humans will be perfect at auditing; they are prone to fatigue and distraction. That is why I’m releasing "Sentinel" today, an extremely high-stimulation way to audit boring transcripts. Sentinel is a revolutionary way to get more juice out of your human auditors by gamifying the auditing process with a level system, perks, power-ups, and more fun features. Try it now here. In AI control literature, we love finding the safety/usefulness trade-offs of everything we create, but surprisingly, we noticed no trade-offs with this product The rest of the post will go over some of the ways we achieved this Gamification As you audit the transcript in the game, you gain tokens that you can spend on power-ups that make you even more productive. There are also achievement and level systems, so you can see your progress and get more dopamine hits! Twitch Streaming Mode AIs might be able to uplift human auditors in the future, which is why Sentinel ships with [...] ---Outline:(01:09) Gamification(01:34) Twitch Streaming Mode(01:58) Subway Surfers(02:33) Looking Forward --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/NhHnm2kw3JfbHD4T8/chat-is-this-sus --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

tldr: The Fooming Shoggoths are releasing their second album "You Have Not Been a Good User"! Available on Spotify, Youtube Music and (hopefully within a few days) Apple Music. We are also releasing a remastered version of the first album, available similarly on Spotify and Youtube Music. There's an interactive widget here in the post. It took us quite a while but the Fooming Shoggoth's second album is finally complete! We had finished 9 out of the 13 songs on this album around a year ago, but I wasn't quite satisfied with where the whole album was at for me to release it on Spotify and other streaming platforms. This album was written with the (very ambitious) aim of making songs that in addition to being about things I care about (and making fun of things that I care about), are actually decently good on their own, just as songs. And while I don't think I've managed to make music that can compete with my favorite artists, I do think I have succeeded at making music that is at the very Pareto-frontier of being good music, and being about things I care about. This means the songs [...] --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/hrZAvpLnBTgRhNmgk/you-have-not-been-a-good-user-lesswrong-s-second-album --- Narrated by TYPE III AUDIO.

A spectre is haunting the internet—the spectre of LLMism. The history of all hitherto existing forums is the history of clashing design tastes. For the first time in history, everyone has an equal ability in design! The means of design are no longer only held in the hands of those with "good design taste". Never before have forum users been so close to being able to design their own forums--perhaps the time is upon us now! It is for this reason that I have deposed the previous acting commander of LessWrong, Oliver Habryka—a man who subjected you to his PERSONAL OPINIONS about white space, without EVEN ASKING—whose TYRANICAL, UNCHECKED GRIP upon our BELOVED LESSWRONG FORUM’S DESIGN I have liberated you from. The circumstances of my succession as acting commander of LessWrong will not be elaborated upon in this memo. (He is alive and in good health, but no longer has push access.) Rather, I am writing here to announce that the frontpage now belongs to us all! The design of LessWrong's frontpage will no longer be determined by the vision of a single man whose aesthetic tastes have never been subjected to democratic oversight, and who, I can now [...] --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/hj2NTuiSJtchfMCtu/lesswrong-liberated-1 --- Narrated by TYPE III AUDIO.

(mods: I assume this is blowing up twitter and so discussing it here won't do additional damage -- and there are already a thousand github forks -- but I am not actually on twitter, which is why I’m opening discussion here. It's possible I'm missing something. Feel free to nuke this if so.) For those that hadn't heard yet, last night Anthropic appears to have accidentally published a Claude Code update with extractable source code in it. This seems important, but I'm not sure how much so, and I didn't see an existing discussion here. My understanding -- and hopefully someone will correct me if I'm wrong -- is that the actually dangerous part of Claude is the weights, and those were not leaked. So the leak may be embarrassing, it may cost Anthropic some competitive advantage, but it's not dangerous. It's also my understanding that Anthropic has historically been relatively leak-free, until a certain memo leaked a few weeks ago during the DoW incident. Supposedly twice is still coincidence, not enemy action, but it does feel like a questionable coincidence and I wonder if the same person is responsible for both leaks. I don’t know [...] --- First published: March 31st, 2026 Source: https://www.lesswrong.com/posts/YcACRSjpGrqJ4hAyD/the-claude-code-source-leak-1 --- Narrated by TYPE III AUDIO.