The Nonlinear Library LW - AI #39: The Week of OpenAI by Zvi
Nov 23, 2023
43:46
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #39: The Week of OpenAI, published by Zvi on November 23, 2023 on LessWrong.
The board firing Sam Altman, then reinstating him, dominated everything else this week. Other stuff also happened, but definitely focus on that first.
Table of Contents
Developments at OpenAI were far more important than everything else this read. So you can read this timeline of events over the weekend, and this attempt to put all the information together.
Introduction.
Table of Contents.
Language Models Offer Mundane Utility. Narrate your life, as you do all life.
Language Models Don't Offer Mundane Utility. Prompt injection unsolved.
The Q Continuum. Disputed claims about new training techniques.
OpenAI: The Saga Continues. The story is far from over.
Altman Could Step Up. He understands existential risk. Now he can act.
You Thought This Week Was Tough. It is not getting any easier.
Fun With Image Generation. A few seconds of an Emu.
Deepfaketown and Botpocalypse Soon. Beware phone requests for money.
They Took Our Jobs. Freelancers in some areas are in trouble.
Get Involved. Dave Orr hiring for DeepMind alignment team.
Introducing. Claude 2.1 looks like a substantial incremental improvement.
In Other AI News. Meta breaks up 'responsible AI' team. Microsoft invests $50b.
Quiet Speculations. Will deep learning hit a wall?
The Quest for Sane Regulation. EU AI Act struggles, FTC AI definition is nuts.
That Is Not What Totalitarianism Means. People need to cut that claim out.
The Week in Audio. Sam Altman, Yoshua Bengio, Davidad, Ilya Sutskever.
Rhetorical Innovation. David Sacks says it best this week.
Aligning a Smarter Than Human Intelligence is Difficult. Technical debates.
People Are Worried About AI Killing Everyone. Roon fully now in this section.
Other People Are Not As Worried About AI Killing Everyone. Listen to them.
The Lighter Side. Yes, of course I am, but do you even hear yourself?
Language Models Offer Mundane Utility
GPT-4-Turbo substantially outperforms GPT-4 on Arena leaderboard. GPT-3.5-Turbo is still ahead of every model not from either OpenAI or Anthropic. Claude-1 outscores Claude-2 and is very close to old GPT-4 for second place, which is weird.
Own too much cryptocurrency? Ian built a GPT that can 'bank itself using blockchains.'
Paper says AI pancreatic cancer detection finally outperforming expert radiologists. This is the one we keep expecting that keeps not happening.
David Attenborough narrates your life how-to guide, using Eleven Labs and GPT-4V. Code here. Good pick. Not my top favorite, but very good pick.
Another good pick, Larry David as productivity coach.
Language Models Don't Offer Mundane Utility
Oh no.
Kai Greshake: PSA: The US Military is actively testing and deploying LLMs to the battlefield. I think these systems are likely to be vulnerable to indirect prompt injection by adversaries. I'll lay out the story in this thread.
This is http://Scale.ai's Donovan model. Basically, they let an LLM see and search through all of your military data (assets and threat intelligence) and then it tells you what you should do..
Now, it turns out to be really useful if you let the model see news and public information as well. This is called open-source intelligence or OSINT. In this screenshot, you can see them load "news and press reports" from the target area that the *adversary* can publish!
We've shown many times that if an attacker can inject text into your model, you get to "reprogram" it with natural language. Imagine hiding & manipulating information that is presented to the operators and then having your little adversarial minion tell them where to strike.
…
Unfortunately the goal here is to shorten the time to a decision, so cross-checking everything is impossible, and they are not afraid to talk about the intentions. There will be a "human in the loop"...
The board firing Sam Altman, then reinstating him, dominated everything else this week. Other stuff also happened, but definitely focus on that first.
Table of Contents
Developments at OpenAI were far more important than everything else this read. So you can read this timeline of events over the weekend, and this attempt to put all the information together.
Introduction.
Table of Contents.
Language Models Offer Mundane Utility. Narrate your life, as you do all life.
Language Models Don't Offer Mundane Utility. Prompt injection unsolved.
The Q Continuum. Disputed claims about new training techniques.
OpenAI: The Saga Continues. The story is far from over.
Altman Could Step Up. He understands existential risk. Now he can act.
You Thought This Week Was Tough. It is not getting any easier.
Fun With Image Generation. A few seconds of an Emu.
Deepfaketown and Botpocalypse Soon. Beware phone requests for money.
They Took Our Jobs. Freelancers in some areas are in trouble.
Get Involved. Dave Orr hiring for DeepMind alignment team.
Introducing. Claude 2.1 looks like a substantial incremental improvement.
In Other AI News. Meta breaks up 'responsible AI' team. Microsoft invests $50b.
Quiet Speculations. Will deep learning hit a wall?
The Quest for Sane Regulation. EU AI Act struggles, FTC AI definition is nuts.
That Is Not What Totalitarianism Means. People need to cut that claim out.
The Week in Audio. Sam Altman, Yoshua Bengio, Davidad, Ilya Sutskever.
Rhetorical Innovation. David Sacks says it best this week.
Aligning a Smarter Than Human Intelligence is Difficult. Technical debates.
People Are Worried About AI Killing Everyone. Roon fully now in this section.
Other People Are Not As Worried About AI Killing Everyone. Listen to them.
The Lighter Side. Yes, of course I am, but do you even hear yourself?
Language Models Offer Mundane Utility
GPT-4-Turbo substantially outperforms GPT-4 on Arena leaderboard. GPT-3.5-Turbo is still ahead of every model not from either OpenAI or Anthropic. Claude-1 outscores Claude-2 and is very close to old GPT-4 for second place, which is weird.
Own too much cryptocurrency? Ian built a GPT that can 'bank itself using blockchains.'
Paper says AI pancreatic cancer detection finally outperforming expert radiologists. This is the one we keep expecting that keeps not happening.
David Attenborough narrates your life how-to guide, using Eleven Labs and GPT-4V. Code here. Good pick. Not my top favorite, but very good pick.
Another good pick, Larry David as productivity coach.
Language Models Don't Offer Mundane Utility
Oh no.
Kai Greshake: PSA: The US Military is actively testing and deploying LLMs to the battlefield. I think these systems are likely to be vulnerable to indirect prompt injection by adversaries. I'll lay out the story in this thread.
This is http://Scale.ai's Donovan model. Basically, they let an LLM see and search through all of your military data (assets and threat intelligence) and then it tells you what you should do..
Now, it turns out to be really useful if you let the model see news and public information as well. This is called open-source intelligence or OSINT. In this screenshot, you can see them load "news and press reports" from the target area that the *adversary* can publish!
We've shown many times that if an attacker can inject text into your model, you get to "reprogram" it with natural language. Imagine hiding & manipulating information that is presented to the operators and then having your little adversarial minion tell them where to strike.
…
Unfortunately the goal here is to shorten the time to a decision, so cross-checking everything is impossible, and they are not afraid to talk about the intentions. There will be a "human in the loop"...