The Nonlinear Library LW - DeepMind: Evaluating Frontier Models for Dangerous Capabilities by Zach Stein-Perlman
Mar 21, 2024
01:47
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: DeepMind: Evaluating Frontier Models for Dangerous Capabilities, published by Zach Stein-Perlman on March 21, 2024 on LessWrong.
To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning.
[Evals for CBRN capabilities are under development.] We do not find evidence of strong dangerous capabilities in the models we evaluated, but we flag early warning signs. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.
At last, DeepMind talks about its dangerous capability evals. With details! Yay!
(My weak guess is that they only finished these evals after Gemini 1.0 deployment: these evals were mentioned in an updated version of the Gemini 1.0 report but not the initial version. DeepMind hasn't yet made RSP-like commitments - that is, specific commitments about risk assessment (for extreme risks), safety and security practices as a function of risk assessment results, and training and deployment decisions as a function of risk assessment results.
Demis recently suggested on Dwarkesh that DeepMind might make RSP-like commitments this year.)
Random interesting note: DeepMind hired 8 superforecasters to make relevant predictions, most notably about when some eval-thresholds will trigger.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning.
[Evals for CBRN capabilities are under development.] We do not find evidence of strong dangerous capabilities in the models we evaluated, but we flag early warning signs. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.
At last, DeepMind talks about its dangerous capability evals. With details! Yay!
(My weak guess is that they only finished these evals after Gemini 1.0 deployment: these evals were mentioned in an updated version of the Gemini 1.0 report but not the initial version. DeepMind hasn't yet made RSP-like commitments - that is, specific commitments about risk assessment (for extreme risks), safety and security practices as a function of risk assessment results, and training and deployment decisions as a function of risk assessment results.
Demis recently suggested on Dwarkesh that DeepMind might make RSP-like commitments this year.)
Random interesting note: DeepMind hired 8 superforecasters to make relevant predictions, most notably about when some eval-thresholds will trigger.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org