The Evaluation-Real World Gap

The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider becoming a free or paid subscriber.

* 00:00 - Introduction

* 01:38 - Sabotage, Deception and Evaluation Integrity

* 04:16 - Sandbagging On Dangerous Capability Evaluations

* 06:33 - Situational Awareness

* 08:20 - Inhibiting Evaluation Awareness (6.5)

* 10:07 - Self-Preference

* 11:59 - UK AISI Testing

* 13:10 - Apollo Research Testing

* 16:00 - Responsible Scaling Policy Evaluations

* 17:19 - CBRN (mostly Biology)

* 20:10 - Autonomy

* 28:07 - Autonomy Benchmarks

* 32:43 - Cyber

* 34:17 - Ship It Anyway

* 36:39 - You Are Not Ready

https://open.substack.com/pub/thezvi/p/claude-opus-46-system-card-part-2?utm_campaign=post-expanded-share&utm_medium=web