Joel Becker

Joel Becker, researcher leading METR (model evaluation and threat research), explains the origin and limits of the Time Horizon chart. He walks through how tasks were chosen and biases in benchmarks. The conversation covers developer productivity studies, milestone-like model jumps, threat modeling for discontinuous takeoff, and trade-offs in building evaluation scaffolds versus real-world deployment.

Joel Becker, METR researcher on AI evaluation, and Chris Painter, METR president focused on AI risk, unpack the viral chart tracking how long a task would take a human versus an AI. They get into why software work is the key signal, how human baselines and success thresholds are chosen, where autonomous systems still break down, and why fast capability gains are raising pressure on labs and investors.

Joel Becker, a METR researcher who builds time-horizon benchmarks, explains how METR measures the human-hours tasks models can do about half the time. He discusses recent models nearing task-suite saturation and why single tasks can swing estimates wildly. They talk about extending benchmarks to longer, messier tasks and the challenges of running AI-assisted programming studies.