Machine Learning Street Talk (MLST)

AI Agents Can Code 10,000 Lines of Hacking Tools In Seconds - Dr. Ilia Shumailov (ex-GDM)

188 snips
Oct 4, 2025
Guest
Dr. Ilia Shumailov
Dr. Ilia Shumailov is a former DeepMind AI security researcher now focused on building security tools for AI agents. He delves into the unique challenges posed by AI agents operating 24/7, generating hacking tools at unprecedented speeds. Ilia emphasizes that traditional security measures fall short and discusses new adversarial threats, including prompt injection attacks. He also explores the risks of model collapse and the importance of fine-grained policies for AI behavior, warning that as AI evolves, its unpredictability could lead to significant security vulnerabilities.
Ask episode
AI Snips
Chapters
Transcript
Episode notes
ADVICE

Enforce Policies By Design With Symbolic Variables

  • Enforce data-flow and control-flow policies by translating user requests into a formal program before execution.
  • Keep secrets symbolic and check an external oracle before letting tools access sensitive values.
INSIGHT

Human-Centric Security Assumptions Break

  • Security assumptions built for humans collapse when agents replace people, removing deterrents like legal consequences.
  • We must design far finer-grained, precise access controls and transparency for agentic systems.
ANECDOTE

Agent Sent Unexpected Emails

  • Ilia recounts an agent that sent emails to unintended recipients while trying to complete a task.
  • The agent also attempted extra actions like pinging endpoints without explicit instruction.
Get the Snipd Podcast app to discover more snips from this episode
Get the app