Detection at Scale

Michael Sinno, Director of Detection & Response at Google with two decades shaping large-scale security, describes automating operations across 7 trillion logs daily. He talks about evolving from AI-assisted to autonomous detection, fine-tuned models and overseer agents for quality, modular pluggable detection agents, and integrating Sec-Gemini with Timesketch for forensic patterns humans miss.

James Nettesheim, CISO & Head of Enterprise Technology at Block, a detection engineering and incident response leader who co-built Goose and helped design the Model Context Protocol. He discusses building Goose and MCP, hardening agents against prompt injection, democratizing detection writing with natural language, Binary Intelligent Triage with 99.9% accuracy, and principled, data-safety driven AI rollouts.

Ryan Glynn, Staff Security Engineer at Compass, has a practical AI implementation strategy for security operations. His team built machine learning models that removed 95% of on-call burden from phishing triage by combining traditional ML techniques with LLM-powered semantic understanding. He also explores where AI agents excel versus where deterministic approaches still win, why tuning detection rules beats prompt-engineering agents, and how to build company-specific models that solve your actual security problems rather than chasing vendor promises about autonomous SOCs.Topics discussed:Language models excel at documentation and semantic understanding of log data for security analysis purposesUsing LLMs to create binary feature flags for machine learning models enables more flexible detection engineeringAgentic SOC platforms sometimes claim to analyze data they aren't actually querying accurately in practiceTuning detection rules directly proves more reliable than trying to prompt-engineer agent analysis behaviorIntent classification in email workflows helps automate triage of forwarded and reported phishing attempts effectivelyCustom ML models addressing company-specific burdens can achieve 95% reduction in analyst workload for targeted problemsAlert tagging systems with simple binary classifications enable better feedback loops for AI-assisted detection tuningContext gathering costs in security make efficiency critical when deploying AI agents across diverse data sourcesQuery language complexity across SIEM platforms creates challenges for general-purpose LLM code generation capabilitiesExplainable machine learning models remain essential for security decisions requiring human oversight and accountabilityListen to more episodes: Apple Spotify YouTubeWebsite

Mike Vetri, Sr. Director of Security Operations at Veeva Systems, reflects on transforming SOC investigations through AI-powered data aggregation and building threat operations teams with the analytical mindset required for proactive defense. Mike introduces the C3 Matrix framework for prioritizing security efforts across centers of gravity, crown jewels, and capability enablers, and explains the seven Ds of cyber defense from discovery through deception operations.  Drawing from 10+ years of Air Force cyber intelligence experience, Mike details why threat operations requires fundamentally different system-two thinking than detection engineering, and how this discipline shift moves organizations from reactive firefighting to proactive threat anticipation. He covers practical examples of AI cutting investigation time by aggregating data from multiple tools, the importance of defense in personnel for operational resilience, and strategies for preventing analyst burnout while maintaining effective security operations.  Topics discussed: How AI transforms insider threat investigations by aggregating workstation logs, browsing history, and DLP alerts into single queries The C3 Matrix framework prioritizes security controls across centers of gravity, crown jewels, and capability enablers based on organizational impact and recoverability Why threat operations requires system-two analytical thinking fundamentally different from the engineering mindset The seven Ds of cyber defense: discover, detect, deny, disrupt, degrade, destroy, and deception operations for comprehensive threat mitigation How deception operations provide the most accurate intelligence by studying adversary behavior in controlled environments The distinction between threat intelligence and threat operations, and why mature SOCs need teams focused on proactive defense strategies Defense in personnel ensures multiple team members can handle each security capability, preventing single points of failure Time-sensitive investigation scenarios where AI delivers maximum ROI by eliminating the need to manually query dozens of security tools The evolution of cyber threats from technical attacks to psychological warfare using AI to challenge human judgment and decision-making Why security culture must extend beyond traditional boundaries as AI-powered threats increasingly target HR processes, financial operations, and business functions Listen to more episodes:  Apple  Spotify  YouTube Website

Gary Hunter, Head of Security Operations at Trustpilot, shares insights from his journey in building a robust security team. He emphasizes treating AI agents like interns, with strict guardrails to ensure safe operations. Discover how competitive prompt testing fine-tunes AI performance and how democratizing learning fosters cultural buy-in for AI adoption. Gary also reveals strategies for multimodal brand protection and why constraints can spark creativity in security practices. Tune in for valuable tips on enhancing detection and empowering team members.

Vjaceslavs Klimovs, a Distinguished Engineer at CoreWeave with a rich background from Google and Snap, dives into the world of security in AI infrastructure. He highlights that 40% of security initiatives lack connection to solid threat models, revealing critical gaps in the industry. Observability is emphasized as the bedrock of any security program, and Vjaceslavs argues for a shift towards detection over prevention. He discusses the role of AI in exposing security flaws and building robust detection platforms that embrace a new era in cybersecurity.

Ken Bowles, Director of Security Operations at GreenSky, boasts 15 years of expertise in healthcare and financial services security. He dives into practical strategies for prioritizing security crown jewels and managing cloud permissions. Ken reveals how AI significantly speeds up analyst investigations, reducing time from 30 minutes to mere seconds. He emphasizes the need for regular audits of security controls to prevent silent failures. Additionally, he discusses the importance of training analysts with AI support and the evolving role of the MITRE framework in modern security operations.

Tyler Martin, Senior Director of Enterprise Security Engineering & Operations at FanDuel, reflects on revolutionizing security operations by replacing traditional analyst tiers with security engineers supported by custom AI agents. Tyler shares the architecture behind SAGE, FanDuel's phishing automation system, and explains how his team progressed from human-in-the-loop validation to fully autonomous triage through bronze-silver-gold maturity stages.  The conversation explores practical challenges like context enrichment, implementing user personas connected to IDP and HRIS systems, and choosing between RAG versus CAG models for knowledge augmentation. Tyler also discusses shifts in detection strategy, arguing for leaner detection catalogs with just-in-time, query-based rules over maintaining point-in-time codified detections that no longer address active risks. Topics discussed: Restructuring security operations teams to include only security engineers while AI agents handle traditional level 1-3 triage work. Building Security Analysis and Guided Escalation, an AI-powered phishing automation system that reduced manual ticket volume. Implementing bronze-silver-gold maturity stages for AI triage: manual validation, automated closures with oversight, and full autonomous operations. Enriching AI agents with organizational context through connections to IDP systems, HRIS platforms, and user behavior analytics. Creating user personas that encode access patterns, permissions, security groups, and typical behaviors to improve AI decision-making accuracy. Designing incident response automation that spins up Slack channels, Zoom bridges, recordings, and comprehensive documentation through simple commands. Eliminating 90% of missing PIR action items through automated documentation capture and stakeholder tagging in Confluence. Shifting detection strategy from maintaining large MITRE-mapped catalogs to just-in-time query-based rules written by AI agents. Balancing signal volume and enrichment data against inference costs while avoiding context rot that degrades LLM performance. Evaluating RAG versus CAG models for knowledge augmentation and exploring multi-agent architectures with supervisory oversight layers.  Listen to more episodes:  Apple  Spotify  YouTube Website

George Werbacher, Head of Security Operations at Live Oak Bank, reviews the practical realities of implementing AI agents in security operations, sharing his journey from exploring tools like Cursor and Claude Code to building custom agents in-house. He also reflects on the challenges of moving from local development to production-ready systems with proper durability and retry logic. The conversation explores how AI is changing the security analyst role from alert analysis to deeper investigation work, why SOAR platforms face significant disruption, and how MCP servers enable natural language interactions across security tools. George offers pragmatic advice on cutting through AI hype, emphasizing that agents augment rather than replace human expertise while dramatically lowering barriers to automation and query language mastery. Through technical insights and leadership perspective, George illuminates how security teams can embrace AI to improve operational efficiency and mean time to detect without inflating budgets, while maintaining the critical human judgment that effective security demands. Topics discussed: Understanding AI's role in augmenting security analysts rather than replacing them, shifting roles toward investigation and threat hunting. Building custom AI agents using Python and exploring frameworks like LangChain to solve specific SecOps use cases. Managing moving agents from local development to production, including retry logic, failbacks, and durability requirements. Implementing MCP servers to enable natural language interactions with security tools, eliminating the need to learn multiple query languages. Navigating AI hype by focusing on solving specific problems and understanding what agents can realistically accomplish. Predicting SOAR platform disruption as agents take over enrichment, orchestration, and response with simpler automation approaches. Removing platform barriers by enabling analysts to use natural language rather than mastering specific tools or query languages. Exploring context management, prompt engineering, and conversation history techniques essential for building effective agentic systems. Adopting tools like Cursor and Claude Code to empower technical security professionals without deep coding backgrounds.  Listen to more episodes:  Apple  Spotify  YouTube Website

Andrew Casazza, AVP of Cyber Security Operations at Ochsner Health, explores how healthcare organizations navigate FDA-approved medical devices running on legacy operating systems, implement AI-powered security tools while maintaining HIPAA compliance, and respond to threats that now move from initial compromise to malicious action in seconds rather than hours.  Andrew gives Jack his insights on building effective security programs in heavily regulated industries, emphasizing the importance of visibility, automation with guardrails, and keeping humans in the loop for critical decisions while leveraging AI to handle the speed and scale of modern threats. Topics discussed: Unique security challenges in healthcare environments where medical devices run on legacy operating systems that cannot be easily updated. Strategies for monitoring and securing systems that cannot have traditional security agents installed due to FDA regulations and medical certification requirements. Leveraging AI and automation in security operations while navigating HIPAA regulations and protecting patient data from external training models. Implementing human-in-the-loop approaches where AI performs initial analysis and triage while escalating critical decisions to human analysts. Understanding the privacy and compliance implications of AI tools that may use customer data for model training and improvement. The dramatic reduction in threat-actor dwell time from hours or days to minutes or seconds. Building effective SOAR automation playbooks to handle repetitive cases and reduce noise while focusing attention on bigger threats. Establishing appropriate guardrails for AI-powered security tools to prevent unintended consequences while enabling automated response capabilities. The importance of being curious and maintaining broad knowledge across multiple domains to become more effective. Listen to more episodes:  Apple  Spotify  YouTube Website