Evals, gating, and asynchronous checks

Ereli Eran is the Founding Engineer at 7AI, where he’s focused on building and scaling the company’s agentic AI-driven cybersecurity platform — developing autonomous AI agents that triage alerts, investigate threats, enrich security data, and enable end-to-end automated security operations so human teams can focus on higher-value strategic work.

Software Engineering in the Age of Coding Agents: Testing, Evals, and Shipping Safely at Scale // MLOps Podcast #361 with Ereli Eran, Founding Engineer at 7AI

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

MLOps GPU Guide: https://go.mlops.community/gpuguide

// Abstract

A conversation on how AI coding agents are changing the way we build and operate production systems. We explore the practical boundaries between agentic and deterministic code, strategies for shared responsibility across models, engineering teams, and customers, and how to evaluate agent performance at scale. Topics include production quality gates, safety and cost tradeoffs, managing long-tail failures, and deployment patterns that let you ship agents with confidence.

// Bio

Ereli Eran is a founding engineer at 7AI, where he builds agentic AI systems for security operations and the production infrastructure that powers them. His work spans the full stack - from designing experiment frameworks for LLM-based alert investigation to architecting secure multi-tenant systems with proper authentication boundaries. Previously, he worked in data science and software engineering roles at Stripe, VMware Carbon Black, and was an early employee of Ravelin and Normalyze.

// Related Links

Website: https://7ai.com/

Coding Agents Conference: https://luma.com/codingagents

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Ereli on LinkedIn: /erelieran/

Timestamps:

[00:00] Language Sensitivity in Reasoning

[00:25] Value of Claude Code

[01:54] AI in Security Workflows

[06:21] Agentic Systems Failures

[12:50] Progressive Disclosure in Voice Agents

[16:39] LLM vs Classic ML

[19:44] Hybrid Approach to Fraud

[25:58] Debugging with User Feedback

[33:52] Prompts as Code

[42:07] LLM Security Workflow

[45:10] Shared Memory in Security

[49:11] Common Agent Failure Modes

[53:34] Wrap up