Software Engineering Daily New Relic and Agentic DevOps with Nic Benders
29 snips
Apr 14, 2026 
Nic Benders, Chief Technology Strategist at New Relic with 16 years there, explains the shift from instrumentation to AI-driven observability. He discusses combining statistical anomaly detection with LLMs to surface meaningful signals, tackling alert fatigue via automated triage and self-healing, and the new challenges of observing AI systems with metrics like token usage, answer quality, and cost.
AI Snips
Chapters
Books
Transcript
Episode notes
Observability's Three Era Shift
- Observability evolved from instrumentation to data platforms to an intelligence era that surfaces what to ask of massive datasets.
- Nic Benders says dashboards/alerts are insufficient now and tools must tell users which questions and signals matter automatically.
Mix Statistics ML And LLMs For Observability
- AI in observability blends three technique classes: statistics, traditional ML, and neural-net LLMs, each suited to different jobs.
- Nic explains static math handles baselines, ML tunes hyperparameters, and neural nets (transformers) enable current AI capabilities.
Preprocess With Stats Before Calling LLMs
- Use statistical anomaly detection to narrow petabytes of data, then feed relevant time ranges and context into LLMs for reasoning.
- Nic recommends structuring data temporally and spatially and combining graph relationships before invoking large models.
