MLOps.community LLMs at Scale: Infrastructure That Keeps AI Safe, Smart & Affordable // Marco Palladino// # 341
53 snips
Oct 7, 2025 Marco Palladino, CTO and co-founder of Kong, dives into the complexities of AI infrastructure. He discusses the importance of building AI gateways to enforce governance and security as technology evolves. The conversation touches on the role of agentic workloads and the challenges of MCP servers. Marco speculates on how agents could transform user interactions and even SEO dynamics. He also highlights real-world applications across industries and shares insights on product development strategies. Prepare for an enlightening exploration of AI's future!
AI Snips
Chapters
Transcript
Episode notes
Auto-Generate MCP Servers From REST APIs
- Generating MCP servers from existing REST APIs removes heavy developer work and speeds adoption.
- A catalog of MCP servers helps agents discover and use integrations more quickly.
MCP Security Is Unresolved
- Security and authorization across many MCP servers remains an open problem for the industry.
- The MCP spec is evolving (e.g., OAuth) and authorization across servers must be solved for safe impersonation.
Latency Could Become Marketing
- In an agentic world, operational metrics like latency may replace loud advertising as competitive advantage.
- Agents will prefer services that finish tasks faster, so optimize performance to win agentic business.