Durable Execution and Modern Distributed Systems
MLOps.community
Intro
Demetrios and Johann open the episode and introduce the topic of durable execution and Temporal.
Johann Schleier-Smith is the Technical Lead for AI at Temporal Technologies, working on reliable infrastructure for production AI systems and long-running agent workflows.
Durable Execution and Modern Distributed Systems, Johann Schleier-Smith // MLOps Podcast #364
Join the Community: https://go.mlops.community/YTJoinIn
Get the newsletter: https://go.mlops.community/YTNewsletter
MLOps Merch: https://shop.mlops.community/
Big shoutout to @Temporalio for the support, and to @trychroma for hosting us in their recording studio
// Abstract
A new paradigm is emerging for building applications that process large volumes of data, run for long periods of time, and interact with their environment. It’s called Durable Execution and is replacing traditional data pipelines with a more flexible approach. Durable Execution makes regular code reliable and scalable.
In the past, reliability and scalability have come from restricted programming models, like SQL or MapReduce, but with Durable Execution, this is no longer the case. We can now see data pipelines that include document processing workflows, deep research with LLMs, and other complex and LLM-driven agentic patterns expressed at scale with regular Python programs.
In this session, we describe Durable Execution and explain how it fits in with agents and LLMs to enable a new class of machine learning applications.
// Related Links
~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~
Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore
Join our Slack community [https://go.mlops.community/slack]
Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]
Sign up for the next meetup: [https://go.mlops.community/register]
MLOps Swag/Merch: [https://shop.mlops.community/]
Connect with Demetrios on LinkedIn: /dpbrinkm
Connect with Johann on LinkedIn: /jssmith/
The AI-powered Podcast Player
