MLOps.community MLOps Meetup #34: Streaming Machine Learning with Apache Kafka and Tiered Storage // Kai Waehner, Confluent
MLOps Meetup #34! This week, we talk to Kai Waehner about the beast that is Apache Kafka and how many different ways you can use it!
Join the Community: https://go.mlops.community/YTJoinIn
Get the newsletter: https://go.mlops.community/YTNewsletter
// Key takeaways:
-Kafka is much more than just messaging
-Kafka is the de facto standard for processing huge volumes of data at scale in real-time
-Kafka and Machine Learning are complementary for various use cases (including data integration, data processing, model training, model scoring, and monitoring)
// Abstract:
The combination of Apache Kafka, tiered storage, and machine learning frameworks such as TensorFlow enables you to build a scalable, reliable, but also simple infrastructure for all machine learning tasks using the Apache Kafka ecosystem and Confluent Platform. This discussion features a predictive maintenance use case within a connected car infrastructure, but the discussed components and architecture are helpful in any industry.
// Bio:
Kai Waehner is a Technology Evangelist at Confluent. He works with customers across the globe and with internal teams like engineering and marketing. Kai’s main area of expertise lies within the fields of Big Data Analytics, Machine Learning, Hybrid Cloud Architectures, Event Stream Processing, and Internet of Things. He is a regular speaker at international conferences such as Devoxx, ApacheCon, and Kafka Summit, writes articles for professional journals, and shares his experiences with new technologies on his blog: www.kai-waehner.de.
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Kai: contact@kai-waehner.de / @KaiWaehner / LinkedIn (https://www.linkedin.com/in/megachucky/)
________Show Notes_______
Blogpost tiered storage
https://www.confluent.io/blog/streaming-machine-learning-with-tiered-storage/
Blogpost about using Kafka as a database
Example repo on github
Model serving vs embedded Kafka
https://www.confluent.io/blog/machine-learning-real-time-analytics-models-in-kafka-applications/
Istio blog post