Kubernetes Bytes

In this episode of the Kubernetes Bytes podcast, Bhavin sits down with Kai-Hsun Chen, Software Engineer at Anyscale and maintainer of the KubeRay project. The discussion focuses on how the open source Ray project can help organizations use a single tool for data prep, model training, fine tuning and model serving workflows, both for their predictive AI and generative AI models. The discussion also dives into the KubeRay project and how it provides three different Kubernetes CRDs for Data Scientists to deploy Ray clusters on demand.   Check out our website at https://kubernetesbytes.com/  Cloud Native News: https://azure.github.io/AKS/2024/08/23/fine-tuning-language-models-with-kaito https://orca.security/resources/blog/kubernetes-testing-environment/ https://www.redhat.com/en/about/press-releases/red-hat-openstack-services-openshift-now-generally-available  Show links: Kai's LinkedIn: https://www.linkedin.com/in/kaihsun1996/ KubeRay doc: https://docs.ray.io/en/latest/cluster/kubernetes/index.html Ray Summit registration: https://raysummit.anyscale.com/flow/anyscale/raysummit2024/reg/createaccount (code: KaiHsunC15) KubeRay repository: https://github.com/ray-project/kuberay Ray repository: https://github.com/ray-project/ray Ray Slack workspace: https://docs.google.com/forms/d/e/1FAIpQLSfAcoiLCHOguOm8e7Jnn-JJdZaCxPGjgVCvFijHB5PLaQLeig/viewform  Timestamps:  00:02:40 Cloud Native News  00:07:20 Interview with Kai  00:49:15 Key takeaways

In this episode of the Kubernetes Bytes podcast, Bhavin sits down with Alex Lines and Vara Bonthu from AWS to talk about the Data on EKS project. The discussion dives into why AWS decided to build the Data on EKS project and provide patterns for EKS customers to use to deploy data platforms, machine learning and GenAI tools on EKS clusters. They talk about what's included and what's not included with each of these patterns and whats coming down the line.   Check out our website at https://kubernetesbytes.com/  Cloud Native News:  https://kubernetes.io/blog/2024/08/16/kubernetes-1-31-prevent-persistentvolume-leaks-when-deleting-out-of-order/ https://kubernetes.io/blog/2024/08/16/matchlabelkeys-podaffinity/ https://kubernetes.io/blog/2024/08/15/kubernetes-1-31-volume-attributes-class/ https://roadmap.vcluster.com/changelog/vcluster-v020-ga https://www.cloudbees.com/blog/cloudbees-acquires-launchable-to-enable-development-teams-to-iterate-faster?Product=Launchable&Tag=Blog%2CAI%2CLaunchable%20Update  Show links: https://awslabs.github.io/data-on-eks/ https://www.youtube.com/watch?v=G9aNXEu_a8k  https://github.com/awslabs/data-on-eks  https://www.linkedin.com/in/alex-lines-aws/  https://www.linkedin.com/in/varaprofile/  Timestamps:  00:01:45 Cloud Native News  00:12:15 Interview with Alex and Vara  00:58:21 Key takeaways

Sachi Desai, a Product Manager specializing in AI technologies, and Paul Yu, a Senior Cloud Advocate at Microsoft, dive into the KAITO project for deploying open source LLM models on Kubernetes. They discuss how KAITO simplifies running AI applications alongside LLM models and enables users to bring and fine-tune their own models. The conversation highlights innovative techniques like LoRa and Q-LoRa for efficient model training. Additionally, they emphasize community engagement's role in enhancing AI model deployment and future capabilities.

Danielle Cook, VP of Marketing at appCD and Co-chair of the CNCF Cartografos Working Group, dives into the business case for adopting cloud-native technologies. She discusses how technical contributors can align their strategies with business goals, emphasizing growth and efficiency. The conversation explores the Cloud Native Maturity Model, addressing the challenges organizations face during adoption. Danielle also highlights the importance of effective communication tailored for different executive roles to build a compelling case for modernization.

Brandon Jacobs, Infrastructure architect at Coreweave, discusses how Coreweave uses Kubernetes to build an AI hyperscaler. They cover managing Day 0 & 2 operations for AI labs, lessons learned, and best practices for a Kubernetes based cloud. Topics include leveraging bare metal Kubernetes for GPU workloads, storage options for AI labs, observability, monitoring, handling CVEs, and customer cluster support.

Andy Grimes discusses OpenShift's AI Landscape with insights on MLOps, AI model development, and accelerating model deployment using OpenShift AI. The discussion covers local experimentation, governance tools for AI, and the collaboration between IBM and Red Hat in the AI space.

Bernie Wu from Memverge discusses training ML models on Kubernetes, including cost-saving tips with spot instances, efficient model checkpoints, hot restarts, and reclaiming GPU resources. They delve into topics like DAG phases, transparent checkpointing, and GPU snapshotting for AI workloads.

Christian Posta, VP and Global Field CTO at Solo.io, discusses the evolution of service mesh technologies from Linkerd to istio implementations, connecting application components outside Kubernetes. They explore shared responsibilities between developers and platform engineers, using internal developer platforms for service mesh. Topics include on-prem and cloud flexibility, challenges in cloud-based development, and importance of automation and observability in infrastructure architectures.

In this episode of the Kubernetes Bytes podcast, Ryan and Bhavin talk to Torsten Steinbach - VP, Chief Architect for Analytics & AI at EDB about all things Vector Databases, Postgres, and why Data is important for building AI platforms. The discussion dives into how vector databases are different than relational databases and why using Postgres extensions helps organizations use their existing data for AI applications.  Check out our website at https://kubernetesbytes.com/  Kubernetes Community Days (KCD) in New York City on May 22nd, use the promo code “KUBERNETESBYTES” to get a 10% discount on your registration fees!   Episode Sponsor: Nethopper  Learn more about KAOPS:  @nethopper.io  For a supported-demo:  info@nethopper.io  Try the free version of KAOPS now!    https://mynethopper.com/auth Cloud Native News:   https://www.reuters.com/markets/deals/ibm-nearing-buyout-deal-hashicorp-wsj-reports-2024-04-23/ https://www.wiz.io/blog/wiz-acquires-gem-security-to-reinvent-threat-detection-in-the-cloud https://techcrunch.com/2024/04/18/wiz-is-in-talks-to-buy-lacework-for-150-200m-security-firm-was-last-valued-at-8-3b/ https://www.prnewswire.com/news-releases/coreweave-secures-1-1-billion-in-series-c-funding-to-drive-the-next-generation-of-cloud-computing-for-the-future-of-ai-302133328.html https://kubernetes.io/blog/2024/04/17/kubernetes-v1-30-release  https://dok.community/blog/become-a-data-on-kubernetes-in-2024-ambassador/   Show Links:  https://www.enterprisedb.com/news/edb-acquires-splitgraph  https://www.enterprisedb.com/resources/events https://www.enterprisedb.com/  Timestamps:  00:03:22 Cloud Native News  00:17:45 Interview with Torsten  00:57:00 Key takeaways

Join Bhavin Shah and Ryan Wallner for a recap of announcments and news from KubeCon Paris 2024.Kubernetes Community Days (KCD) in New York City on May 22nd, use the promo code “KUBERNETESBYTES” to get a 10% discount on your registration fees!Nethopper Learn more about KAOPS:  @nethopper.io  For a supported-demo:  info@nethopper.io Try the free version of KAOPS now!   https://mynethopper.com/authNews https://about.gitlab.com/blog/2024/03/20/oxeye-joins-gitlab-to-advance-application-security-capabilities/ https://www.redhat.com/en/blog/unveiling-red-hat-openshift-415 https://developer.nvidia.com/blog/nvidia-nim-offers-optimized-inference-microservices-for-deploying-ai-models-at-scale/ https://www.acorn.io/resources/blog/our-new-focus-developing-an-llm-app-platform-based-on-gpt-script-technology?fromOther=true https://loft.sh/blog/deliver-secure-kubernetes-multi-tenancy-with-new-vcluster-in-rancher-integration/ https://www.observeinc.com/blog/stepping-on-the-gas/ https://thenewstack.io/kubecost-2-2-covers-carbon-cost-monitoring-and-more/ https://thenewstack.io/ovhcloud-unveils-roadmap-to-take-on-hyperscalers-from-europe/ https://www.suse.com/c/meet-rancher-prime-3-0/ https://www.suse.com/c/suse-releases-edge-3-0-highly-validated-edge-optimized-stack/ https://www.fermyon.com/blog/introducing-spinkube-fermyon-platform-for-k8s  https://www.cncf.io/blog/2024/03/19/announcing-the-ai-working-groups-new-cloud-native-artificial-intelligence-whitepaper/  https://github.com/Azure/kaito  https://azure.microsoft.com/en-us/updates/public-preview-kubernetes-ai-toolchain-operator-kaito-addon-for-aks/ https://cloudnativenow.com/features/solo-io-delivers-on-cilium-support-promise-for-gloo-networks/  https://docs.solo.io/gloo-network/latest/about/overview/  https://github.com/kosmos-io/kosmos  https://gateway.envoyproxy.io/blog/2024/03/14/announcing-envoy-gateways-1.0-release/  https://newrelic.com/press-release/20240319  https://siliconangle.com/2024/03/29/aviatrix-revolutionizes-networking-security-distributed-cloud-firewall-kubernetes-kubeconeu/