Large Language Models in Production Round-table Conversation

LLM in Production Round Table with Demetrios Brinkmann, Diego Oppenheimer, David Hershey, Hannes Hapke, James Richards, and Rebecca Qian.

// Abstract

Using LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event!

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksLLM in Production Survey: https://docs.google.com/forms/d/e/1FAIpQLSerEryK4xHEZTq0hSu-sVmBHilOzaT71BfCQgXe_uIRgIah-g/viewform

Virtual LLMs in Production Conference registration:

https://home.mlops.community/public/events/llms-in-production-conference-2023-04-13

Chinchilla papers:

https://paperswithcode.com/method/chinchilla, https://arxiv.org/abs/2203.15556

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Diego on LinkedIn: https://www.linkedin.com/in/diego/

Connect with David on LinkedIn: https://www.linkedin.com/in/david-hershey-458ab081/

Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hanneshapke/

Connect with James on LinkedIn: https://www.linkedin.com/in/james-richards-4baa73a7/

Connect with Rebecca on LinkedIn: https://www.linkedin.com/in/rebeccaqian/

Timestamps:

[00:00] Round table success to Virtual LLM in Production Conference on April 13th!

[00:18] Register for the Virtual LLM in Production Conference now!

[00:44] LLM in Production survey

[01:40] Lightning round of introduction of speakers

[04:34] Large Language Models definition

[09:17] What do we consider large?

[10:35] Thought process in use cases production

[14:30] LLM open source huge movements

[16:50] Problems with qualifications

[19:25] Production use cases frameworks directions

[25:25] Open-source language models tokenizer

[26:25] Language models democratization

[29:25] Three categories for LLMs in Production

[31:22] Latency at 2 levels

[33:27] Defining production

[34:57] Hitting the latency problems

[38:20] Fundamental latency barrier

[40:39] Latency use case requirement

[44:25] Costs and the use cases

[48:12] Product management involvement in costing

[49:38] LLMs Hallucination definition

[52:05] Building deterministic systems trust

[55:21] Wrap up