What's AI Podcast by Louis-François Bouchard Building LLM Apps & the Challenges that come with it. The What's AI Podcast Episode 16: Jay Alammar
7 snips
Jun 27, 2023 AI Snips
Chapters
Transcript
Episode notes
Rerank Results With LLM Scorers
- Rerank search results by scoring query-document relevance with an LLM classifier.
- Use the reranker score alone or combine it with other signals like recency or proximity.
Validate LLM Value Before Adoption
- Evaluate if an LLM improves a concrete metric over existing baselines before adopting it.
- Focus on user pain first and prototype with hosted models rather than training infrastructure yourself.
Keep Sensitive Data In Your Cloud
- For private data, deploy models in your virtual private cloud or use managed VPC options.
- This keeps sensitive data inside your infrastructure while running model endpoints close to it.