Deep Papers RAG vs Fine-Tuning
Feb 8, 2024
This podcast explores the tradeoffs between RAG and fine-tuning for LLMs. It discusses implementing RAG in production, question and answer generation using JSON and LOM models, using GPT for test question generation in agriculture, evaluating relevance in email retrieval, and the use of RAG and fine-tuning for QA pair generation.
AI Snips
Chapters
Transcript
Episode notes
Traditional Tools Used For PDF Structure Extraction
- The authors used traditional PDF extraction tools rather than LLM-based structure extraction.
- They relied on libraries like GROBID to keep document structure in JSON form.
QA Generation Split Between LLM And RAG
- The paper generated questions from JSON-structured documents and then created answers via a RAG pipeline.
- GPT-4 was used to generate and evaluate QA content in multiple steps.
Inconsistent Fine-Tuning Methods Complicate Comparison
- They fine-tuned several open models (OpenLLaMA variants) with distributed training and used LoRA for GPT-4 due to cost.
- Inconsistent tuning methods across models add variables that complicate direct comparisons.