LLM Foundations
Fine-tuning vs. RAG
When to train your model and when to retrieve context.
Interview: High - Critical architecture decision
The Two Approaches to Customizing LLMs
Decision Framework
When you need an LLM to work with your specific data, you have two main strategies. Choosing wrong can waste months and hundreds of thousands of dollars.
Fine-tuning
Updating the model's weights on your specific dataset. The model "learns" your patterns permanently.
- Best for: Changing the model's behavior, style, or format. Teaching it domain-specific jargon.
- Cost: High upfront (training compute), low per-query.
- Data staleness: Frozen at training time — doesn't know about changes after training.
- Examples: Making GPT respond in your brand's tone, teaching medical terminology.
RAG (Retrieval-Augmented Generation)
Keep the base model unchanged. Instead, retrieve relevant documents at query time and inject them into the prompt as context.
- Best for: Knowledge-intensive tasks where data changes frequently.
- Cost: Low upfront, slightly higher per-query (retrieval + longer prompts).
- Data freshness: Always up-to-date — just update your document store.
- Examples: Customer support bots, internal documentation search, legal research.
RAG Architecture Diagram
Decision Framework
| Factor | Fine-tuning | RAG |
|---|---|---|
| Data changes frequently | Bad fit | Great fit |
| Need specific output format | Great fit | Possible |
| Limited budget | Expensive | Cheaper |
| Need factual accuracy | Hallucination risk | Grounded |
| Latency-critical | Direct inference | Retrieval adds latency |
Use Cases
Internal knowledge base chatbots (RAG)
Brand voice customization (Fine-tuning)
Legal document analysis with ever-changing regulations (RAG)
Code generation in a proprietary framework (Fine-tuning)
Common Mistakes
Fine-tuning when RAG would work — fine-tuning doesn't reliably add new factual knowledge
Using RAG without proper chunking strategy — garbage in, garbage out
Not evaluating retrieval quality separately from generation quality
Choosing based on hype rather than actual requirements analysis