RAG Frameworks
Frameworks for building retrieval-augmented generation applications.
Comparison
| Framework | Best For | Learning Curve | Flexibility |
|---|---|---|---|
| LangChain | Agents, chains, tools | Steeper | Highest |
| LlamaIndex | Data indexing, simple RAG | Gentle | Medium |
| Sentence Transformers | Custom embeddings | Low | High |
LangChain
Orchestration framework for building complex LLM applications.
Core concepts:
- Chains: Sequential operations (retrieve → prompt → generate)
- Agents: LLM decides which tools to use
- LCEL: Declarative pipeline syntax with
|operator - Retrievers: Abstract interface to vector stores
Strengths: Rich ecosystem, many integrations, agent capabilities Limitations: Abstractions can be confusing, rapid API changes
Key concept: LCEL (LangChain Expression Language) for composable pipelines.
LlamaIndex
Data framework focused on connecting LLMs to external data.
Core concepts:
- Documents → Nodes: Automatic chunking and indexing
- Index types: Vector, keyword, tree, knowledge graph
- Query engines: Retrieve and synthesize answers
- Chat engines: Stateful conversation over data
Strengths: Simple API, great for document QA, data connectors Limitations: Less flexible for complex agent workflows
Key concept: "Load data, index it, query it" - simpler mental model than LangChain.
Sentence Transformers
Generate high-quality embeddings for semantic similarity.
Popular models:
| Model | Dimensions | Quality | Speed |
|---|---|---|---|
| all-MiniLM-L6-v2 | 384 | Good | Fast |
| all-mpnet-base-v2 | 768 | Better | Medium |
| e5-large-v2 | 1024 | Best | Slow |
Key concept: Bi-encoder architecture - encode query and documents separately, compare with cosine similarity.
RAG Architecture Patterns
| Pattern | Description | When to Use |
|---|---|---|
| Naive RAG | Retrieve top-k, stuff in prompt | Simple QA |
| Parent-Child | Retrieve chunks, return parent docs | Context preservation |
| Hybrid Search | Vector + keyword search | Better recall |
| Re-ranking | Retrieve many, re-rank with cross-encoder | Higher precision |
| Query Expansion | Generate variations of query | Ambiguous queries |
Decision Guide
| Scenario | Recommendation |
|---|---|
| Simple document QA | LlamaIndex |
| Complex agents/tools | LangChain |
| Custom embedding pipeline | Sentence Transformers |
| Production RAG | LangChain or custom |
| Quick prototype | LlamaIndex |
| Maximum control | Build custom with Sentence Transformers |
Resources
- LangChain: https://python.langchain.com
- LlamaIndex: https://docs.llamaindex.ai
- Sentence Transformers: https://sbert.net