RAG Architecture
Ground the model in facts it wasn't trained on — or that changed since it was — without fine-tuning.
Intermediate
# ingestion (offline, run once / on updates)
documents -> chunk -> embed -> store in a vector index
# query time (per user request)
user_query -> embed -> similarity search against the index -> top-k chunks
-> insert chunks into the prompt as context -> generate answer
| RAG | Fine-tuning on the same documents | |
|---|---|---|
| Update latency for new/changed docs | Re-index the changed doc; minutes | Retrain; hours-days |
| Attribution / citing sources | Natural — you know exactly which chunks were retrieved | Hard — knowledge is baked into weights with no traceable source |
| Reduces hallucination on the covered facts? | Yes, when the right chunk is retrieved | Not reliably — fine-tuning teaches style/format more than it teaches new facts to recall faithfully |
| Failure | Usual cause |
|---|---|
| Right document exists, wrong/no chunk retrieved | Poor chunking boundaries or an embedding model mismatched to the domain — see Chunking & Retrieval Strategies |
| Retrieved chunks are relevant but answer is still wrong | Too many/irrelevant chunks crowding the context, or important chunk placed where "lost in the middle" hurts it — see Context Windows |
| Model ignores retrieved context and answers from parametric memory anyway | Prompt doesn't clearly instruct the model to prefer the provided context over its own knowledge, especially when they conflict |