Chunking & Retrieval Strategies

fixed-size chunking

def chunk_fixed(text, size=512, overlap=64):
    chunks = []
    start = 0
    while start < len(text):
        chunks.append(text[start:start + size])
        start += size - overlap        # overlap keeps context from being severed mid-idea
    return chunks

Simple and fast, but indifferent to structure — a fixed-size window will happily cut a chunk boundary through the middle of a sentence, a code block, or a table row, which then embeds as a fragment that doesn't represent any complete idea well.

semantic / structure-aware chunking

Split on natural boundaries instead of a fixed character count — headings, paragraphs, function/class definitions for code, or points where consecutive sentence embeddings show a large semantic jump (indicating a topic shift). More expensive to compute than fixed-size, but each resulting chunk is more likely to be a coherent, retrievable unit of meaning.

Content type	Natural boundary to chunk on
Markdown/prose docs	Headings, then paragraphs within a section
Source code	Function/class definitions, not arbitrary line counts
Tables	Row-preserving splits, with headers repeated in each chunk
Transcripts	Speaker turns or topic-shift boundaries

hybrid search: vector + keyword

Vector similarity finds semantically related text even without shared words, but is comparatively weak at exact-match lookups — a product SKU, an error code, an acronym — where a plain keyword match is both easier and more reliable. Hybrid search runs both a vector query and a keyword/full-text query (see Full-Text Search) and merges the ranked results, usually with reciprocal rank fusion, to get the strengths of both.

score = (alpha * normalize(vector_score)
       + (1 - alpha) * normalize(keyword_score))
# alpha tuned per corpus -- exact-match-heavy corpora (logs, code) skew keyword-weighted

chunk size trade-offs

	Small chunks	Large chunks
Retrieval precision	Higher — less irrelevant text per chunk	Lower — more chance of mixed-relevance content
Context loss	Higher risk — a chunk may lack surrounding context needed to make sense	Lower — more self-contained
Chunks needed to answer a broad question	More, filling the context window faster	Fewer

There's no universally correct size — it's an empirical tuning knob, evaluated the way anything else in a RAG pipeline should be. See Reranking & RAG Evaluation.

where to go from here

RAG Architecture — where chunking fits in the larger pipeline.
Reranking & RAG Evaluation — improving on raw retrieval results, and measuring the whole pipeline.
Full-Text Search — the keyword half of hybrid search.