Diagnose retrieval before blaming the model
When answers are wrong, attention goes to the model and the prompt. Frequently the model performed correctly on the material it was given, and the material was wrong. Before tuning anything generative, look at what retrieval actually returned for the failing queries. It is a fast check and it redirects a surprising proportion of investigations.
What dense retrieval is bad at
Embeddings capture meaning, which is precisely why they blur exact tokens. Error codes, SKUs, invoice numbers, surnames, version strings, acronyms — semantically these are near-identical to their neighbours, so a query for one happily returns another.
This is the failure users notice most, because searching for a specific identifier and receiving a similar-looking different one reads as broken in a way that a vaguely mediocre answer does not.
What sparse retrieval is bad at
Keyword search handles exact tokens correctly and fails on paraphrase. A user asking about 'turning off notifications' does not match a document about 'disabling alerts', and no amount of tuning fixes vocabulary mismatch.
The two methods fail in complementary directions, which is the entire argument for running both.
Reranking earns its latency
Combining two result sets creates a new problem: ordering. A reranking pass over the merged candidates is usually the highest-value single addition to a retrieval pipeline, because it is the step that decides what actually reaches the context window.
It costs latency and it is generally worth it, particularly if the alternative is stuffing more marginal documents into the prompt — which is both slower and worse.
Chunking decides the ceiling
No retrieval strategy recovers from chunking that destroyed the unit of meaning. Splitting a table from its header, a clause from its heading, or a function from its signature produces fragments that are individually retrievable and collectively useless.
The corpus decides the strategy. Prose, tabular data, code and transcripts each want different treatment, and the default settings in a tutorial are calibrated for none of them specifically. This is worth an afternoon of actually looking at the documents before choosing.