By submitting, you consent to our use of your data. Privacy Policy.
Category
AI Agents
Share the article
Ask an AI agent to "find order SKU-2847-B" and a pure semantic search will often miss it. Vector search matches meaning, and an order number has no meaning to embed. It's a string. That single gap is why enterprise agents doing real back-office work should default to hybrid search, which runs keyword and vector retrieval side by side, rather than semantic search alone. On mixed-query benchmarks, hybrid retrieval lifts ranking quality about 26–31% over dense-vector search (per Atlan's Hybrid RAG analysis). Here is when semantic wins, when keyword wins, and why hybrid is the safe default when your agent lives in a world of exact identifiers.
What semantic search actually does, and where it quietly fails
Semantic search turns text into vector embeddings, mathematical representations that capture meaning. A query for "car repairs" returns documents about "automotive maintenance" and "vehicle servicing" even though they share no words, because those concepts sit near each other in embedding space. That is the whole point, and it is genuinely useful. Semantic search handles paraphrase, synonyms, and "how do I..." questions that keyword search can't touch.
The failure mode is specific and easy to miss in a demo. Embeddings represent meaning, so anything without semantic meaning gets represented poorly. Error code "OOM-2024-047," part number "SKU-2847-B," an invoice ID, a policy number, a person's name spelled one exact way. These are strings, not concepts. As Redis puts it in its breakdown of the two approaches, "error codes don't have semantic meaning in the same way natural language does," so semantic search "might miss it entirely."
This is the trap for enterprise agents. In a clean demo over a curated FAQ, semantic search looks perfect. Then it goes to production over real tickets, invoices, and CRM records full of exact identifiers, and it starts surfacing plausible-looking near matches instead of the exact record. It fails quietly, because it always returns something.
Worse, the failure is hard to catch in testing. Semantic search doesn't throw an error when it misses an ID. It confidently returns the nearest neighbor, which often looks right at a glance. The gap only shows up when someone checks the exact string against the record and finds they don't match. By then the agent may have already acted on it.
What keyword search still does better
Keyword search, or lexical search, matches the actual words you type. It runs on an inverted index and ranks with BM25, a probabilistic function that has been the standard for decades. It is fast, deterministic, and reproducible. The same query returns the same results every time, which matters for compliance and debugging.
Where keyword search wins is exactly where semantic search struggles. Unique identifiers: SKUs, model numbers, database IDs, medical codes, legal citations. Boolean logic and field filters, so an agent can ask for one exact value in one exact field. Scenarios where "close enough" is wrong and you need the literal string.
Its weakness is the mirror image of semantic's strength. Search "car repairs" and keyword search won't find "automotive maintenance," because those exact words aren't there. It has no concept of meaning. So neither approach is complete on its own, and that is the entire case for hybrid.
Semantic vs keyword vs hybrid, side by side
Approach | Best at | Where it fails | When to use |
|---|---|---|---|
Semantic (vector) | Meaning, paraphrase, synonyms, natural-language and "how do I" queries, multilingual matching | Exact identifiers, codes, SKUs, names; returns plausible near misses instead of the literal record | Knowledge-base Q&A, conceptual retrieval, conversational context |
Keyword (lexical / BM25) | Exact strings, IDs, codes, Boolean logic, field filters; fast, deterministic, reproducible | Synonyms and paraphrase; misses a match if the exact words aren't present | Lookups by identifier, compliance, structured filtering |
Hybrid (both, fused) | Covers both query types; catches the exact record and the conceptual match in one pass | More moving parts to tune; fusion weights need tuning to your data | Enterprise agents over mixed real-world data (the usual default) |
Hybrid search runs both retrievers in parallel and merges the results, most often with Reciprocal Rank Fusion (RRF), which combines the two ranked lists by rank rather than trying to reconcile incompatible score scales. On BEIR, Atlan reports hybrid models lifting nDCG@10 from 43.42 for BM25 alone to 52.59, a 9.17-point gain. Redis makes the same architectural point: production systems keep a vector index and an inverted index over the same documents and fuse the results, because the two failure modes are complementary.
Why hybrid is the right default for enterprise agents
The case is not that hybrid is universally superior on every query. It's that enterprise back-office work is exactly the mix where pure semantic search breaks.
Think about what an RPO or finance agent actually reads. Candidate IDs. Requisition numbers. Invoice numbers, PO numbers, GL codes. Error strings in a support ticket. Company names that have to match one exact legal entity, not a similar-sounding one. Every one of those is a string an embedding will smear into its nearest neighbors. When the agent needs the invoice numbered 44815 and semantic search hands it invoice 44851 because the surrounding text looked similar, that is not a rounding error. It's a wrong answer delivered with full confidence.
Hybrid search fixes this without giving up the natural-language strength. The keyword side nails the exact identifier. The vector side handles the paraphrased question wrapped around it. You stop choosing between "understands the question" and "finds the right record," and get both. That is why most production RAG and agentic workflows that handle real operational data land on hybrid rather than either extreme.
The usual objection is cost. Hybrid means a second index and a fusion step, so it looks heavier than a single vector store. In practice the overhead is small. Both indexes point at the same documents, so you are not duplicating content, and RRF fuses ranks without expensive score normalization. Redis and other production stores now ship hybrid as a single API call. Weigh that modest setup cost against the alternative: an agent that retrieves the wrong record and acts on it. In back-office work, a wrong invoice or the wrong candidate isn't a slightly worse search result. It's a bad action taken automatically.
When to pick each: a simple decision rule
You don't need to overthink this. Start from the data and the queries, not the technology.
Default to hybrid if your agent touches real operational data with any exact identifiers in it: IDs, SKUs, codes, invoice numbers, exact names. That is most enterprise work.
Pure semantic is fine only when queries are all natural language over conceptual content and exact-string matching genuinely doesn't matter. Think an internal policy Q&A bot with no IDs.
Pure keyword is enough when every query is a lookup by exact value and there is no paraphrase to handle. Think a strict record-retrieval-by-ID step.
Tune the fusion weights toward keyword when your users search with exact terms, toward semantic when they ask in full sentences. The weighting is a dial, not a fixed setting.
The tie-breaker: if you're unsure, use hybrid. The cost of adding a keyword index is small. The cost of an agent silently retrieving the wrong record in production is not.
Bad retrieval is how agents hallucinate
This is not just a search-quality debate. It is a reliability one. An agent generates its answer from whatever retrieval hands it. If retrieval returns the wrong record, the model answers faithfully from bad context and produces a confident, wrong result. Pick the wrong retrieval method for your data and you are manufacturing hallucinations upstream of the model, before the model ever runs.
That reframes the choice. Semantic versus hybrid isn't an optimization detail. It's a decision about whether your agent hands back the right record or a convincing near miss. In production, retrieval quality decides reliability more than the model choice does, which is also why the right architecture is model-agnostic. Reliability is engineered into how you retrieve and check, not bought by swapping the underlying model.
Beam's production deployments are built this way. A debt-collection agent classifying cases at 96% accuracy with under 2% regression isn't a story about a magic model. It's a story about grounding retrieval on the right data with the right method and checking the output. Get the retrieval layer right and the agent stays reliable. Get it wrong, and no model saves you.
The take
Semantic search is not the upgrade it looks like in a demo. For enterprise agents working over data full of exact identifiers, pure vector search quietly fails on the exact strings that matter most, and it fails without ever admitting it. Hybrid is the right default: keep the semantic understanding for paraphrase, add keyword matching for the IDs and codes, fuse the two. Reserve pure semantic for the narrow case of all-natural-language, no-identifier content. When in doubt, run both. Your agent is only as reliable as what it retrieves.
