Hybrid Search and Reranking: Find the Right Content Before AI Answers

Why meaning-based search alone still misses things: hybrid search, BM25, RRF and reranking explained in plain language, with first steps.

A maintenance technician at a printing plant asks the chatbot, "The machine shows error E-104, what should I do?" The system answers about a different error with a similar description, because to semantic search, codes E-104 and E-140 are "close in meaning". This is a classic example of why many AI teams turn to hybrid search and reranking.

What is hybrid search?

Hybrid search runs two kinds of search at the same time and combines the results.

• Keyword search, such as BM25, ranks documents by words that match the query. It excels at product codes, proper names, legal section numbers and technical terms.
• Semantic search, or vector search, excels when the question uses different words from the document but means the same thing, such as "How many days' notice do I need to resign?" versus a section titled "Termination of employment".

Each misses in different ways, so using both covers each other's weak spots.

How do you merge two result lists?

The problem is that the two systems score on different scales. BM25 scores have no fixed range, while cosine similarity sits in a fixed range. An article from Redis explains that simply adding raw scores often lets BM25 win by default.

A popular approach is Reciprocal Rank Fusion (RRF), proposed back in 2009. Instead of scores, RRF looks only at each document's rank in each list. Documents ranked well in both lists get a high combined score, even if they are not first in either. Its advantages: no training data required, and it works with any kind of search system.

A simple example from the Redis article: document A ranks 2nd in keyword search and 5th in semantic search, while document B ranks 1st in keyword search but does not appear in the semantic results at all. Merged with RRF, document A wins, because both systems agree it is relevant.

What is reranking?

After hybrid search returns several dozen candidates, the next step is reranking: re-ordering them with a more careful model.

Why rank twice?

The first pass must be fast because it sweeps the whole library, so it compares question and documents roughly. A reranker reads the question and each document together in detail. It is more accurate but much slower, so it is used only on a few dozen candidates. It is like screening resumes quickly in the first round, then having the manager read only the shortlisted ones carefully.

Reported results

In Anthropic's tests of its Contextual Retrieval approach (September 2024), as reported by Maginative, contextual embeddings combined with BM25 reduced failed retrievals by 49%, and adding reranking on top reduced them by 67%. These numbers come from Anthropic's test datasets and your organisation's results may differ, but the direction is clear: multi-stage search really helps.

An example from a Thai organisation

A law firm builds a system to search its internal contracts and legal opinions. Lawyers often search by section number or counterparty name, which keyword search handles well. But they also ask things like "Which contracts have a non-compete clause?", which needs semantic search because each contract words it differently. The team uses hybrid search merged with RRF, then has a reranker narrow it down to a few documents before the AI summarises, with links to the original contracts for lawyers to check.

Watch out for

• Every added layer increases both response time and cost. Weigh it against the accuracy you gain.
• RRF does not always win. If you have a question set with answers, a tuned weighted combination may do better.
• Thai keyword search needs correct word segmentation first, or words won't be found.
• Some rerankers handle Thai poorly. Test with real Thai questions before choosing one.

AI can't answer correctly if we hand it the wrong documents to read.

How to get started

1. Collect the questions where search failed, and sort them into failures on specific terms versus failures on meaning.
2. If codes or names are often missed, add keyword search alongside vector search.
3. Merge results with RRF as a default, since it needs no tuning.
4. Add a reranker on the top few dozen candidates, and measure before and after.
5. Track response time alongside accuracy so the system doesn't get so slow that people stop using it.

To understand the basics of vector search first, read What is a vector database. To evaluate your search systematically, continue with Evaluating RAG and reducing hallucination.