Reranking is the process of reordering an initial set of retrieved passages by true relevance to a query before they are used. In a document AI system for commercial real estate, reranking takes the candidate clauses that a first-pass search returns for a question like a co-tenancy trigger and reorders them so the most relevant lease passage sits at the top for the model to read.
How Reranking Works
Reranking works as the second stage of a two-stage retrieval pipeline. A fast first stage, a vector search or keyword search, casts a wide net and returns many candidates with high recall. A slower, more precise reranker then scores each candidate against the query and reorders them, so the final short list passed to a language model is ranked by true relevance rather than approximate similarity.
The reranker is usually a cross-encoder. A first-stage bi-encoder embeds the query and each document separately, so their tokens never interact, per a ZeroEntropy comparison. A cross-encoder concatenates the query and one candidate into a single input and reads them together, which lets it weigh negation, intent, and context that separate embeddings miss. That precision costs compute, so it runs only on the top candidates.
The gains are measured on retrieval benchmarks. Adding a cross-encoder reranker over embedding retrieval improves NDCG@10 by roughly 5 to 20% across verticals, with the largest gains in legal, healthcare, and finance where keyword overlap is a poor proxy for relevance, per a Towards Data Science review. Candidate depth matters: reranking quality rises sharply from 50 to 100 first-stage candidates.
Why Reranking Matters
Reranking matters because a retrieval-augmented system can only answer from the passages it is handed, and first-stage search often ranks a loosely related clause above the exact one. If the true co-tenancy trigger sits at rank 8 but only the top 3 passages reach the model, the answer is wrong or unsupported even though the right clause was retrieved. Reranking moves it to the top.
The lift is concentrated where CRE queries live. Lease language is nuanced and negation-heavy, so vector similarity alone confuses "Tenant may terminate" with "Tenant may not terminate." One benchmark reported MRR@3 rising from 0.433 to 0.605, a 39.7% relative gain, after adding a cross-encoder reranker. In a document context, that reordering is what puts the governing clause in front of the model instead of a similar-sounding distractor.
Example
Reranking is clearest when the true answer sits below the top of first-stage results and the reranker lifts it. An operator asks which clause governs a co-tenancy termination right.
Passage | First-stage rank | Reranker score | Reranked position |
Co-tenancy termination clause | 8 | 0.94 | 1 |
General default clause | 1 | 0.42 | 4 |
Renewal option clause | 2 | 0.31 | 5 |
Co-tenancy definition | 5 | 0.88 | 2 |
Operating covenant clause | 3 | 0.61 | 3 |
Suppose only the top 3 passages are passed to the model. Before reranking, the governing clause at rank 8 never reaches it, so the answer is unsupported. After reranking, the clause moves to position 1 on a 0.94 relevance score and is passed. The reciprocal rank for the true clause improves from 1/8 = 0.125 to 1/1 = 1.00, an 8x gain in how findable the correct passage is.
Variations and Edge Cases
Reranking behaves differently by architecture and setting. The cases below distinguish common approaches.
Case | Behavior |
Cross-encoder | Reads query and document together; highest precision, highest cost |
Bi-encoder rerank | Reuses embeddings; faster, less precise than cross-encoder |
Listwise reranker | Scores the candidate set jointly rather than pair by pair |
Candidate depth | Too few candidates caps recall; 50 to 100 is a common range |
Latency budget | Reranking adds compute, so it runs on a top-k slice, not all |
Domain mismatch | A general reranker underweights lease-specific phrasing |
Reranking vs Semantic Search
Reranking is often confused with semantic search, but they are two stages of one pipeline. Semantic search is the first stage: it embeds the query and documents and retrieves candidates by vector similarity, fast and high-recall but approximate. Reranking is the second stage: it rescores those candidates with a more precise model and reorders them by true relevance.
Semantic search optimizes for recall, getting the right passage into the candidate set at all. Reranking optimizes for precision at the top, getting the right passage to position 1. They compose: semantic search returns 100 candidates cheaply, and the reranker spends compute only on those 100 to produce the top few the model actually reads.
Frequently Asked Questions
What is reranking in a document AI system?Reranking is the process of reordering an initial set of retrieved passages by true relevance to a query before they are used. It is the second stage of a two-stage pipeline that rescores first-stage candidates with a precise model so the most relevant lease or document passage reaches the top.
How much does reranking improve retrieval accuracy?Adding a cross-encoder reranker over embedding retrieval improves NDCG@10 by roughly 5 to 20% across verticals, with the largest gains in legal, healthcare, and finance, per a Towards Data Science review. One benchmark reported MRR@3 rising from 0.433 to 0.605, a 39.7% relative gain.
What is the difference between reranking and semantic search?Semantic search is the fast first stage that retrieves candidates by vector similarity, optimized for recall. Reranking is the precise second stage that rescores those candidates and reorders them by true relevance, optimized for precision at the top. They run together in a two-stage retrieval pipeline.
Related Terms
Retrieval Augmented Generation
Semantic Search
Vector Embedding
Source Citation