DecisionRAG is a retrieval method from Cornell University and ASAPP (arXiv 2610.35868) that drops the first-stage retriever. A decision model walks the corpus tree level by level, from clusters or sections to pages to passages, and scores every candidate at a level in one forward pass without generating tokens.

What is a decision model?

A model that takes a context and a set of questions and returns a probability distribution over each question's answer options in a single forward pass, with no autoregressive sampling. DecisionRAG uses TypeSafe's Jev 1.13.

How fast and how expensive is DecisionRAG?

Latency grows with tree depth, not corpus size: roughly 0.2 s per level. On Natural Questions over 2.68M Wikipedia paragraphs it takes a median 1.73 s and 2.33 cents per query.

How does DecisionRAG compare with dense retrieval and a reranker?

With no task-specific training it beats dense retrieval plus an 8B reranker by 13.8 R@1 on MultiDoc2Dial and 17.4 Hit@1 on Natural Questions, and it is 34 to 145 times faster than GPT-5 making the same navigation decisions by generating text.

Is DecisionRAG really index-free?

It needs no passage index and encodes no passages at query time. Structured corpora reuse their existing hierarchy. Unstructured ones like Wikipedia need an offline clustering pass with small page-level embeddings and one-sentence group descriptions.

When should you not use DecisionRAG on its own?

When a missed document is costly, as in compliance, legal or safety lookups, because pruning cannot be undone. Keep dense or lexical candidates feeding the final stage. It also depends on one decision-model API and on the quality of your corpus structure.

DecisionRAG Explained: Retriever-Free Retrieval by Walking a Corpus with a Decision Model

[!NOTE] TL;DR

DecisionRAG (arXiv 2610.35868) drops the first-stage retriever. A decision model walks the corpus tree (clusters or help centers, then pages, then passages) and returns $P(\text{yes})$ for every candidate at a level in one forward pass. It generates no tokens.

Candidates are judged together in one shared context, so their scores can be compared. That alone is worth up to 7.7 points of R@1 in the paper's ablations.

Latency grows with tree depth, not corpus size: $T(N) \approx 0.2,\text{s} \cdot \lceil \log_b N \rceil$. Natural Questions over 2.68M Wikipedia paragraphs takes a median 1.73 s and 2.33¢ per query.

With no task-specific training it beats dense retrieval plus an 8B reranker by 13.8 R@1 on MultiDoc2Dial and 17.4 Hit@1 on Natural Questions. It is 34–145× faster than GPT-5 making the same navigation decisions by generating text.

The caveats: it depends on one decision-model API, needs a corpus with usable structure or a decent clustering, and pruning cannot be undone. I would ship it as a hybrid with dense candidates, not as a full replacement.

[!IMPORTANT] Credit where it's due. This is a paper read and explainer, not my research. The work is by Chao Wan* and Paul Jünger* (equal contribution), with Beixi Du, Bodong Liu, Navaneetha Krishnan and Kilian Q. Weinberger, from Cornell University's Department of Computer Science and ASAPP. Every result, number and method detail below is theirs, from arXiv 2610.35868. The worked example on Python's docs and the illustrative code are mine. Any mistakes in explaining it are mine too.

Most RAG failures I debug are not generation failures. The model was just given the wrong five passages.

That gap keeps growing. Generators get better every few months. The thing that decides what they see is still BM25, cosine similarity in a fixed embedding space, or a reranker that can only reorder what that first stage returned. The paper "Retriever-Free Retrieval: Navigating Documents with Decision Models" (Wan, Jünger, Du, Liu, Krishnan, Weinberger; Cornell and ASAPP; arXiv 2610.35868, 26 Oct 2026) attacks that directly. This is my read of it as someone who builds ingestion pipelines and agents for a living.

The problem: a strong LLM behind a weak retriever

The paper calls it a capability asymmetry. Whatever the retriever misses, no amount of generator capability can recover.

The usual fixes each cost something:

Rerankers are cheap and help, but they only reorder the first stage's candidate pool. If the gold passage is not in the top 100, the reranker never sees it.
Agentic search, where an LLM reads and decides what to open next, brings real model intelligence to retrieval. It also generates tokens at every step, which makes it slow and expensive. I covered one flavour of this in PageIndex vs Vector RAG, where an LLM reads a table-of-contents tree and picks sections.

DecisionRAG sits between them. It keeps the "navigate a tree with a capable model" idea from PageIndex-style systems. It replaces generation with a different kind of model call that answers many yes/no questions at once.

The paper's framing (its Figure 1): a simple retriever throttles what a strong LLM can see. A decision model brings comparable capability to the choice of what to retrieve.

What a decision model is

A decision model takes a context and a set of questions. For each question it returns a probability distribution over that question's answer options, all in a single forward pass. There is no autoregressive sampling.

The paper uses TypeSafe's Jev 1.13 and mentions OpenAI's decision model as another example of the category. It relies on two of Jev's modes.

Parallel binary judgments. Given a shared context $s$ and questions $Q_1, \dots, Q_n$:

$$ p_i^{\text{bin}} = P(\text{Yes} \mid s, Q_i) $$

Each probability is normalised only over "Yes" and "No", not across questions. The scores need not sum to one. In the paper's running example the three page scores are 0.96, 0.81 and 0.07, which sum to 1.84. Because of that, there is no option limit beyond the context length, and large levels can be split across parallel requests.

Multiple-choice judgments. Given a context, one question $Q$ and options $\mathcal{A} = {a_1, \dots, a_m}$:

$$ p_i^{\text{mul}} = P(a_i \mid s, Q, \mathcal{A}), \qquad \sum_{i=1}^{m} p_i^{\text{mul}} = 1 $$

Here the options compete directly. Jev caps the option count at 255, so the paper applies this mode only to a shortlist at the end.

How the walk works

The corpus is treated as a tree. Leaves are passages. Internal nodes are anything that groups them: help centers, product areas, topic clusters, documents, section headings. Every node has a short description: the passage text, a page title, or a one-sentence cluster summary.

Algorithm 1 in the paper reduces to three steps.

Screen. Put the query and every candidate at the current level into one context. Ask one yes/no question per candidate ("Is the information needed to answer the user's question on Page $n$?"). Keep the top $k$, then expand only their children.
Score passages jointly. At the last level, put the passages of all surviving pages into one shared context, score them the same way, and keep the top $r$.
Optionally rerank. Run one multiple-choice judgment over the top $m \le r$ passages and reorder them by $p^{\text{mul}}$. The paper uses $m = 20$, or $m = 5$ on Natural Questions. On MTRAG it only promotes the single top choice to rank one.

Pruning is irreversible. A branch dropped at one level is never revisited for that query. So the budgets lean towards recall, and a budget can also be a threshold. On MultiDoc2Dial the paper keeps the runner-up help center whenever the top score is below 0.8.

One forward pass per level. Pruned clusters (x) are never expanded, so their pages and passages are never scored. Shapes and budgets are illustrative.

On Natural Questions the tree is cluster → page → paragraph. The model scores 1,000 cluster descriptions and keeps 150. It then scores roughly 17,500 page titles from those clusters, keeps 30 pages, and jointly scores about 230 of their paragraphs. Each paragraph's probability is multiplied by its page's. The paper notes this means reading only 16% of the page titles.

A worked example of my own

The paper's examples are Wikipedia questions and customer-support dialogues. To check that I actually understood the method, I walked it by hand on a corpus I know well: the official Python documentation. The scores below are my illustration of how the walk behaves, not outputs from Jev or numbers from the paper.

The question is one I've seen in many forms:

"My background asyncio task sometimes stops halfway and never finishes. There's no error. Why?"

The answer is a short note in the docs for asyncio.create_task: save a reference to the returned task, because the event loop only keeps weak references to tasks, so an unreferenced task can be garbage collected before it's done.

That note is hard to find by keywords. The query says "never", "finishes" and "no error". The passage that shares those words is a different one, on the "Developing with asyncio" page, about the RuntimeWarning: coroutine ... was never awaited warning. BM25 loves it. A dense retriever probably does too, because both passages are about asyncio work that silently doesn't happen. But it's the wrong answer: a never-awaited coroutine never starts, and this task starts and then disappears.

Here is the walk, level by level.

Level 1: chapters of the library reference. One context holds the question and the chapter descriptions, with one yes/no question per chapter. "Networking and Interprocess Communication" (where asyncio lives) scores high. "Concurrent Execution" (threading, concurrent.futures) scores reasonably, because "background task" sounds like threads. "Debugging and Profiling" scores lower. With $k = 2$, the first two survive and everything else is never opened.

Level 2: page titles inside the kept chapters. "Coroutines and Tasks" and "Developing with asyncio" score well. "Event Loop" is lukewarm. The threading pages fall away now that they sit next to asyncio pages that fit better. Keep two pages.

Level 3: passages of both pages, in one shared context. This is where joint scoring earns its keep. Scored alone, the never-awaited warning looks relevant: it is about asyncio work that silently does nothing. Scored next to the weak-reference note, it loses, because the model can see a passage that explains stopping midway exactly. The cancellation and debug-mode passages land in between.

My worked example on the Python docs. Scores are illustrative, not model outputs. The keyword-friendly "never awaited" passage looks relevant alone and drops once the real answer sits beside it.

Pick a question below to watch a walk run level by level. Two more Python questions are in there too, each with its own easy-to-confuse passage.

try it · pick a question

level 1 · library chapters · keep 2

Networking and IPC (asyncio)0.93

Concurrent Execution0.71

Debugging and Profiling0.38

Text Processing0.04

level 2 · pages · keep 2

Coroutines and Tasks0.88

Developing with asyncio0.74

Event Loop0.41

threading0.12

level 3 · passages, one shared context · keep 1

create_task: keep a reference0.94

Task cancellation0.36

asyncio debug mode0.29

"coroutine was never awaited"0.18

→ asyncio.create_task: save a reference to the task, the event loop only keeps weak references.

Scripted walk over the Python docs. Scores are illustrative, not model outputs.

Three things this example made concrete for me:

The tree did the recall work. No index was asked "what is similar to this query?". The model only answered "is the answer under here?" a few dozen times, and the correct page was never at risk after level 1.
The hard negative was beaten at the passage level, not before. Both the right passage and the decoy lived on kept pages. Shared-context scoring separated them, which is exactly the paper's claim about comparable scores.
Where it could still fail. If the chapter description for "Networking and Interprocess Communication" never mentioned asyncio, level 1 could have pruned the whole branch, and nothing later can recover it. Node descriptions are the new retrieval surface, so they deserve the care we used to give embeddings.

Why scoring in one shared context matters

This is the paper's most transferable idea. Pointwise scoring asks "is this relevant?" with nothing to compare against. A page that mentions the right entity looks relevant on its own. In a shared context the model sees the alternatives too: "a candidate that would look relevant on its own receives a lower $P(\text{Yes})$ when a clearly better one is present."

Query: "Who is the main character in Christopher Nolan's film from the year 2000?" Right-hand scores are from the paper's Figure 3a. Left-hand scores are my illustration of what isolated scoring can look like, not a paper result.

The ablations back this up:

On MultiDoc2Dial design turns, scoring all kept pages' passages in one context instead of page by page raises R@1 from 36.7% to 44.4% (+7.7, 95% CI [3.3, 12.2]).
On Natural Questions pools of 5 to 1,000 candidates, shared-context binary judgments consistently beat one request per candidate. The paper reports a 6.5-point Hit@1 gain for joint over separate scoring.
With a fixed pool of 200 candidates, Hit@1 goes from 47.8% (one candidate per request) to 54.3% (50 per request), with little further gain from larger requests. Multiple-choice over all 200 reaches 60.6%.

That 50-candidate plateau is what makes the system practical. Big levels can be split into parallel requests of about 50 without losing calibration. The paper also notes that splitting lowers Hit@1 but leaves R@10 unchanged, so pooled scores from separate requests are fine for screening. Only the final ranking benefits from one shared request.

[!IMPORTANT] The shared-context effect is the part I would reuse even without the tree. A listwise "here are all the candidates, judge each" call is a different and better-calibrated problem than N independent pointwise calls.

Latency and cost model

Every candidate at a level is judged in parallel, so latency depends on the number of levels, not passages. For a roughly balanced tree with at most $b$ children per node:

$$ T(N) \approx t_{\text{req}} \cdot \lceil \log_b N \rceil, \qquad t_{\text{req}} \approx 0.2,\text{s} $$

With $b = 100$, even $10^9$ passages need five sequential decisions, about one second. This assumes there is enough serving capacity to run each level's requests at once.

Cost scales differently. It grows with the number of judgments: $b$ children for each of the $k$ nodes kept, so no level scores more than $k \cdot b$ candidates. Depth bounds the wall-clock time. The budgets bound the bill.

A million times more passages adds three sequential decisions. Real runs carry overhead: Natural Questions over 2.68M paragraphs measured a median 1.73 s.

The measured numbers are close to that model. Median latency per query is 0.68 s on MultiDoc2Dial, 0.82 s on ConditionalQA, 1.76 s on MTRAG and 1.73 s on Natural Questions. Natural Questions, the largest corpus, runs as fast as MTRAG. The optional multiple-choice step adds 0.12–0.15 s and raises API cost by 4–33%.

The results that matter

The headline comparison is training-free and has no first-stage retriever. Dense means Qwen3-Embedding-8B. All numbers are from the paper's Table 2, in percent.

Method	MD2D R@1	MD2D R@10	CQA Hit@1	MTRAG R@5	NQ Hit@1	NQ R@10
BM25	20.6	56.4	24.7	18.8	16.4	42.3
Dense	27.9	70.9	47.2	26.0	46.2	87.2
Dense top-100 + Qwen3-Reranker-8B	33.6	81.1	64.6	32.5	48.0	89.0
Dense top-20 + GPT-5 rerank	47.5	78.6	73.4	31.8	60.8	88.9
DecisionRAG (Jev)	47.4	83.6	69.7	34.2	65.4	86.7
DecisionRAG + dense candidates	48.5	84.7	73.4	36.6	66.4	88.9

What I take from this table:

The headline gains are real. Against the 8B reranker pipeline, DecisionRAG adds 13.8 R@1 on MultiDoc2Dial and 17.4 Hit@1 on Natural Questions.
GPT-5 reranking is the honest competitor. It ties DecisionRAG on MultiDoc2Dial R@1 and wins ConditionalQA Hit@1 (73.4 vs 69.7) until dense candidates are added.
Pure navigation loses some recall at depth. On Natural Questions its R@10 of 86.7 is below dense retrieval's 87.2. Adding the dense top 20 to the last stage lifts it to 88.9: the two approaches recover complementary evidence.

Downstream answers move too. With Qwen3.5-27B as the generator on each system's top five passages, MTRAG judged correctness goes from 39.4% (dense + Qwen3-Reranker) to 44.1% with DecisionRAG and 47.3% with dense candidates. Exact match on Natural Questions reaches 54.8% with dense candidates, against a gold-passage ceiling of 65.3%.

The paper also separates navigation from the model. It gives Jev the dense top 100 and has it rerank them exactly as DecisionRAG's last stage would. Navigating still wins Hit@1 by 7.7 points on ConditionalQA and 7.0 on Natural Questions, and the two are level on MultiDoc2Dial and MTRAG. That holds even though the dense top 100 contains the gold evidence more often (98.4% vs 94.0% on Natural Questions). The authors' guess is that navigation shows whole pages in reading order rather than isolated chunks. That is the same intuition behind Late Chunking: a passage is easier to judge with its document around it.

Decisions versus generation

This is the comparison with PageIndex-style systems. The paper keeps the same hierarchy and budgets but swaps Jev for GPT-5 (low reasoning effort), which reads each level and writes out the numbers to keep.

Natural Questions (first 100)	Hit@1	R@10	Median s	¢/query
GPT-5, generative	59.0	82.5	250.8	57.2
DecisionRAG	65.0	86.5	1.73	2.33

On ConditionalQA, GPT-5 leads by 4.1 Hit@1 (73.8 vs 69.7) at 28.2 s and 6.1¢ against 0.82 s and 0.29¢. Across both datasets, generating the decisions is 34–145× slower and 21–25× more expensive, with no consistent accuracy gain.

Median latency per query from the paper's Table 6. The decision model reads every candidate at a level in one pass instead of reasoning about them and naming its picks token by token.

Training helps as well. A Qwen3-8B judge with a scoring head, trained on MultiDoc2Dial data, reaches 60.7 R@1. The best published task-trained retriever the paper compares against reaches 55.0, so that is 5.7 points better. The approach is not tied to Jev.

How each benchmark was configured

Configurations were chosen on design sets kept separate from the evaluation sets.

MultiDoc2Dial: help center → page → passage over 4 centers, 413 pages and 4,110 passages. It keeps the top ten pages and scores their passages together with those from up to two "notepad" pages, the pages retrieved most recently in the conversation.
ConditionalQA: page → paragraph over 652 pages and 58,483 paragraphs. All 652 guides go in one request. It keeps three pages and scores their paragraphs together.
MTRAG (IBM Cloud): page → heading → passage over 8,578 pages and 72,442 passages. The first pass scores all titles in parallel requests and pools the top 50. A second pass rescores those 50 in one context and keeps ten, because the first-pass scores came from different contexts. Pages with more than 60 passages get an extra heading stage.
Natural Questions (DAPR): cluster → page → paragraph over 1,000 clusters, 108,626 pages and 2.68M paragraphs. Evaluated on 500 test questions.

On MTRAG, sending the first page pass as requests of at most 10k tokens instead of 150k (about 40 parallel requests instead of three) left R@5 statistically unchanged and cut median latency from 3.09 s to 1.76 s.

An illustrative implementation

Below is a small version of the walk. It is not the paper's code and decide/choose are not a real vendor API. They are stubs that stand in for whatever decision model you have. The structure follows Algorithm 1: screen each inner level and keep the top $k$, score the final level jointly, then optionally rerank with a multiple-choice call whose probabilities sum to 1.

import asyncio
import math
from dataclasses import dataclass, field
 
MAX_PER_REQUEST = 50   # the paper sees no loss from splitting at ~50 candidates per request
MAX_CHOICES = 255      # Jev's option limit for multiple-choice judgments
CONCURRENCY = 8        # the paper's timing runs use up to eight concurrent requests
 
 
@dataclass
class Node:
    id: str
    text: str                      # cluster summary, page title, or passage text
    children: list["Node"] = field(default_factory=list)
 
 
# Illustrative stand-ins, NOT the paper's code or any vendor's real API.
async def decide(context: str, questions: list[str]) -> list[float]:
    """One forward pass: P(yes) for every question against the same context."""
    raise NotImplementedError
 
 
async def choose(context: str, question: str, options: list[str]) -> list[float]:
    """One forward pass: a distribution over options that sums to 1."""
    raise NotImplementedError
 
 
def render_context(query: str, batch: list[Node]) -> str:
    listing = "\n".join(f"[{i}] {n.text}" for i, n in enumerate(batch, 1))
    return f"Query: {query}\n\nCandidates:\n{listing}"
 
 
def chunks(items: list, size: int) -> list[list]:
    return [items[i:i + size] for i in range(0, len(items), size)]
 
 
async def score_jointly(query: str, nodes: list[Node], gate: asyncio.Semaphore) -> list[float]:
    async def score_batch(batch: list[Node]) -> list[float]:
        questions = [
            f"Is the information needed to answer the query in candidate [{i}]?"
            for i in range(1, len(batch) + 1)
        ]
        async with gate:
            return await decide(render_context(query, batch), questions)
 
    per_batch = await asyncio.gather(*(score_batch(b) for b in chunks(nodes, MAX_PER_REQUEST)))
    return [p for scores in per_batch for p in scores]
 
 
def top(nodes: list[Node], scores: list[float], k: int) -> list[tuple[Node, float]]:
    return sorted(zip(nodes, scores), key=lambda pair: pair[1], reverse=True)[:k]
 
 
async def rerank_top(query: str, ranked: list[Node], m: int) -> list[Node]:
    if m > MAX_CHOICES:
        raise ValueError(f"multiple-choice is limited to {MAX_CHOICES} options")
    head, tail = ranked[:m], ranked[m:]
    probs = await choose(
        render_context(query, head),
        "Which candidate is most relevant to the query?",
        [f"[{i}]" for i in range(1, len(head) + 1)],
    )
    if not math.isclose(sum(probs), 1.0, abs_tol=1e-6):
        raise ValueError("choice probabilities must sum to 1")
    reordered = [n for n, _ in sorted(zip(head, probs), key=lambda pair: pair[1], reverse=True)]
    return reordered + tail
 
 
async def retrieve(query: str, root: Node, budgets: list[int], r: int, m: int = 0) -> list[Node]:
    """budgets[l] = how many nodes to keep at inner level l; r passages come back."""
    gate = asyncio.Semaphore(CONCURRENCY)
    frontier = root.children
    for k in budgets:
        kept = top(frontier, await score_jointly(query, frontier, gate), k)
        frontier = [child for node, _ in kept for child in node.children]
    ranked = [n for n, _ in top(frontier, await score_jointly(query, frontier, gate), r)]
    m = min(m, len(ranked))
    return await rerank_top(query, ranked, m) if m else ranked

A few deliberate choices:

Large levels are chunked into requests of at most 50 and run concurrently under a semaphore. Scores from different chunks are pooled. The paper shows that is fine for screening.
The passage level uses the same chunking. If context allows, send the final level as one request, as the paper does with about 230 paragraphs on Natural Questions.
Only the top $m$ passages get reordered. The rest keep their binary order, as in the paper.

The estimator implements the latency formula with integer depth, because floating-point log can land just above an integer (math.log(125, 5) is 3.0000000000000004) and add a phantom level. Cost is counted in judgments and requests. Pricing is provider-specific, so you supply it.

def depth(n_leaves: int, branching: int) -> int:
    """ceil(log_b N) with integers, so 10**4 at b=100 is exactly 2."""
    levels, reach = 0, 1
    while reach < n_leaves:
        reach *= branching
        levels += 1
    return levels
 
 
def estimate(n_leaves: int, branching: int, budgets: list[int], *, t_req: float = 0.2,
             rerank: bool = False, price_per_request: float | None = None) -> dict:
    levels = depth(n_leaves, branching)
    if len(budgets) != levels - 1:
        raise ValueError(f"need {levels - 1} budgets for {levels} levels")
    judged = [branching] + [k * branching for k in budgets]
    requests = sum(math.ceil(j / MAX_PER_REQUEST) for j in judged) + int(rerank)
    stages = levels + int(rerank)
    return {
        "stages": stages,
        "latency_s": round(stages * t_req, 2),
        "judgments": sum(judged),
        "requests": requests,
        "cost": None if price_per_request is None else requests * price_per_request,
    }
 
 
assert estimate(10**9, 100, budgets=[10, 10, 10, 10])["latency_s"] == 1.0  # paper: 5 decisions, ~1 s

I ran this against a fake keyword-overlap decide on 120 clusters with one planted answer. The walk found it and split the clusters into requests of 50, 50 and 20. That proves the plumbing, nothing about retrieval quality.

[!TIP] Treat the rerank call's "sums to 1" check as a contract test on your provider. If a multiple-choice endpoint ever hands back unnormalised scores, you want it to fail loudly, not quietly reorder your top 20 on garbage.

Is it really index-free?

Mostly, and the paper is careful about the wording. At query time there is no first-stage retriever and no precomputed passage index. DecisionRAG encodes no passages. For the first three benchmarks the hierarchy comes from existing structure and metadata: help centers, HTML headings, URL paths.

Natural Questions is the exception. Wikipedia has no structure above the page, so the authors run an offline pass. They embed each page's title and first sentence with e5-base-v2, k-means them into 1,000 groups of 15 to 350 pages (median 106), and have Qwen3-8B write a one-sentence description of each group. That is an embedding step, just a far smaller one than a passage index. I would call it "no passage index" rather than "no preprocessing".

What I would steal for production today

Shared-context scoring as the last stage. Even if I keep my existing retriever, the evidence that listwise judgment in one context beats pointwise scoring is strong and cheap to act on. The +dense candidates result points the same way: merge the navigated and retrieved candidates and judge them together.

Use the structure ingestion already produces. Document pipelines already recover headings, sections and page titles, which is what PageIndex-style trees are built from. That tree is the hierarchy DecisionRAG wants. No new index required.

Budget for recall at the top, precision at the bottom. The Natural Questions cluster sweep makes the trade-off concrete. Keeping 15% of clusters matches flat scoring of every page title (86.2% vs 84.5% R@10) at about a fifth of the cost (2.3¢ vs 12.1¢) and a third of the latency (1.7 vs 4.5 s). Below 5% the cluster holding the gold page is often dropped, and at 1% R@10 falls to 63.7%.

Keep the multiple-choice step. On design data it took MultiDoc2Dial R@1 from 50.1 to 51.3 and ConditionalQA Hit@1 from 71.9 to 76.5, for one extra request.

Where I would be careful

[!WARNING] Every training-free result in the paper runs on one hosted model, Jev 1.13. Its 255-option limit, roughly 0.2 s per request and pricing all shape the design. If you build on it, you depend on one vendor's API for your entire retrieval path. The open-weight 8B judge shows a way out, but it was trained on in-domain data.

Structure quality is now retrieval quality. The paper's own limitations section says it: poorly organised documents or uninformative node descriptions hurt retrieval. A corpus of scanned PDFs titled scan_0042.pdf gives the model nothing to screen on. You would have to generate descriptions first, and their quality becomes your ceiling.

Cluster quality on unstructured corpora. With built hierarchies, a weak cluster summary hides its whole subtree for good. The paper's mitigation is generous top-level budgets (150 of 1,000 clusters) plus dense candidates. I would monitor where gold evidence gets pruned, not just the final recall.

Cost at scale is not obviously cheaper. On Natural Questions, DecisionRAG costs 2.33¢ per query against 1.40¢ for GPT-5 reranking the dense top 20. That GPT-5 figure counts API cost only, and the dense retrieval it reranks is not part of the figure. Cost grows with judgments. At high query volume, a 2¢ retrieval step is a line item you need to plan for.

Latency comparisons are not controlled. Baselines ran on local GPUs, DecisionRAG on a live API, with different samples. The paper says so itself.

[!CAUTION] Pruning is irreversible by design. If your domain punishes a missed document, as compliance, legal or safety lookups do, don't run navigation alone. Keep a dense or lexical lane feeding the last stage.

Should you try this?

Your corpus has real structure (sections, titles, product areas), or you can cluster it and write a useful description for every group.
Your current failure mode is recall: the right passage never reaches the reranker.
A retrieval budget of roughly 1–2 s and a couple of cents per query is acceptable.
You can live with depending on a decision-model API, or you can train a judge on in-domain data.
You will keep dense candidates in the final stage and measure where gold evidence gets pruned.

If most boxes are ticked, I'd prototype it on one corpus. If not, I'd still move shared-context joint scoring into my existing reranking stage. It is the cheapest idea in the paper to adopt.

DecisionRAG is the faster, non-generative cousin of PageIndex: a capable model still chooses where to look, but one forward pass per level replaces paragraphs of reasoning. If decision models keep improving, retrieval improves with them. That is the paper's bet.

Credits, thanks and how to cite

Thank you to Chao Wan, Paul Jünger, Beixi Du, Bodong Liu, Navaneetha Krishnan and Kilian Q. Weinberger for a paper that is unusually easy to learn from: a clear algorithm, honest ablations, latency and cost reported next to accuracy, and limitations stated plainly. The decision model they build on is Jev 1.13 from TypeSafe. Per the paper's acknowledgements, the work was supported by the National Science Foundation, NewYork-Presbyterian through the NYP-Cornell Cardiovascular AI Collaboration, and arXiv.

Read the original: Retriever-Free Retrieval: Navigating Documents with Decision Models (arXiv 2610.35868). Correspondence goes to the first authors, Chao Wan and Paul Jünger at Cornell.

BibTeX

@article{wan2026retrieverfree,
  title   = {Retriever-Free Retrieval: Navigating Documents with Decision Models},
  author  = {Wan, Chao and J{\"u}nger, Paul and Du, Beixi and Liu, Bodong and Krishnan, Navaneetha and Weinberger, Kilian Q.},
  journal = {arXiv preprint arXiv:2610.35868},
  year    = {2026}
}