MQR / multi-query retrieval feature
Turn one user question into several retrieval queries, then combine their results. It can recover vocabulary or viewpoint mismatches, but it adds model, embedding, and search work and can lower the rank of the first relevant result.
Query lane unit
One query variant entering retrieval. In this design, lane one is always the original question and lanes two and three are optional generated alternatives.
Dense embedding representation
A fixed-length vector produced by an embedding model. Similar meanings tend to be near one another even when they use different words. Ragz’s benchmark used OpenAI text-embedding-3-large at 1,024 dimensions.
Sparse embedding / BM25-style search representation
A mostly-empty vector whose non-zero entries represent words or terms. It is good at exact names, numbers, acronyms, and rare vocabulary. Ragz computes it with FastEmbed and combines it with dense search.
Hybrid retrieval search
Use dense semantic search and sparse lexical search together. The goal is to cover both “same meaning, different wording” and “exact token matters” cases.
RRF / reciprocal rank fusion fusion
A rank-combination method. A result receives roughly 1 / (constant + rank) from each list, and the contributions are added. It combines rankings without pretending dense and sparse raw scores mean the same thing.
Reranking second pass
Retrieve a broad candidate pool first, then ask a cross-encoder to score each candidate against the original question. It may improve order, but the provider call can dominate latency and cost.
top_k and candidate pool cutoff
top_k is how many chunks the answer path finally keeps. P in the benchmark is the pre-rerank pool size, such as P=50. P is not nucleus sampling and does not mean top_p.
Recall@k retrieval quality
Of the questions with a known relevant item, how often did at least one relevant item appear in the first k results? A Recall@5 of 0.60 means 60% of the benchmark questions had a relevant result in the top five.
MRR@k / mean reciprocal rank retrieval quality
Look only at the first relevant result. Score it as 1 / rank: rank 1 = 1.0, rank 2 = 0.5, rank 3 = 0.333, and no relevant result in the cutoff = 0. Higher MRR means the first useful result appears earlier.
nDCG@k retrieval quality
Measures ranked quality when relevance can have grades, not just relevant/not relevant. Higher-ranked items count more, then the score is normalized against an ideal ordering. 1.0 is the ideal for the judged cutoff.
Qrels / relevance judgments evaluation data
A table saying which retrieved unit is relevant for each query. In this report the retrieval qrel unit was the exact physical PDF page. Changing qrels, chunking, or corpus changes what the metric means.
Context relevance RAG-Triad
Does the retrieved context contain information that is useful and necessary for answering? High context relevance does not guarantee the model used it correctly or that the answer is complete.
Groundedness / “groundness” RAG-Triad
Are the answer’s claims supported by the supplied retrieved context? High groundedness means the answer stays close to evidence; it does not prove the evidence itself is the correct reference page or that the answer covers every part of the question.
Answer relevance RAG-Triad
Does the answer address what the user asked, rather than wandering or answering a neighboring question? A relevant answer can still be factually wrong or weakly supported.
Correctness answer quality
How closely the answer matches the reference facts or expected answer. Correctness is distinct from groundedness: a model can faithfully quote the wrong retrieved passage, or answer correctly with insufficient citation evidence.
Abstention F1 refusal quality
A balance of precision and recall for refusing questions the corpus cannot answer while answering questions it can. It combines false refusals and missed refusals into one harmonic-mean score; it is not the same as a product “no answer” flag.
Citation precision and citation recall provenance
Citation precision asks how many citations point to the correct evidence unit; citation recall asks how much of the required evidence was cited. A structurally valid citation can still miss the exact qrel page.
p50 / p95 / p99 latency percentiles
p50 is the median: half of requests are faster. p95 means 95% are faster and 5% are slower. p99 exposes the long tail. Percentiles are not averages and should be reported with sample size and boundary conditions.
TTFT / time to first token streaming
Time from request start until the first streamed answer token arrives. It measures perceived responsiveness, not total completion time. A benchmark cache lookup or retrieval microbenchmark is not authenticated chat TTFT.
Cold, warm, hit, miss cache vocabulary
A cold or cache-off request computes the expensive value. A hit reuses it. “Warm” describes a populated cache, not necessarily a user-facing answer replay. The query-vector cache and complete-response cache are different layers.
Cache invalidation and freshness correctness
When source documents, ACLs, prompts, models, or retrieval policy change, an old cached value may no longer be safe. A production cache needs versioned keys or authoritative epochs, bounded retention, and authorization checks before reuse.
Pareto frontier decision tool
A configuration is on the frontier if no other configuration is at least as good on every chosen dimension and strictly better on one. Here the report maximized Recall, MRR, nDCG and minimized throttle-adjusted retrieval p95.
Provider throttle / quota wait latency boundary
A provider or account rate limit can make a request wait or return 429. That queue time is not model computation. Report it separately, honor Retry-After, and do not call the observed throttled latency intrinsic model latency.
Reasoning effort model setting
A model-control setting that trades internal deliberation and latency. The supplied benchmark report says Luna expansion was pinned to low, but the reviewed branch’s query_expansion.py does not currently send an explicit reasoning_effort field. Treat the A/B as benchmark evidence from its stated revision, not as a confirmed current-code contract.
Utility model control-plane role
A designated inexpensive or fast model used for supporting work such as query expansion. It is not necessarily the answer model. If it is missing, the MQR path falls back to the original query.
No-answer threshold / score space decision
Ragz compares the best available score against min_score. Without reranking that is dense-cosine space; with reranking it is cross-encoder score space. The same numeric threshold is not automatically equivalent across those spaces.
RAG-Triad evaluation family
A three-part answer evaluation commonly framed as context relevance, groundedness, and answer relevance. It complements retrieval metrics; it does not replace exact evidence matching, correctness review, security tests, or latency measurements.