Benchmarks

Benchmark retrieval quality from the public CLI against local fixtures before changing models, modes, or defaults. GNO's own research loop runs the same way, and every result ships in the repository against a pinned corpus, so a ranking change can be replayed before it is adopted.

Install GNO Read the docs

Use cases: Evaluating a new embedding or reranker before shipping; Comparing candidates against a stable baseline; Catching retrieval regressions with a fixture your team owns

What it gives you

Public gno bench <fixture> command for repeatable checks
Agentic outcome benchmark measures evidence coverage, tool calls, context, and stopping behavior
Promoted Context Capsule result: 100% task completion, 48.94% fewer calls, and 44.12% less visible context
Frozen fixture lanes record degraded fallback behavior so regressions surface before adoption
BM25, vector, and hybrid mode comparisons from the same fixture
Recall@K, precision, nDCG, MRR, and latency metrics
Per-query misses and hit lists for debugging regressions
JSON output for CI, model experiments, and release notes
Example fixture schema for team-owned benchmark corpora

Try it yourself

Representative commands and entry points. Full reference lives in the documentation.

gno bench docs/examples/bench-fixture.json

gno bench fixture.json --modes bm25,hybrid

gno bench fixture.json --json

Frozen agent outcome

One task. Three lanes. Raw receipts included.

One controlled exact-identifier task—not a general superiority claim. All lanes share the task, fixture agent, corpus, effective index, trial, seed, and cold lifecycle.

Selection and prototype boundary

Selection rule: Selected task t0a1b2c3 is the sole cold-lifecycle current-GNO failure / Capsule success case among the authoritative 24-task cohort.

Prototype boundary: The Capsule lane is an evaluation-only lexical prototype. Its 2.191 ms latency is not the shipped Context Capsule path and is not product-equivalent.

Task and exact evidence

Find the incident identifier assigned to the failed north gateway deployment.

Expected answer: INC-4827

gno://c001/d001.md:3-3

Source SHA-256: f02c60996c5114f6ceb2f7cbfc96094f7a8df034ffd1bb62dfb841fba37099aa
Span SHA-256: db94cbe64c5c6c10d582eff7ff7d18fd2410bcbe8bbb1af42281bbfb7d69f320

Lane	Stop outcome	Success	Evidence coverage	Agent calls	Context bytes	Tokens	Cold end-to-end
Lexical-only baseline	complete: INC-1042	0	0	2	2744	unavailable	1.283 ms
Current GNO primitives	complete: INC-1042	0	0	2	2222	unavailable	1375.758 ms
Context Capsule	complete: INC-4827	1	1	1	1295	unavailable	2.191 ms

Method and variance

Hidden-oracle exact values and line/hash coordinates score the final structured envelope without an LLM judge. Model-visible UTF-8 bytes include each complete normalized tool-result envelope.

One deterministic frozen trial. No statistical variance estimate; latency is only the matching cold lifecycle on the recorded environment.

Capsule request: search {"collection":"c001","query":"incident identifier"}
Effective index: 9c4a65f08850a70ff6d2ec6fe069b4cd38c9cbf3481583afd76a74264cf0591a
Fallbacks: none

Download the evidence

Canonical demo fingerprint: bb5c059572017e71a126233dfdeb38e547c4d87013882c833d45570da578d85f

Separate answer-enforcement proof

Verified Ask is not a fourth retrieval lane

Frozen 22-pair raw Ask / verified Ask cohort. Two expected-missing-evidence tasks excluded: t234cd5e, t345de6f.

Answer accuracy stayed 0.8181818181818182 / 0.8181818181818182. Unsupported substantive claims moved from 4 to 0. These are answer-enforcement metrics, not retrieval metrics.

Canonical fingerprint: 53931a6ad5717a707e2a486d47fafb988960b0e0f45f201c5875d1104034ee28. Download the immutable Verified Ask artifact.

Keep reading