Guides

Private retrieval learning & replay

Turn a real retrieval miss into a content-free regression fixture, then replay a candidate pipeline against it before you change anything.

Search quality normally improves by watching you. Clicks become implicit labels, labels become a model, and one morning ranking is different and nobody can reconstruct why. That trade is bad for a tool pointed at your private notes, so GNO inverts it: nothing is recorded until you switch it on, only judgments you state explicitly count, exported fixtures carry no source text, and replay never changes anything by itself.

What you get instead is a regression workflow. A query returned the wrong thing; you capture that as a fixture; you try a different pipeline against the fixture; you read the comparison and decide. The loop is the same one a search team would run, except it happens on your machine over your corpus and the artifacts belong to you.

The lifecycle

Record. With tracing enabled, retrieval operations emit a receipt ID on stderr. A trace spans the query, any Context Capsule built from it, document reads, and which evidence was opened, cited, or pinned.
Judge. You label outcomes. Nothing is inferred.
Export. Completed traces become a deterministic, content-free qrels file: identities and outcomes, no passages.
Replay. A candidate pipeline runs against that frozen baseline and reports what changed.
Decide. Replay can recommend promotion. It always returns applied: false. You apply it or you do not.

Inspecting traces

gno trace list --md
gno trace show <trace-id> --json

Traces are bounded and retention-limited. Listing is paginated with opaque cursors. A trace stays open while its operation is still in flight and only terminal traces can be exported, which stops a half-finished query from becoming evidence.

Labelling honestly

gno trace label <trace-id> --label relevant \
  --target gno://notes/decision.md
gno trace label <trace-id> --label missing-expected \
  --target '#abcdef'

Three labels, and the distinction between them matters:

Label	Means	Constraint
`relevant`	This result was actually useful.	Must match evidence the trace already recorded.
`irrelevant`	This result was returned and was noise.	Must match evidence the trace already recorded.
`missing-expected`	The document that should have ranked did not appear at all.	Accepts a `gno://` URI, a docid, or an immutable source hash. Document text is never copied into the judgment.

missing-expected is the one that pays for the whole feature. It encodes the recall failure you noticed, which is exactly the case a click-based system can never see, because you cannot click a result that was not shown.

Repeating a label is idempotent. A correction is appended rather than rewriting history, so the record of what you believed and when stays intact. Absence of a label is never a negative: a result you simply did not judge stays unjudged forever.

Exporting a fixture

gno trace export <trace-id> <another-trace-id> --output traces.json
gno trace export <trace-id> --format qrels --output qrels.json

Qrels export has a higher bar than plain export: it needs replay-mode receipts with complete query, filter, rank, hash, and exact-span provenance. What it writes is identities and outcomes. Source and mirror text are not copied, which is what makes a fixture safe to keep in a repository, attach to an issue, or hand to someone else, even when the underlying corpus is confidential.

Completed, partial, failed, and cancelled outcomes stay distinct in the export. None of them silently becomes a negative label.

Replaying a candidate

gno trace replay <qrels-export-id> --candidate bm25 --md
gno trace replay <qrels-export-id> --candidate hybrid \
  --candidate-limit 100 --no-expand --json

Replay verifies the local aggregate manifest before it runs, then:

reports final rank separately from planner rank, so you can see whether a change came from retrieval or from reranking;
classifies every source as unchanged, stale, missing, inactive, or unindexed, so a corpus that moved under the fixture is disclosed rather than quietly scoring worse;
preserves capability and fallback truth, so a comparison run without a rerank model does not read as a pipeline regression;
performs no network upload and mutates no ranking, prompt, model, configuration, trace, or user file.

The output can say a candidate looks better. It cannot enact that. Promotion is a human action, deliberately.

Retention, deletion, and purge

gno trace delete <trace-id>
gno --yes trace purge --json

The purge receipt reports physicalCleanup as completed, wal_busy, or failed. Only completed confirms the SQLite write-ahead log was actually truncated: a purge that says it deleted your data while the bytes are still recoverable in the WAL would be a lie, so GNO reports the difference. Recording can be turned off independently of managing receipts you already stored.

A worked example

You search for the incident postmortem and the wrong quarter’s document ranks first. The right one exists, you know its path, and it is nowhere in the results.

# 1. The query printed a receipt ID on stderr
gno trace show tr_9f2c --json

# 2. Record both facts: what was wrong, what was missing
gno trace label tr_9f2c --label irrelevant --target gno://notes/postmortem-q1.md
gno trace label tr_9f2c --label missing-expected --target gno://notes/postmortem-q3.md

# 3. Freeze it
gno trace export tr_9f2c --format qrels --output incident-recall.json

# 4. Does thorough mode fix it?
gno trace replay qx_51ab --candidate hybrid --candidate-limit 100 --md

Keep incident-recall.json and the next time you change an embedding model, rerun it. That is a regression test for retrieval quality, built from a real failure rather than a synthetic one, and it contains none of your text.

For fixtures you author from scratch rather than capture, see benchmarks and gno bench. For diagnosing a single miss without building a fixture at all, gno query diagnose "…" --target <uri> reports which retrieval stage dropped the document.