Retrieval Verification Contracts

Offline CLI

No network/auth/models/writes/uploads/retries/cleanup or citation-URL following. Use retained responses, not custom parsers/dumps or new queries. Never change effort/output/models/permissions/limits.

Exit 0: review-required, not acceptance. Exit 2: blocked. overall_pass is always false. structural_status: passed/unverified/failed; failure dominates. Review semantics separately. REST isn’t agent proof; offline timestamps don’t prove live provenance.

Closed input

One source, two exact requests; paths resolve beneath the input’s directory. Reject absolute/traversing/link paths; URI/network paths block before filesystem parsing on every platform. UTF-8 JSON: input/snapshot 1 MiB each, response 5 MiB each, nesting 48, arrays at most 1000 entries. Duplicate keys, malformed/truncated JSON, missing payloads and non-200 HTTP (including 206) cannot pass.

Never refresh timestamps. Questions differ; basis: title/content versus unrelated records judgment, not semantic classification. File path: exact fileName. For Blob/ADLS use source_kind: azureBlob, path as exact native blobUrl, and etag instead of file_id.

Local snapshot: captured_at, endpoint, api_version, knowledge_base, knowledge_source, original. Definitions are complete retained GET bodies with @odata.etag; KB knowledgeSources must be exactly [{"name":"<source_name>"}]. original: uniquely selected successful File-list record across all retained pages, with fileName, fileId, lastUpdatedAt, errorMessage: null. For Blob, use { "url":"<original blob URL>", "etag":"<original HEAD ETag>" }, not an indexed chunk. Bind to authorized inventory; no file listing/Blob HEAD/content inspection or verification of the operator’s unique selection.

Definitions/originals must match exactly, including ETags/models. Order: before ≤ both responses ≤ after ≤ current time, with before at most one hour old. Missing/unsupported/old evidence blocks; retain failure, request evidence, no ingestion replay. Equality isn’t ingestion-version proof or atomic Storage.

Responses: knowledge_base_retrieve.py envelopes (http_status, response_body). Optional producer metadata: status must be response-received/blocked; operation must be retrieve. Other values block. blocked needs a first_blocker object and always fails; no blocker on success. Keep exact errors/request IDs private. Output: codes/counts/pointers/hashes and first error/warning locations, never messages/documents/paths/questions/secrets/IDs. Caps: 40 findings/20 diagnostics; total counts and failure severity stay accurate.

Wire and identity limits

GA 2026-04-01: Blob minimal/extractive; GET has no mode/effort fields. Preview 2026-08-01-preview: File/Blob/ADLS, minimal/low/medium, extractive/synthesis; snapshot selects mode/effort/model. Minimal uses intents above; low/medium uses {"messages":[{"role":"user","content":[{"type":"text","text":"<query>"}]}]}. Overrides block. Synthesis/low/medium need KB models, not invocation proof.

REST response[].content[].text is JSON-encoded extract rows with ref_id in extractive mode; synthesis uses native [ref_id:<id>] markers. All text blocks are inspected. Used IDs must resolve to unique references[].id, then integer activitySource to the selected source activity. activity[].error fails; activity[].warning leaves completeness unverified, including token/score drops. Absent/null diagnostics are empty. Warnings must be strings (empty allowed); errors must be objects, even {} fails. Wrong types block before truthiness.

Native File references expose docName, not File ID. Blob references expose blobUrl, not ETag. Thus path-only native citations correctly return original-identity-not-in-native-citation, not a pass. docKey, citationUrl, reference IDs and chunk sourceData.id are never original identity/version proof. sourceData is schema-dependent and can be null.

expected_source.identity_field is only an operator-declared value comparison. declared_identity_value_matches counts equality, not original-source proof. original_identity stays unverified; the native File ID/ETag gap remains even with matching arbitrary data. Missing values are unverified; mismatches fail. No attestation upgrades a mapping. No stronger binding contract is implemented. Don’t invent fields, relabel chunks, fill missing data or retrofit indexes.

Unrelated extractive needs serialized [] with no references, not missing blocks; this proves only structural absence. Synthesis without citations/references does not prove semantic abstention: review all claims/exact-output requirements. Unused references aren’t answer citations. No keyword/phrase matching.

Filename search limitation — explain before ingestion

File/Blob expose no supported switch for searchable generated filenames. File user metadata isn’t searchable/filterable; listing search/prefix isn’t KB full-text retrieval. Citation/filter fields need actual searchable: true. Filename words can still match document content. No generated-index patches, reuploads or extra metadata/schema/models. Customer-managed searchable fields need separate design/cost/data approval, not an option silently added to these executors.

Authorities: failure/conflict/uncertainty only. Checked 2026-09-14:

• Retrieve wire definitions (opens in a new tab)
• GA retrieve (opens in a new tab)
• Response and references (opens in a new tab)
• Synthesis citations (opens in a new tab)
• File metadata and listing (opens in a new tab)
• Source configuration schema (opens in a new tab)