Repository navigation
Preserve incomplete WSL retrieval evidence and enforce provenance checks #156
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
81d72a8
Qualify bounded WSL source and lexical retrieval evidence
seathatflowsinourveins 628fa86
Register historical WSL retrieval under current evidence policy
seathatflowsinourveins ed6acc4
Anchor generated explorer inputs to current retrieval evidence
seathatflowsinourveins 16a0835
fix: register retrieval lockfile and lifecycle evidence
seathatflowsinourveins 35cf9e3
fix: bound historical retrieval evidence and preserve command provenance
seathatflowsinourveins 0cfa066
docs: anchor explorer to reviewed retrieval scope
seathatflowsinourveins File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
118 changes: 118 additions & 0 deletions
118
blueprints/convergence-practice/wsl-retrieval/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| # Bounded native WSL retrieval | ||
|
|
||
| The retained September 20, 2026 Linux x86_64 runs qualify two selected retrieval | ||
| fixtures: **22 source checks across six native commands** and **70 QMD lexical | ||
| checks across 17 native commands**. [source-receipt.json](source-receipt.json) | ||
| and [qmd-receipt.json](qmd-receipt.json) preserve selected command outputs, | ||
| stdout/stderr digests, exact frozen inputs, and outcomes. These are historical | ||
| WSL observations. The September 23 completion work performs offline checks only. | ||
|
|
||
| Under the current [acceptance evidence policy](../../../docs/acceptance-evidence-policy.md), | ||
| these are locally authored integration checks with synthetic document fixtures | ||
| and a frozen repository source file. No unchanged upstream test suite was recorded | ||
| for these attempts. The retained native command results and independent artifact | ||
| audit support this limited fixture scope, not upstream-suite acceptance. | ||
|
|
||
| The source fixture uses ripgrep **15.2.0** and ast-grep **0.45.3** against the | ||
| frozen accepted planner from repository revision | ||
| `6f74bc503ccecaaf6ccebd53677b23aedab50921`. Independent Python string and AST | ||
| oracles check the exact file, lines, byte spans, and returned source text. | ||
| Absent queries return empty results; ripgrep and ast-grep both returned the | ||
| expected exit 1 for these recorded negatives. | ||
|
|
||
| QMD **2.8.3**, Node **24.21.0**, better-sqlite3 **13.0.3**, and sqlite-vec | ||
| **0.1.9** use the exact [package lock](package-lock.json). The three-document | ||
| primary collection and one-document decoy exercise five positive queries, | ||
| two absent/cross-collection queries, Unicode paths and bodies, a one-line `get`, | ||
| process reopen, update, and deletion. Search output must preserve the named | ||
| index, collection, path, line, and exact body. Successful searches return one | ||
| document within 2,048 bytes; the one-line `get` is within 1,024 bytes. The final | ||
| database has integrity `ok`, three active documents, and zero vector rows. | ||
|
|
||
| These fixtures establish selected lexical and structural retrieval behavior. | ||
| They do not qualify semantic RAG, a production corpus, default-index routing, | ||
| Codex/Claude integration, a service, or model inference. The source test does not | ||
| cover general ignore rules or every language. Result limits are fixture | ||
| acceptance checks after process capture, not a streaming memory limit. | ||
|
|
||
| ## Preserved failure and provenance | ||
|
|
||
| The first QMD attempt stopped after five native commands because its checker | ||
| compared an entire URI with a bare collection/path. QMD returned the expected | ||
| document with `?index=wsl-retrieval-fixture`. The retained | ||
| [initial receipt](qmd-attempt-1.json) reports `qmd-positive-0_paths` as failed; | ||
| [run-initial.py.txt](run-initial.py.txt) preserves its executed bytes. The | ||
| correction in [run.py](run.py) parses the URI, checks the named index and | ||
| collection, and decodes the path. Attempt 2 passed. Both attempts remain in the | ||
| [convergence contract](experiment.json); the failed attempt is not an absent | ||
| answer or an extra successful run. | ||
|
|
||
| [pins.json](pins.json) preserves the exact release, binary, package, and license | ||
| identities. [source-review.json](source-review.json) extracts the existing | ||
| September 20 official-source preflight without private host paths. Its release | ||
| currency is dated to that review, not September 23. The package-install | ||
| [receipt](install-receipt.json) records successful lock creation and `npm ci` | ||
| with `--ignore-scripts --omit=optional`; [inventory](install-inventory.json) | ||
| records the Linux sqlite-vec binding and no optional llama backend or model | ||
| files. No install or download occurs in the offline completion checks. | ||
|
|
||
| Each native runner writes full per-command stdout/stderr, a partial receipt, | ||
| and the final receipt under its private run directory. The prior handoff reports | ||
| that these logs were retained, but their exact private subdirectories were not | ||
| available to the offline completion worker. The checked-in digests and selected | ||
| facts remain inspectable; the full historical raw logs were not reopened. | ||
| Hashes establish consistency of the retained evidence, not independent proof of | ||
| historical execution. The independent [audit](audit.py) verifies frozen runner | ||
| mappings and inputs, exact source spans and QMD bodies, the failed URI result, | ||
| dependency-lock identity, and declared scope. | ||
|
|
||
| ## Offline verification | ||
|
|
||
| From the repository root: | ||
|
|
||
| ```sh | ||
| python3 blueprints/convergence-practice/wsl-retrieval/audit.py | ||
| python3 -m unittest tests.test_wsl_retrieval -v | ||
| python3 scripts/validate_convergence.py \ | ||
| blueprints/convergence-practice/wsl-retrieval/experiment.json --root . --json | ||
| python3 scripts/validate.py | ||
| ``` | ||
|
|
||
| [verification.json](verification.json) records completion check exits and | ||
| limits. The 18 focused tests reject mismatched spans, bodies, URI scope, | ||
| unbounded `get` content, error-as-absence, stale updates, model artifacts, | ||
| erased failed attempts, changed frozen inputs, and unsupported usage claims. | ||
| The contract binds the receipts, both runners, fixtures, dependency metadata, | ||
| audit, and tests to their SHA-256 identities. Shared catalog registration and | ||
| publication belong to the coordinator. | ||
|
|
||
| ## Deliberate native reproduction | ||
|
|
||
| Reproduction requires a reviewed Linux x86_64 host, the exact pinned native | ||
| executables and an already prepared private package prefix matching the lock. | ||
| Use a new private directory outside the checkout for each mode. The runner | ||
| does not install dependencies or fetch models: | ||
|
|
||
| ```sh | ||
| python3 blueprints/convergence-practice/wsl-retrieval/run.py --mode source \ | ||
| --run-dir "$NEW_PRIVATE_SOURCE_RUN" \ | ||
| --source blueprints/convergence-practice/native-worker/accepted/planner.py \ | ||
| --rg "$PINNED_RIPGREP_BINARY" --ast-grep "$PINNED_AST_GREP_BINARY" | ||
| python3 blueprints/convergence-practice/wsl-retrieval/run.py --mode qmd \ | ||
| --run-dir "$NEW_PRIVATE_QMD_RUN" \ | ||
| --node "$PINNED_NODE_BINARY" --package-prefix "$OWNED_PACKAGE_PREFIX" | ||
| ``` | ||
|
|
||
| The current runner includes the URI correction; it cannot recreate the initial | ||
| checker's failure. Both original source-run and failed-QMD receipts map their | ||
| `run.py` hash to `run-initial.py.txt`. Reproduction would be a new observation, | ||
| not a refresh of the historical receipts. | ||
|
|
||
| QMD gets explicit `QMD_CONFIG_DIR`, `XDG_CACHE_HOME`, `QMD_FORCE_CPU=1`, and | ||
| `--index wsl-retrieval-fixture`. The real HOME is preserved. Only the copied | ||
| synthetic corpus is indexed, updated, or deleted. No persistent service starts, | ||
| no model command runs, and no native client, account, global index, or history | ||
| capture is configured. This is scoped execution, not OS confinement or an | ||
| independent network-traffic audit. Rollback is retiring only owned trial files | ||
| after preserving required evidence. Whole-task parent/child/retry/cache usage | ||
| remains unknown; no token-saving claim is made. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| """Offline assessment of frozen inputs and selected native retrieval results.""" | ||
| import ast | ||
| import hashlib | ||
| import json | ||
| from pathlib import Path | ||
| from urllib.parse import parse_qs, unquote, urlsplit | ||
|
|
||
| ROOT = Path(__file__).resolve().parent | ||
|
|
||
|
|
||
| def require(condition, message): | ||
| if not condition: | ||
| raise ValueError(message) | ||
|
|
||
|
|
||
| def audit(source, qmd, failed, inventory, root=ROOT): | ||
| oracle = json.loads((root/'oracle.json').read_text()) | ||
| code = (root/'seed/planner.py').read_bytes() | ||
| require(hashlib.sha256(code).hexdigest() == oracle['source_sha256'], 'changed source oracle') | ||
| for receipt, mapping in [(source, {'run.py': 'run-initial.py.txt'}), | ||
| (failed, {'run.py': 'run-initial.py.txt'}), (qmd, {})]: | ||
| require(receipt['frozen_file_mapping'] == mapping, 'wrong executed runner provenance') | ||
| for name, digest in receipt['frozen_inputs'].items(): | ||
| require(hashlib.sha256((root/mapping.get(name, name)).read_bytes()).hexdigest() == digest, | ||
| 'changed frozen input: '+name) | ||
| require(receipt['cleanup'] == {'persistent_services_started': 0, 'timed_out_commands': 0}, 'incomplete cleanup') | ||
| for claim in ['global_or_client_changes', 'inference_invoked', 'os_confinement_established']: | ||
| require(receipt[claim] is False, 'unsupported claim: '+claim) | ||
| require(receipt['whole_task_provider_usage'] is None, 'whole-task usage is unknown') | ||
| require(receipt['platform']['system'] == 'Linux' and receipt['platform']['architecture'] == 'x86_64', 'wrong native target') | ||
| for receipt, mode, check_count, commands in [(source, 'source', 22, 6), (qmd, 'qmd', 70, 17)]: | ||
| require(receipt['passed'] is True and receipt['mode'] == mode and len(receipt['checks']) == check_count | ||
| and all(v is True for v in receipt['checks'].values()), 'required native checks incomplete') | ||
|
seathatflowsinourveins marked this conversation as resolved.
Resolved
|
||
| require(receipt['native_commands'] == commands and len(receipt['facts']) == commands, 'native commands missing') | ||
|
|
||
| def named(receipt): | ||
| items = {f['label']: f for f in receipt['facts']} | ||
| require(len(items) == len(receipt['facts']), 'duplicate command label') | ||
| return items | ||
|
|
||
| src = named(source) | ||
| require(set(src) == {'ripgrep-version', 'ast-grep-version', 'rg-positive', 'rg-negative', 'ast-positive', 'ast-negative'}, 'source command set changed') | ||
| require(all(f['exit_code'] == 0 for label, f in src.items() if label not in ['rg-negative', 'ast-negative']), 'source command failed') | ||
| require(src['rg-negative']['exit_code'] == 1 and src['rg-negative']['match_count'] == 0, 'wrong rg negative') | ||
| require(src['ast-negative']['exit_code'] in [0, 1] and src['ast-negative']['matches'] == [], 'wrong AST negative') | ||
| require(src['ripgrep-version']['version_output'].startswith('ripgrep 15.2.0 '), 'wrong rg version') | ||
| require(src['ast-grep-version']['version_output'] == 'ast-grep 0.45.3', 'wrong AST version') | ||
| lines = code.decode().splitlines(keepends=True) | ||
| expected = [i for i, line in enumerate(lines, 1) if oracle['source_literal'] in line] | ||
| matches = src['rg-positive']['matches'] | ||
| require([m['line'] for m in matches] == expected == oracle['source_expected_line_numbers'], 'wrong lexical lines') | ||
| for match in matches: | ||
| require(match['path'] == 'frozen/seed/planner.py' and match['line_text'] == lines[match['line']-1], 'wrong lexical source') | ||
| raw = match['line_text'].encode() | ||
| start = raw.index(oracle['source_literal'].encode()) | ||
| require(match['spans'] == [{'start': start, 'end': start+len(oracle['source_literal'].encode()), 'text': oracle['source_literal']}], 'wrong literal byte span') | ||
| expected_ast = sorted([{'start_line': n.lineno, 'start_column': n.col_offset, 'end_line': n.end_lineno, 'end_column': n.end_col_offset} | ||
| for n in ast.walk(ast.parse(code)) if isinstance(n, ast.Raise) and isinstance(n.exc, ast.Call) | ||
| and isinstance(n.exc.func, ast.Name) and n.exc.func.id == 'ValueError'], key=lambda x: x['start_line'] or 0) | ||
| matches = src['ast-positive']['matches'] | ||
| spans = [{'start_line': m['range']['start']['line']+1, 'start_column': m['range']['start']['column'], | ||
| 'end_line': m['range']['end']['line']+1, 'end_column': m['range']['end']['column']} for m in matches] | ||
| require(spans == expected_ast == oracle['source_ast_spans'], 'wrong AST ranges') | ||
| for match in matches: | ||
| byte_range = match['range']['byteOffset'] | ||
| require(match['file'] == 'frozen/seed/planner.py' and len(match['text'].encode()) <= 512 | ||
| and code[byte_range['start']:byte_range['end']] == match['text'].encode(), 'wrong AST source bytes or bound') | ||
| qfacts = named(qmd) | ||
| expected_labels = {'node-version', 'qmd-version', 'qmd-add-primary', 'qmd-add-decoy', 'qmd-bounded-get', | ||
| 'qmd-reopen', 'qmd-update', 'qmd-updated', 'qmd-old-marker', 'qmd-deleted'} | ||
| expected_labels |= {'qmd-positive-'+str(i) for i in range(5)} | {'qmd-negative-'+str(i) for i in range(2)} | ||
| require(set(qfacts) == expected_labels and all(f['exit_code'] == 0 for f in qfacts.values()), 'QMD command missing or failed') | ||
| require(qfacts['node-version']['version_output'] == 'v24.21.0' and '2.8.3' in qfacts['qmd-version']['version_output'], 'wrong QMD runtime') | ||
|
|
||
| def query(label, text, path, body=None): | ||
| fact = qfacts[label] | ||
| require(fact['query'] == text, 'query substituted') | ||
| rows = fact['rows'] | ||
| if path is None: | ||
| require(rows == [], 'negative query returned content') | ||
| return | ||
| require(len(rows) == 1 and fact['stdout_bytes'] <= 2048, 'result count or byte bound exceeded') | ||
| uri = urlsplit(rows[0]['file']) | ||
| require(uri.scheme == 'qmd' and uri.netloc == oracle['qmd_collection'] and unquote(uri.path) == '/'+path | ||
| and parse_qs(uri.query) == {'index': [oracle['qmd_index']]} and not uri.fragment, 'wrong source collection/path/index URI') | ||
| require(rows[0]['body'] == (body if body is not None else oracle['corpus']['primary/'+path]), 'wrong exact source body') | ||
| require(rows[0]['line'] == 2, 'wrong returned source line') | ||
|
|
||
| for i, case in enumerate(oracle['queries']): | ||
| query('qmd-positive-'+str(i), case['query'], case['file']) | ||
| for i, text in enumerate(oracle['negative_queries']): | ||
| query('qmd-negative-'+str(i), text, None) | ||
| query('qmd-reopen', 'Cobaltcheckpoint', 'recovery.md') | ||
| query('qmd-updated', oracle['after_update_marker'], 'recovery.md', oracle['updated_recovery']) | ||
| query('qmd-old-marker', 'Cobaltcheckpoint', None) | ||
| query('qmd-deleted', 'Amberproject', None) | ||
| fact = qfacts['qmd-bounded-get'] | ||
| header, body = fact['output'].split('\n---\n\n', 1) | ||
| require(header.startswith('qmd://'+oracle['qmd_collection']+'/recovery.md #') | ||
| and body == oracle['corpus']['primary/recovery.md'].splitlines()[1]+'\n' | ||
| and fact['stdout_bytes'] <= 1024, 'get escaped its exact one-line provenance bound') | ||
| require(qmd['database'] == {'integrity': 'ok', 'active_documents': 3, 'content_vectors_table': True, 'content_vector_rows': 0}, 'QMD state or embedding scope changed') | ||
| require(qmd['model_files_in_run'] == [] and inventory['model_files_under_owned_prefix'] == [] | ||
| and inventory['installed_optional_llama_backends'] == [], 'model/backend artifact appeared') | ||
| require(inventory['installed_qmd'] == '2.8.3' and inventory['installed_better_sqlite3'] == '13.0.3' | ||
| and inventory['installed_sqlite_vec_linux_x64'] == '0.1.9' and inventory['all_registry_payloads_have_integrity'] is True, 'locked native dependency scope changed') | ||
| require(inventory['lock_sha256'] == hashlib.sha256((root/'package-lock.json').read_bytes()).hexdigest(), 'dependency lock changed') | ||
| require(failed['passed'] is False and failed['native_commands'] == 5 and len(failed['facts']) == 5 | ||
| and failed['checks']['qmd-positive-0_paths'] is False | ||
| and failed['failure'] == {'type': 'AssertionError', 'message': 'qmd-positive-0_paths'}, 'initial native failure erased or relabeled') | ||
| failed_row = named(failed)['qmd-positive-0']['rows'] | ||
| require(failed_row == qfacts['qmd-positive-0']['rows'], 'initial failure was not the documented URI format assumption') | ||
| return {'evidence_consistent': True, 'source_checks': 22, 'qmd_checks': 70, | ||
| 'accepted_native_commands': 23, 'retained_failed_commands': 5, | ||
| 'whole_task_provider_usage': None, 'semantic_rag_or_client_integration': False} | ||
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| load = lambda name: json.loads((ROOT/name).read_text()) | ||
| print(json.dumps(audit(load('source-receipt.json'), load('qmd-receipt.json'), | ||
| load('qmd-attempt-1.json'), load('install-inventory.json')), indent=2)) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.