Repository navigation
Sim-to-paper fill agreement: replay a retained Alpaca paper trial through NautilusTrader rc5 (audit gap #7) - #218
Merged
seathatflowsinourveins merged 1 commit intoSep 24, 2026
Conversation
…ough NautilusTrader rc5 (audit gap #7) Replays paper trial 20260923g-main-passed (5 orders) through NautilusTrader 2.0.0rc5's simulated exchange on the same SIP quotes, retained as a hash-ledgered page cache. Scores sim-vs-paper fill agreement per order. Results (receipts/20260923g-main-passed.json): - Agreement is 3/5 at 0-50 ms latency and 5/5 at 70 ms and above. - Bisection puts the flip between 69.216918 ms (disagrees) and 69.217529 ms (agrees). It is explained by order 4 being marketable for 69.273 ms after submit. - Order 5's flip depends on order 4, confirmed by a counterfactual sweep. Disclosures: - Engine settlement semantics: a command settles on its own instrument's quotes or on any due timer. - Clock provenance: the host leads the broker by at least 28-41 ms. - Cancel pairing is a uniqueness rule whose assumptions and counterexamples are stated. This is one trial and an agreement measurement, not a calibration. Reviewed across five rounds by Claude and Codex (PR #217, where the per-round commits live). This branch carries the final tree as one commit so that no intermediate commit holds the request digests an earlier receipt stored under a field named "key", which gitleaks' generic-api-key rule flags. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
seathatflowsinourveins
deleted the
claude/sim-paper-fill-compare-20260924b
branch
September 24, 2026 22:52
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 25, 2026
…resolve every allocated generation, and harden the warm-monitor preflight Four low-severity items from the final independent review, plus the test gaps two of them exposed: 1. An ambiguous-only response (a previously confirmed record whose row loses its governing date on a later fetch) wrote an empty list over the last good records in complete_attempt, turning a confirmed must_flatten back into mere block-and-flag and losing an already-live intraday exit signal. Fixed: an ambiguous symbol with zero valid records in the new response now degrades (preserving the last good records) instead of overwriting them with []. 2. Mutation G4 (drop the scheduler's refresh_timed_out call for a two-phase guard) survived because every existing short-timeout test drove a legacy single-arg guard, never the real CorporateActionMonitor's own generation-aware timeout path. Added a test driving the real monitor through a short-timeout scenario, asserting the symbol is flagged and the next attempt is allowed after retry_seconds (60s), not throttled for the ordinary refresh_seconds (900s). 3. A pre-allocated generation was left permanently unresolved if Thread.start() itself raised, or if an unexpected exception escaped the worker's own run_attempt call -- refreshes would then be silently throttled with nothing ever flagged for up to 900s. Both paths now call fail_attempt on the pre-allocated generation before propagating. 4. _preflight_corporate_action_refresh unconditionally awaited _schedule_corporate_action_refresh's return value, which is None (not a task) when the two-phase begin_attempt itself reports a throttled no-op -- a second preflight call on an already-warmed monitor raised TypeError from awaiting None. Now only awaits when a task was actually created. The review's own mutation script also caught two gaps in the round-5 fixes themselves once the two-phase API was exercised more directly: G5 (a cancellation mutated to a no-op) survived because the existing test only checked that a clean end state stayed clean, which held regardless of whether invalidate_attempt ran; strengthened to assert the generation counter is actually bumped past the cancelled attempt. A generation- specific refresh_timed_out check reverted to a round-4-era coarse heuristic (comparing only _generation to _committed_generation) also survived; added a test where a stale timeout for an already-superseded (but not yet committed) generation must be a complete no-op. Merged origin/main (added #206 and #218 since the round-5 push); source-hashes.json and manifests/evidence.json were recomputed from current file content for every changed path, not resolved by picking a side. No overlap between main's changes and this branch's files beyond manifests/evidence.json, which merged cleanly. Re-ran: the full adaptive suite (373 tests) with network-blocked pagination tests, test_native_faults_min, test_order_throughput, test_order_contract, test_trading_gates, all six validators (validate.py, validate_catalogs.py, landscape.py, evidence_manifest.py, verdict_review_gate.py, trading_gates.py -- all passed), and the reviewer's own mutate_rev5.py retargeted at this worktree (65 mutations, all killed except one PATTERN_COUNT=0 entry already superseded by an equivalent round-5 mutation of my own). Ran gitleaks git history scan (509 commits, 1.12 GB) with the pinned binary and repo config: all 76,054 findings are pre-existing sourcegraph-access-token/generic-api-key pattern matches inside the generated docs/ecosystem/index.html artifact; zero findings in any file this branch touches. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 25, 2026
…-09-24/25 merges (#225) Updates four checkpoint entries: the paper lane, the simulation lane, alpaca-paper-operational and corporate-action-data. They now reflect #198, #199, #212, #218, #219 and #222. tests.test_grand_dashboard passes, and all state strings satisfy progress.py's public-token rule. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Supersedes #217, where the five review rounds and their per-round commits live. This branch carries that final tree as one commit on current main, so no intermediate commit holds the request digests an earlier receipt stored under a field named
key. gitleaks'generic-api-keyrule flags those digests, and the repository's.gitleaksignorepolicy reserves fingerprints for two reviewed narrative files, so this is resolved by history rather than an ignore entry. The receipt's page field is nowrequest_digest.Trading-lane audit gap #7 (sim-to-paper agreement), first measurement.
blueprints/us-equities/sim-paper-compare/replay_compare.pyreplays the retained Alpaca paper trial20260923g-main-passed(5 orders) through NautilusTrader 2.0.0rc5's simulated exchange. It uses the same SIP quotes, retained as a hash-ledgered page cache. It then scores sim-vs-paper fill agreement per order.This is one trial and an agreement measurement, not a calibration.
Result (
receipts/20260923g-main-passed.json)Bisected flip. The simulator disagrees with paper at 69.216918 ms and agrees at 69.217529 ms. Order 4 was already marketable at submit and stopped being marketable 69.273 ms later. With less latency than that, the simulator fills it; paper did not.
Disclosed engine semantics (pinned rc5
engine.rs). A delayed command settles on either of two events:A foreign instrument's quote alone does not settle it. So sweep times and prices are not a function of the latency L alone.
Clock provenance. Submit times come from the broker. Cancel times come from host stamps taken before send, and the host leads the broker by at least 27.99–40.77 ms. Cancel pairing uses an open-order uniqueness rule that refuses ambiguous cases. Its assumptions and two constructed counterexamples are stated in the README. Logging
client_order_idon cancel requests would remove the heuristic.Review
Five fix rounds with two model families.
--replay, and ran a 200k-trial differential fuzz of the pairing rule. They mutation-tested with about 25 mutants, each now killed by a behavioural test.Checks.
-b.manifests/evidence.json.--replayagainst the retained cache.Not covered
🤖 Generated with Claude Code