Phase C: NautilusTrader vs hftbacktest cross-check with an independent oracle; engines agree after fixing our driver's use-after-free

[seathatflowsinourveins]

What

This is Phase C of the sim-lane engine work. The same bounded order stream is replayed through NautilusTrader 2.0.0rc5, the sim-capacity engine from #235, and hftbacktest 2.4.4, the independent engine installed in #232. Each order's outcome is then checked against a new engine-independent oracle built from the retained SIP quotes.

• Sample: SPY and NVDA, 2026-09-24 14:05–14:13Z.
• Order stream: 1,439 orders at 3 per second, marketable limits with a 1¢ collar, quantity 5.
• Configurations: 70 ms and 250 ms latency, and both hftbacktest exchange models.

Result

The engines agree, and the sim-capacity overturn condition does not fire.

Config	Fill rate	Median time-to-fill	Oracle agreement
Nautilus 70 ms, as in sim-capacity	97.64%	111.23 ms	99.65%
Nautilus 70 ms, exact latency	97.71%	70.00 ms	99.72%
hftbacktest 70 ms (either exchange model)	97.98%	70.00 ms	100.0%
Nautilus 250 ms, as in sim-capacity	94.23%	290.60 ms	99.72%
Nautilus 250 ms, exact latency	94.44%	250.00 ms	99.93%
hftbacktest 250 ms (either exchange model)	94.51%	250.00 ms	100.0%

• No fill beats the displayed quote. There are zero better-than-touch fills across all 8 runs, and the report build now fails if one appears.
• The overturn rule is quoted verbatim from sim-capacity/experiment.json and evaluated on these corrected numbers. Fill rates differ by 0.30–0.36% (the threshold is 20%), and the median time-to-fill ratio is 1.16–1.59× (the threshold is 2×).

The first attempt was wrong, and it is kept on record

The first attempt (d5cc0c5) reported a divergence of more than 20%, all of it in NVDA (hftbacktest 53% against Nautilus 97%). It explained the gap as a model difference (F1) and a suspected upstream bug (F2).

An independent adversarial review (Opus 5.5) built its own oracle and refuted both. The real cause was a use-after-free in our own driver: NVDA's event array was freed while hftbacktest's Rust reader still held a non-owning pointer to it.

• 72,134 of 74,835 NVDA events changed in memory.
• Reruns on the same input gave different results, and one run hung.

The repair round (08a684e):

• keeps every event array referenced until the engine closes;
• submits orders at exact times, and counts filled quantity as qty − leaves_qty;
• adds oracle.py, a hard better-than-touch check and a determinism command;
• withdraws F1 and F2.

The superseded receipt is kept byte-identical as receipts/20260925-crosscheck-superseded.json. The sim-capacity record marks the old observation failed, then appends the correction.

A real, separate Nautilus finding

This is a timing-realism issue. It does not make fills optimistic.

• Behaviour: Nautilus releases a latency-delayed order at the next quote on its own instrument, or at the next timer.
• Size of the slack: a median of 28 ms on SPY and 53 ms on NVDA beyond the configured latency.
• Fix, tested: one time alert per order at submit + latency. Every order then resolves at exactly the configured latency.
• Effect on prices: they stayed within 1¢ of the touch at submit + latency.
• Follow-up: apply the fix to sim-capacity's own order generator in a separate PR, then re-run its receipts.

Review

1. Substance review (Opus 5.5). Refuted the first conclusion, as above.
2. Repair (the builder).
3. Re-check of the fix (Opus 5.5). Ready.
   • It matched all 8 configurations with its own oracle.
   • It confirmed the fix from hftbacktest source.
   • The isolation test failed 5 of 5 with the bug patched back in.
   • Its minor corrections (which tests really guard, attribution of the residual gap, the locked-quote note, and three citation nits) are recorded in the README's "Independent re-check" section. The test now pins hftbacktest's agreement at its measured 100%.

Codex cross-family review is unavailable until 2026-09-30, so it was skipped.

Checks

• This branch's tests: tests.test_sim_engine_crosscheck and tests.test_sim_capacity pass on system Python: 113 tests, of which 32 are engine-gated skips. The builder ran the cross-check tests on both pinned runtimes, and the re-check reran them with hftbacktest installed.
• Validators: validate.py, evidence_manifest.py --check and validate_convergence.py --all-recorded pass.
• Secret scan: gitleaks is clean on every commit.
• Main merged in at 4b594d4, keeping both sides' appended limitations in sim-capacity/experiment.json.

Evidence class and limits

• Class: a simulation cross-check, infrastructure evidence only. It is not broker throughput and not strategy performance (NS-06).
• Coverage: one 8-minute, two-symbol sample, 2 latency points and 2 exchange models.
• Private files: per-order outcomes and the sample stay host-local. Only aggregates and hashes are committed.
• Open residuals: the report attaches oracle agreement but does not enforce it at build time, and metrics.py skips fills that have no quote in force.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.