diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/README.md b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/README.md new file mode 100644 index 000000000..cdd9380bb --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/README.md @@ -0,0 +1,524 @@ +# IBKR paper orders on NautilusTrader 2.0.0rc5 + +This build serves the trading north star's bounded SPY paper order qualification +on the selected NautilusTrader destination. It uses the upstream **built-in +ExecTester**, LiveNode and native Interactive Brokers factories. The four case +labels mirror the [frozen 1.231.0 trial](../ibkr-paper-orders/README.md): accept a +resting buy, cancel it, fill a one-share buy, flatten and independently prove flat. + +Round r2 enables the bounded trial through runner quote admission and quantity +one while keeping the native risk engine enabled and its cap configured. The +selected pin skips the cap on the SMART/IB route. The receipt retains the +requested statement exactly: "engine notional cap configured, not enforced on +this route (#4946, fixed on develop, unreleased); bound held by qty=1 and quote +admission". Its adjacent release verification corrects the dated "unreleased" +clause: v2.0.0rc6 was published on 2026-10-05 and includes the fix. This build +still pins rc5. The coordinator owns review and broker execution; offline tests +do not establish the `ibkr-local-acceptance` gate. + +## Source and decision, 2026-10-05 + +The selected source is `nautechsystems/nautilus_trader`, tag `v2.0.0rc5`, commit +`1b0a49d2792a9432a3aca3fcb617ce7a630d905e`: + +- [exec_tester.py](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/examples/live/interactive_brokers/exec_tester.py) + supplies the builder, IB configs/factories and ExecTester registration. +- [_common.py](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/examples/live/interactive_brokers/_common.py) + supplies supported timeout and shutdown builder methods. +- [ExecTester strategy](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/testkit/src/testers/exec/strategy.rs) + supplies the actual market entry, resting-limit maintenance, cancel and close + behavior. The runner does not implement another order strategy. +- [Hosted LiveNode bindings](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/live/src/python/node.rs) + support `run_async()` on a Python event loop and independent cache/control + handles captured before starting it. The wrapper owns signal handling. +- [Risk-engine source](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/risk/src/engine/mod.rs#L1218) + returns success when the account lookup for the instrument venue fails, before + applying the configured notional check. The example's instrument venue is + `SMART`; the adapter uses the broker venue `IB`. + +The installed rc5 signatures were inspected and the actual ExecTester, IB and +LiveRiskEngineConfig constructors were exercised offline. The +[rc5 release notes](https://github.com/nautechsystems/nautilus_trader/releases/tag/v2.0.0rc5) +and original tagged Rust sources were also checked. Upstream +[#4946](https://github.com/nautechsystems/nautilus_trader/issues/4946) is **closed**, +but the missing-account branch remains in this selected tag; closed issue state +does not establish a fix in the installed pin. This is source evidence, not an +observed broker failure. + +The command center's r2 decision preserves rc5 and the native strategy and +holds admission in the runner. Official client 92 requests timestamped +`reqTickByTickData(..., "BidAsk", 0, False)` followed by `reqCurrentTime()`. +The installed official ibapi 10.45.1 callback signatures were inspected. A +positive, uncrossed quote must be at most ten seconds old against the broker +clock; notices 10089/10167, delayed tick types and delayed data modes refuse. +Admission requires `ask × qty × 1.05 + USD 10 headroom ≤ USD 1000 cap`. The receipt +records bid, ask, age, timestamps, headroom and derived offset. The child checks +the age again and recomputes the cap and offset before constructing the node. + +The resting offset is `floor(0.5 * admitted_bid / 0.01)`, following the frozen +trial's half-bid placement. A synthetic USD 770.01 bid gives 38500 ticks; the +native price is its current bid minus USD 385. This keeps the probe far below +market during the bounded trial. An unexpected resting fill remains a failed +case. MARKET entry and close retain upstream behavior; quote headroom is +admission evidence, and observed fill-bound breaches are failures. + +Source currency correction, verified through GitHub on 2026-10-05: +[fix ed6fc8bf](https://github.com/nautechsystems/nautilus_trader/commit/ed6fc8bf47fd37dda97d63b9b2df160719ee2bac) +is an ancestor of [v2.0.0rc6](https://github.com/nautechsystems/nautilus_trader/releases/tag/v2.0.0rc6), +published at 03:03:18 UTC. "Fixed on develop, in no release" was accurate in the +retained October 1 metadata and is superseded by that release. Pin migration +and a discriminating native over-limit test are the engine-cap re-run trigger. + +The official-ibapi checker, broker `liquidHours` parser and redaction are reused +directly from [the frozen harness](../ibkr-paper-orders/run.py). Only its pure +helpers and official read-only checker are loaded. Its 1.231.0 strategy is never +constructed. The receipt binds that dependency with `source_hashes` in addition +to the new runner and plan hashes. + +Round r3 keeps the flat pre-check and quote admission in **one client-92 +session**. A small session proxy postpones the frozen check's final disconnect +until the quote is admitted, then closes and joins the message-loop and EReader +threads within the unchanged 30-second check deadline. The official ibapi +10.45.1 `client.py` source supplied with the installed package justifies this: +`connect()` creates EReader and starts the API, `run()` drains its message queue, +and `disconnect()` closes/resets the transport without joining either reader. +Keeping that session removes the immediate same-id reconnect boundary. + +Quote diagnostics retain the last milestone (`connected`, `nextValidId`, +`accounts`, `tick requested`, `quote`, `clock`), elapsed and per-stage timings, +and reqIds/codes with redacted text for errors and information. A deadline +exception retains the same diagnostics. Only quote-request 9202 errors, +connectivity failures and delayed-data codes refuse quote admission; unrelated +farm notices remain information. The original trial's exact timeout cause is +unproven, so the next coordinator run can distinguish the remaining callbacks. + +Admission also checks `(ask - bid) × qty + 2 × (commission_allowance_per_order_usd + USD 0.02)` +against the existing `max_roundtrip_loss_usd`. A bid of 770 and ask of 990 +therefore refuses with `refused_quote_loss_bound`: the spread loss is 220 USD +and the plan's USD 1.00 per-order allowance plus the USD 0.02 margin adds +USD 2.04 for round-trip commissions against the USD 5 loss bound. +The quote receipt retains that stage and all the compared numbers. No node starts +after this refusal. The plan numbers are unchanged. + +## Environment and commands + +Use the prepared Python 3.12 environment. To reproduce its three pins with uv: + +```sh +orders_env="${XDG_CACHE_HOME:-$HOME/.cache}/ibkr-rc5-orders/venv" +rtk uv venv --python 3.12 "$orders_env" +rtk uv pip install --python "$orders_env/bin/python" \ + 'nautilus_trader==2.0.0rc5' \ + 'nautilus-ibapi==10.45.1' 'protobuf==5.29.6' +``` + +The official TWS API Python package is distributed as `nautilus-ibapi`; the +runner checks `ibapi.__version__`, not a guessed distribution named `ibapi`. +The recipe uses the base rc5 wheel; its installed metadata and +[tagged package configuration](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/python/pyproject.toml) +list `visualization` as the extra, and the native IB adapter ships with the wheel. +The engine must be exactly `2.0.0rc5` and ibapi exactly `10.45.1`, otherwise it +returns `refused_unpinned_runtime`. Protobuf and Python versions are recorded. + +From the repository root, inspect admission without connecting: + +```sh +rtk "$orders_env/bin/python" \ + blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/run.py --help +rtk "$orders_env/bin/python" \ + blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/run.py --plan-only +``` + +The second command returns exit 0 inside the window with pinned dependencies, +and records only structural validation. It refuses outside regular hours and when +the planned order/close window reaches the liquid session end minus 25 minutes: +run starts from 15:29 ET on a normal day refuse. `--plan-only` uses the plan's +session end because it does not fetch broker liquid hours; an order run rechecks +the broker's liquid session end, including early closes, before requesting a quote. +Neither command connects or fetches a quote. The coordinator's order recipe is: + +```sh +orders_run_id="rc5-paper-orders-unique-run-id" +orders_output="${XDG_STATE_HOME:-$HOME/.local/state}/native-agent-stack/ibkr-paper/runs/$orders_run_id" +umask 077 +rtk install -d -m 700 "$orders_output" +# TWS_ACCOUNT must already contain the intended DU paper account, as in upstream. +rtk "$orders_env/bin/python" \ + blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/run.py \ + --port 4002 --receipt "$orders_env/../receipt-rc5-orders.json" \ + --run-id "$orders_run_id" --private-output-dir "$orders_output" \ + > "$orders_output/console.log" +rtk chmod 600 "$orders_output/console.log" +``` + +That recipe performs the flat pre-check and quote admission before starting the +node. `7497` is the only alternative port. The parent reads upstream's +`TWS_ACCOUNT` environment variable to identify the account lease before any +broker request. It is never accepted as an argument or copied into the child +environment. Official client 92 must then discover exactly that one DU account +with zero nonzero positions +and zero open orders. Its account identifier stays in memory and travels to the +node process only over an anonymous pipe, then into the native execution config. + +Before the client-92 pre-check, the parent acquires a non-blocking exclusive +`fcntl.flock` on +`${XDG_STATE_HOME:-$HOME/.local/state}/native-agent-stack/ibkr-paper/locks/.lock`. +The directory is 0700 and the file 0600. This follows +`account_lock_fingerprint` in the frozen Alpaca safety module at +`dca821cca85dce3647fa7b488d5a23fbe5b85d4a`; [safety.py](safety.py) contains the +bounded IBKR adaptation. A held lease returns `not_started`, exit 3, before any +client connects. The lease lasts through the independent flat proof; the child +inherits its file descriptor so an orphaned child retains the lock until exit. +The receipt retains only `account_lease: {"acquired": true|false}`. The lock name +and its account-derived hash stay on the host and never enter the receipt or +journal. The lease excludes only cooperating processes that use the same lock +path and the same `XDG_STATE_HOME` (or the same HOME-derived default). The frozen +1.231 harness does not take this lease. It cannot exclude nonparticipating +writers or a writer on another host, including the WSL distro hosting the Gateway. + +Each order run creates `.jsonl` in its private output directory outside +the repository, beside the private, unredacted `console.log`. The default run id is a fresh +UUID; the default directory is `private-` beside the receipt. Supply the +same run id and directory for a retry: an existing journal refuses before +admission. A pre-admission refusal preserves any existing receipt byte for byte +and reports its new reason only to the console. If the receipt is absent, the +runner publishes the refusal with atomic create-if-absent semantics. Creation +of the journal uses `O_EXCL`; appends use `O_APPEND`, mode 0600, flush and +`os.fsync` after every record. The parent records `run_start` with plan/harness +hashes, then the child records owned native order events, client/venue identity +mappings and the stop request. The parent resumes appending after child exit to +record the independent flat proof. No account is written. The sanitized receipt +contains only the journal's SHA-256, record count and record kinds. + +## Recorded residuals + +The journal preserves `OrderInitialized` before `OrderSubmitted` in native +event-history order, including pre-submission denials. Its capture is through +the public live cache. **The pre-venue fsync barrier remains unimplemented:** +polling cannot guarantee that intent reaches disk before the native adapter +sends it. The installed rc5 Python APIs and its release notes do not expose the +native persistence factory; upstream's Rust +[BusTap](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/common/src/msgbus/mod.rs#L222) +and [with_event_store](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/live/src/node/builder.rs#L390) +provide the relevant pre-dispatch seam. `add_stream_processor` handles external +ingress, and Python `subscribe_topic` handles Python messages; neither observes +these native outbound events. Satisfying that part of the paper-lane policy +needs a native persistence integration beyond the unchanged Python node/configuration. + +## Frozen bounds and deviations + +[plan.json](plan.json) retains the frozen trial's numeric limits: SPY STK, +SMART/ARCA, USD, one share per order, at most six orders, at most USD 1000 per +order and at most USD 5 round-trip loss. The node uses client 91; the independent +official-ibapi checks use 92. The endpoint is loopback on 4002 or 7497, exactly +one managed account must begin with DU, and the account starts flat. + +The whole 420-second run must fit inside 09:30–15:50 America/New_York on a +weekday. The ten-minute close buffer is included in that window. Once the +read-only pre-check has established account scope, today's SPY `liquidHours` +must also cover the remaining run; closed, missing, malformed or shortened +broker hours refuse the order phase. Raw hours and account values are not saved. + +Changes from the upstream example are bounded configuration and observation: + +| Setting | Deviation and reason | +| --- | --- | +| Instrument and endpoint | AAPL becomes SPY, using upstream RAW `SPY=STK.SMART`; 7497/101 become 4002/91, with 7497 allowed. | +| Account discovery | Replace `TWS_ACCOUNT` with the official pre-check's in-memory account. | +| Risk engine | Set `bypass=False` and `max_notional_per_order={"SPY=STK.SMART": "1000"}`. State the route's cap limitation and hold admission through quantity one, a fresh ask stressed upward by 5%, and USD 10 headroom. | +| Submit rate | `6/00:07:00` limits native dispatch over the whole run instead of the example's default rate. | +| Quotes | REALTIME and `batch_quotes=False` replace DELAYED. Delayed data cannot establish bounded current-price admission. | +| Sell quotes and other orders | Disable limit sells, stops and brackets. Entry quantity and limit quantity remain one; entry stays MARKET IOC on the first quote. | +| Resting order | Derive `tob_offset_ticks=floor(0.5 * admitted_bid / 0.01)`; pass and validate it through the pipe. Set `use_post_only=False` because rc5's IB adapter denies every post-only submit. | +| Resting lifetime | Use GTD with seven-minute expiration and explicitly disable modify/cancel-replace. Upstream `limit_order_is_one_shot` treats an expiration as one attempt, preventing repeated rejected or filled limit buys. | +| Stop | Preserve native cancel/close-on-stop and `reduce_only_on_stop=False`; enable `use_individual_cancels_on_stop=True`. Nonterminal MARKET and IOC/FOK orders remain in flight through acceptance until terminal; other orders defer stop when SUBMITTED or lacking a venue id. The deadline bounds the wait. Native close remains MARKET GTC. | +| Node timeouts | Use `_common.py`'s builder methods with the frozen 60-second connection budget, 5-second reconciliation/portfolio/disconnection timeouts and a 45-second post-stop grace for callbacks. | +| Runtime control | Host the node with upstream `run_async()`, capture cache/handle first and handle SIGINT, SIGTERM and SIGHUP. A parent process reserves the final 60 seconds for independent observation and requests graceful stop before enforcing the hard deadline. | +| Logging | Set native stdout/file log levels OFF and `print_config=False`; send child stdout/stderr to DEVNULL and remove RUST_LOG, NAUTILUS_LOG and TWS_ACCOUNT from its environment. Read receipt evidence from the shared cache. | + +Admission also requires spread times quantity plus the frozen commission +allowance to fit the USD 5 loss bound. The runner adds an explicit USD 0.02 +margin to the plan's USD 1.00 allowance for each order, covering run 3's USD 1.02 +sell commission. Expected round-trip commissions are therefore USD 2.04, and +an admitted one-share spread can be at most USD 2.96. The plan values are unchanged. + +The half-bid placement reuses the frozen harness's distant resting probe. +IB precautionary settings are configurable; this build has not observed the +Gateway's actual settings or a current quote, so it cannot certify that this +price lies inside this account's configured limits. IB documents these controls +in [Define Precautionary Settings](https://www.interactivebrokers.co.uk/en/software/tws.bak/usersguidebook/configuretws/define_precautionary_settings.htm). +No precautionary-limit override is sent. A venue rejection cannot pass C1. + +The built-in ExecTester opens its market position before maintaining resting +limits. C1–C4 label case obligations; their labels do not impose the frozen +custom strategy's sequential chronology. The wrapper watches until C1 acceptance +and C3's full fill are present, or a failure, signal, case timeout or deadline +requests stop. The native strategy then supplies C2 and C4. Native cache events +with `reconciliation=True` cannot establish venue acceptance or cancellation. +Case outcomes, fills, the round trip and stop decisions use only this run's +ExecTester intents. An in-memory client-order-id registry requires matching +non-reconciled `OrderInitialized` events for `EXEC_TESTER-001` and `TESTER-001` +after node launch. The native factory's client-order-id date/time, trader tag, +strategy tag and counter establish that an initialized intent belongs to this +run before it reaches `OrderSubmitted`. Its native generated identity follows +[client_order_id.rs](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/common/src/generators/client_order_id.rs#L51). +Pending intents defer stop; a denied intent stays owned, counts toward the order +budget and records its denial in the case. Terminal denials release deferral. +Strategy identity and initialization time alone +are insufficient: startup reconciliation can claim external SPY orders for the +same strategy. The submission lineage follows upstream's +[IB submit path](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/adapters/interactive_brokers/src/execution/core_orders.rs#L142) +and distinguishes the +[external-order materialization path](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/execution/src/engine/mod.rs#L1101). +Source correction from r6: external materialization sets `reconciliation=true` +in that engine and the +[live execution manager](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/live/src/execution/manager.rs#L4379). +The subsequent +[MarketOrder conversion](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/model/src/orders/market.rs#L533) +calls the constructor, which creates another initialization event with +`reconciliation=false`. That conversion explains the cache receipt's false +external-init flag; the original external materialization did not set it false. +The installed rc5 reconstruction regression checks the constructor behavior, +and old generated identities without a local submission remain external. +All other cache orders appear under `reconciled_external` with stable `X` labels, +types, sides, quantities, fill prices and event timestamps. They do not contribute +to cases, bounds observations or deferred stop. A reconciliation event on an +already owned order still fails that order's cases. Startup reconciliation and +`external_order_instrument_ids=[SPY]` remain enabled as in the upstream example. +An observer exception also waits for in-flight entry resolution before native +stop, bounded by the deadline, then awaits the running task; +snapshot, receipt, signal removal and disposal each have independent cleanup. +The handle receives stop only once. Parent deadline enforcement and independent +flat proof remain in place if graceful shutdown cannot complete. + +The captured cache is a **live shared view**, verified in rc5's original +[node binding](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/live/src/python/node.rs#L911) +and [cache binding](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/common/src/python/cache.rs#L79). +`py_cache` passes the kernel's `Rc>` into `PyCache::from_rc`; +each `orders()` call borrows that same cache and returns cloned order objects. +The runner re-reads those objects on every poll. + +PR #754 submission-time notional disposition: rc5 does not enforce the engine +cap on the SMART/IB route affected by #4946. The fix ships in v2.0.0rc6; +qualifying rc6 moves the bound to the engine. This selected rc5 trial holds its +bound through quantity one and quote admission without wrapping or modifying +upstream ExecTester's submission. Admission and the child recheck require +`ask × qty × 1.05 + USD 10 ≤ USD 1000`. At quantity one and a USD 0.01 tick, +USD 942.85 is the highest admitted ask; USD 942.86 refuses. The 0.05 constant +names the National Market System Plan to Address Extraordinary Market Volatility +(Limit Up-Limit Down Plan) Tier 1 core-session band. The guard leaves the existing +USD 10 headroom intact after a 5% rise from any admitted ask. + +The roughly 60-second startup budget covers entry. C4's close-on-stop MARKET +SELL can begin as late as `node_stop_at`, 300 seconds after the parent starts: +420 seconds overall minus 60 for independent proof, 45 for cleanup, and 15 for +stop allowance. Its fill can arrive during shutdown. The admission guard applies +one 5% price stress; LULD does not freeze the price for this entire interval. +Its bands use a rolling reference price and can move; Tier 1 bands above USD 3 +are 5%, doubling to 10% during the last 25 minutes. Since February 2020, the +opening 15 minutes use single-width bands, correcting the earlier opening-window +claim. Trades cannot execute outside active bands and a persistent limit state +triggers a pause. See the [LULD Plan overview](https://www.luldplan.com/) and +its [2021 Annual Report](https://cdn.luldplan.com/reports/LULD-2021-Annual-Report.pdf). +The runner's preliminary plan-session check can refuse before any broker request. +Once the flat pre-check supplies broker `liquidHours`, admission rechecks the +same session source used by the frozen window's `session_window` helper, before +requesting a quote or constructing the node. The closing-period cutoff is the +liquid session end minus 25 minutes, when the Tier 1 band doubles to 10%. +The window extends from the parent start through `node_stop_at` plus the close +horizon: 300 seconds plus 45 seconds of cleanup and 15 seconds of stop allowance, +or 360 seconds total. With a normal 16:00 ET liquid end, a start at 15:28:59 ET +ends at 15:34:59 and fits this guard; a start at 15:29:00 ET or later refuses +with `refused_luld_closing_period`. With a 13:00 ET early close, the closing +period starts at 12:35 ET and run starts from 12:29 ET refuse. +The receipt's separate `closing_period_admission` records the refusal stage, +reason, session source, liquid end, window end and 10% closing-band constant. +`quote_admission` stays null on the preliminary refusal, or records `not_run` +after the flat pre-check when the liquid-session cutoff prevents a quote request. +This earlier admission cutoff preserves all plan numbers and prevents +the order/close window from reaching double-width bands. +The admission stress is not a guaranteed submission-time cap over 300 seconds; +rc6 qualification moves that bound to the engine. The independent client-92 +flat proof still runs at the end. + +## Receipts and remaining acceptance + +The receipt kind is `ibkr_paper_orders_nautilus_2_0_0rc5`, with evidence class +`native_paper`. A refusal records unperformed checks as null and cases as +`not_run`; it never invents zero counts or fills. Every refusal maps to its +blocked acceptance step and cases. A provisional `cleanup_required` receipt is +atomically written before starting the node and refreshed as observations arrive. +A killed child leaves that provisional state for the parent and reviewer. +Pre-node child refusals write a `refused_child_*` reason before exiting; the +parent retains that cause after successful independent flat proof. +The parent generates an account-independent UUID nonce, passes it through the +stdin payload, and requires that nonce on the child's receipt before adopting +evidence into its existing object. A different run's receipt at the same path +leaves this run's provisional evidence intact and yields `incomplete` with +`child_receipt_nonce_mismatch`; independent flat proof still runs. The nonce is +public correlation evidence, not an account identifier or a secret. Concurrent runs on different accounts must use +separate `--receipt` paths. With a shared path, both runs fail safe, but each can overwrite the other's evidence. +An unreadable or malformed child receipt, including non-dict JSON, also preserves +provisional evidence and records `child_receipt_unreadable`. Independent flat +proof still runs: a passing proof yields `incomplete`; an unproven flat state +retains `cleanup_required`. +The parent marks the child launch before `Popen`. A later storage failure cannot revert to +`not_started`: unavailable flat proof leaves `cleanup_required`, while successful +flat proof plus a final write error yields `incomplete`. An already observed +bound or case failure remains `failed`. +The shared client-92 session freezes `pre_check.observed` with a deep copy before +quote admission. A pre-check timeout or exception records `pre_check` as +`incomplete` with its phase-specific cause and leaves quote admission `not_run`. + +The reader records order aliases, types, sides, limit prices, event types, +Nautilus event/init timestamps, fills, quantities, commissions and currencies. +It does not serialize native account, client-order, venue-order, trade or event +identifiers. It computes gross, commissions and net in Decimal, reports missing +or zero commissions as unresolved, and cannot pass with a loss-bound breach, +an unexpected resting fill, an incomplete round trip or missing evidence. +Observed quantity, instrument, order-budget and notional violations are also +failures. Post-submit observations do not replace a missing pre-submit guard. +Cancellation and close-phase ordering uses native `ts_init`, since IB execution +event timestamps can have coarser resolution; both timestamps are retained. + +After the node process exits, official client 92 independently checks the same +account again. Zero positions and open orders, complete callbacks and unchanged +account scope are required. Failure or an unavailable proof returns +`cleanup_required`, exit 3. The independent proof runs after graceful shutdown +or forced termination, including an incomplete node phase. It places no orders. + +Receipt text uses the frozen account-id pattern `\b(?:D?[UF]|I)\d{5,}\b`, IPv4, +absolute-path and account-amount redaction, with a final pass over serialized +text for the in-memory account and id-shaped text. Structured execution prices +and fees remain available for review. Node child output is discarded. **Console +output is unredacted** if the parent official clients emit diagnostics; the +console is not the sanitized receipt. The parent's JSON summary receives the +final id-shaped-text scrub. + +Exit codes: **0** passed, **1** failed/incomplete, **2** not connected, **3** refused +or cleanup required. A passing C1–C4 subset would still leave the broader +[acceptance procedure](../acceptance-plan.md#5-ibkr-paper-procedure) open: + +- Step 3 reconnect with an open order: unexercised; + [#5057](https://github.com/nautechsystems/nautilus_trader/issues/5057) and + [#5060](https://github.com/nautechsystems/nautilus_trader/issues/5060) are open. +- Step 3 restart reconciliation: unexercised; + [#5007](https://github.com/nautechsystems/nautilus_trader/issues/5007), #5057 and + #5060 are open. +- Step 4 kill switch and alert exercise: unexercised; #5060 remains an open + reconciliation issue. Handling a signal is not acceptance of a kill switch. + +These issue states were read from upstream on 2026-10-05. Step 2 reports runner +admission separately from the engine's #4946 cap limitation. No offline result +from this builder advances broker readiness or a strategy gate. + +## Offline checks and completeness review + +Use a scratch directory outside the worktree and outside `/tmp`, with nice 19: + +```sh +orders_scratch="${XDG_CACHE_HOME:-$HOME/.cache}/rc5-orders-r5/tmp" +rtk mkdir -p "$orders_scratch" +rtk env TMPDIR="$orders_scratch" PYTHONDONTWRITEBYTECODE=1 nice -n 19 \ + "$orders_env/bin/python" -m unittest tests.test_ibkr_paper_orders_rc5 +rtk env TMPDIR="$orders_scratch" PYTHONDONTWRITEBYTECODE=1 nice -n 19 \ + python3 -m unittest tests.test_ibkr_paper_orders_rc5 +rtk env TMPDIR="$orders_scratch" PYTHONDONTWRITEBYTECODE=1 nice -n 19 python3 scripts/validate.py +rtk env TMPDIR="$orders_scratch" PYTHONDONTWRITEBYTECODE=1 nice -n 19 python3 scripts/evidence_manifest.py --check +rtk env TMPDIR="$orders_scratch" nice -n 19 git diff --check +``` + +The tests cover plan and endpoint violations, windows, pins, redaction, receipts, +synthetic C1–C4 mapping, quote age/delay/headroom refusals, offset validation, +deferred and single stop, observer cleanup failures, provisional receipt timing, +flat-proof causes and final breach statuses. Round r3 also tests shared-session +admission, missing-callback and deadline diagnostics, farm notice routing, +accepted entry deferral and persisted child refusals. Rounds r5-r7 (88 tests in all) add these tests: +- wide-spread and LULD-stress admission boundaries (942.85 admits, 942.86 refuses); +- pending and denied own orders, and generated-identity exclusions, including an isolated started-second case; +- lease contention, and the lease descriptor passed to the child; +- journal durability, idempotency and scrubbing; +- receipts preserved on refusal, and post-launch storage failures; +- the closing-period cutoff boundary; +- the receipt nonce. + +Round r8 adds normal and early-close liquid-session cutoffs, unrequested quotes +left null or `not_run`, and a non-dict child receipt that still triggers flat proof +(91 tests in all). + +Installed checks exercise the real +config/model constructors, post-only and individual-cancel flags and logger +config and the real builder/cache handle without starting the node. They skip +without rc5. All network readers and node execution are +synthetic; no test connects. These are local integration checks and fixtures, +not unchanged upstream tests or native paper evidence. + +Completeness review: the cross-family review's post-only denial, late-fill stop +race, cancellation targeting, cleanup failures and status defects are addressed. +Original PyO3 source confirms the cache view; the installed official API supplies +timestamped admission. The upstream release check corrected stale #4946 release +metadata. Actual commissions, precautionary settings, entitlements, concurrent +account use, reconnect, restart, alerts, kill switch and paper execution remain +unobserved here. The coordinator's paper trial supplies native case evidence. + +API correction retained from the offline probes: the Python instrument +provider's `determine_venue` accepts a contract dictionary, not an official +ibapi `Contract` instance. A proposed `symbol_to_mic_venue={"SPY": "IB"}` does +not remap this stock's ARCA primary exchange in rc5; the dictionary probe and +[stock-venue resolver](https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/adapters/interactive_brokers/src/providers/instruments.rs#L347) +confirmed it. That proposed workaround is not used. + +Cache probe correction: the installed public `Cache` view does not expose +`add_order`; that method belongs to the separate mutable Rust binding. Shared +observation is established by `PyCache::from_rc` and per-call borrows in the +original binding, rather than by an invented Python mutation method. + +## Paper runs, 2026-10-05 (NativeStack, against the paper Gateway hosted on NativeStack2604, 127.0.0.1:4002) + +The run used one plan, `plan.json` (`24ffca56dfa7…`), the rc5 environment from this README (nautilus_trader +2.0.0rc5, official ibapi 10.45.1, protobuf 5.29.6), and node client 91 with checker client 92. Every attempt is +retained. Each receipt's `harness_sha256` resolves to an archived harness, `evidence/harness/run.py.<12 hex>` (the r2, r3 and r4 bytes). Run times are the +receipts' `started_at` and `ended_at`, truncated to the second. + +- **Run 1.** 17:40:15-17:40:45Z, harness `188e0078fc8f…` (r2), `evidence/receipt-20261005-run1-not-connected.json`. + - Result: `not_connected`, exit 2. + - The flat pre-check on client 92 passed. Quote admission then opened a second client-92 connection and reached + `quote_check_deadline` at its 30-second limit. The node never started and no order was sent. r3 now runs the + pre-check and quote admission in one client-92 session and records per-stage diagnostics. +- **Run 2.** 18:22:52-18:23:42Z, harness `67429251370f…` (r3), `evidence/receipt-20261005-run2-failed.json`. + - Result: `failed`, exit 1. + - Quote admission passed in the shared session. At node start, rc5's startup reconciliation imported the account's + 14:26Z fills from the separate 1.231 harness trial as `reconciliation=true` orders. Upstream's configuration + claims external SPY orders, so the case mapping attributed those fills to C3 and C4, and the node stopped before + ExecTester sent any order. The flat proof passed. r4 now maps cases only from orders this run submitted and + records other orders under `reconciled_external`. +- **Run 3.** 18:55:11-18:56:01Z (14:55 ET), harness `5a061ff06f41…` (r4, archived as `evidence/harness/run.py.5a061ff06f41`), `evidence/receipt-20261005-passed.json`. + - Result: **passed**, exit 0. + - Quote admission: bid 775.24, ask 775.26, fresh. + + | Case | Result | + |---|---| + | C1 resting LIMIT BUY 1 SPY at half the bid (387.62) | accepted | + | C2 cancel | canceled | + | C3 MARKET BUY 1 | filled 775.26, commission 1.00 USD | + | C4 MARKET SELL 1 (close on stop) | filled 775.24, commission 1.02 USD | + + - Gross −0.02 USD and net −2.04 USD, against the 5 USD round-trip bound. + - The independent client-92 flat proof found 0 positions and 0 open orders. + - Two reconciled external orders (six events: initialization, acceptance and fill for each) are recorded separately. + - The receipt's risk note reads: "engine notional cap configured, not enforced on this route (#4946, fixed on + develop, unreleased); bound held by qty=1 and quote admission". v2.0.0rc6 ships that fix. + +The Nautilus console is not redacted, so it stayed private. Acceptance steps 3-4 (reconnect with an open order, +restart reconciliation, the kill switch) were not exercised and remain blocked upstream (#5007, #5057, #5060). + +**The current runner has not run natively.** Run 3 exercised the r4 harness. The r7 `run.py` (`72147d8627d7…`) added these r5-r7 changes, retained by r8: +- the account lease and the journal; +- the 5% LULD stress and the commission margin; +- the closing cutoff, now derived from the liquid session end (15:29 ET run starts on a normal day); +- the receipt nonce; +- ownership by generated identity. + +Those r5-r7 changes were covered by 88 offline tests; r8 adds the liquid-session +and malformed-receipt regressions for 91 offline tests. No native paper run of the +current runner exists yet. diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.188e0078fc8f b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.188e0078fc8f new file mode 100644 index 000000000..040060dfe --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.188e0078fc8f @@ -0,0 +1,997 @@ +#!/usr/bin/env python3 +"""Bounded rc5 paper trial using upstream's built-in ExecTester. + +The rc5 engine cap is configured; quote admission and quantity one hold this +trial's runner bound on the SMART/IB route affected by #4946. --plan-only and +--help never connect. Synthetic tests do not establish broker acceptance. +""" +from __future__ import annotations + +import argparse +import asyncio +import hashlib +import importlib.metadata +import importlib.util +import json +import os +import platform +import signal +import subprocess +import sys +import tempfile +import threading +import time +from datetime import datetime, timezone +from decimal import Decimal, InvalidOperation, ROUND_FLOOR +from functools import lru_cache +from pathlib import Path + +HERE = Path(__file__).resolve().parent +PLAN_PATH = HERE / "plan.json" +FROZEN_RUN = HERE.parent / "ibkr-paper-orders" / "run.py" +KIND = "ibkr_paper_orders_nautilus_2_0_0rc5" +UPSTREAM_COMMIT = "1b0a49d2792a9432a3aca3fcb617ce7a630d905e" +SOURCE = f"https://github.com/nautechsystems/nautilus_trader/blob/{UPSTREAM_COMMIT}" +CASE_IDS = ("C1", "C2", "C3", "C4") +SIGNALS = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP) +TERMINAL_FAILURES = {"OrderDenied", "OrderRejected", "OrderExpired", "OrderCancelRejected"} +TESTER_SETTINGS = { + "order_qty": "1", "open_position_on_start_qty": "1", "open_position_on_first_quote": True, + "open_position_time_in_force": "IOC", "enable_limit_buys": True, "enable_limit_sells": False, + "enable_stop_buys": False, "enable_stop_sells": False, "enable_brackets": False, + "tob_offset_ticks": "floor(0.5 * admitted_bid / 0.01)", "limit_time_in_force": "GTD", "order_expire_time_delta_mins": 7, + "modify_orders_to_maintain_tob_offset": False, "cancel_replace_orders_to_maintain_tob_offset": False, + "use_post_only": False, "use_individual_cancels_on_stop": True, + "cancel_orders_on_stop": True, "close_positions_on_stop": True, + "reduce_only_on_stop": False, "dry_run": False, "log_data": False, +} +CASE_DEFINITIONS = [ + {"id": "C1", "name": "accept_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderAccepted"}, + {"id": "C2", "name": "cancel_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderCanceled for C1 after stop"}, + {"id": "C3", "name": "fill_buy", "order_type": "MARKET", "side": "BUY", "expect": "OrderFilled quantity 1"}, + {"id": "C4", "name": "flatten", "order_type": "MARKET", "side": "SELL", "expect": "OrderFilled quantity 1 after stop and independent flat proof"}, +] + +# Original sources, read at v2.0.0rc5 (UPSTREAM_COMMIT): +# examples/live/interactive_brokers/exec_tester.py: builder + ExecTester config. +# examples/live/interactive_brokers/_common.py: bounded node timeouts. +# crates/testkit/src/testers/exec/strategy.rs: native entry/cancel/close paths. +# crates/risk/src/engine/mod.rs: check_orders_risk_for_account returns true +# when account_for_venue(SMART) is absent, before checking max_notional. The +# IB account is registered under IB. Config support is not enforcement. +RISK_NOTE = "engine notional cap configured, not enforced on this route (#4946, fixed on develop, unreleased); bound held by qty=1 and quote admission" +RISK_ROUTE = { + "issue": "https://github.com/nautechsystems/nautilus_trader/issues/4946", + "issue_state": "closed", + "pin_fix_present": False, + "upstream_fix": "fixed on develop; included in v2.0.0rc6; absent from the selected rc5 pin", + "source": SOURCE + "/crates/risk/src/engine/mod.rs#L1218", + "reason": "rc5 stock orders can skip account-scoped max_notional_per_order on the SMART/IB venue path", +} +RELEASE_VERIFICATION = { + "checked_at": "2026-10-05", + "fix_commit": "ed6fc8bf47fd37dda97d63b9b2df160719ee2bac", + "released_in": "v2.0.0rc6", + "published_at": "2026-10-05T03:03:18Z", + "source": "https://github.com/nautechsystems/nautilus_trader/releases/tag/v2.0.0rc6", + "correction": "The requested note's unreleased clause reflects earlier metadata; the fix is now released in rc6, while this trial stays pinned to rc5.", +} +BLOCKED_STEPS = [ + {"acceptance_step": 3, "operation": "reconnect_with_open_order", "exercised": False, + "issues": [{"number": 5057, "state": "open"}, {"number": 5060, "state": "open"}]}, + {"acceptance_step": 3, "operation": "restart_reconciliation", "exercised": False, + "issues": [{"number": 5007, "state": "open"}, {"number": 5057, "state": "open"}, + {"number": 5060, "state": "open"}]}, + {"acceptance_step": 4, "operation": "kill_switch", "exercised": False, + "issues": [{"number": 5060, "state": "open"}]}, +] +for _step in BLOCKED_STEPS: + _step["issue_states_checked_at"] = "2026-10-05" + for _issue in _step["issues"]: + _issue["url"] = f"https://github.com/nautechsystems/nautilus_trader/issues/{_issue['number']}" + + +@lru_cache(maxsize=1) +def frozen(): + """Reuse the frozen official-ibapi checker, window parser and redaction. + + Source: ../ibkr-paper-orders/run.py, bound by source_hashes in the receipt. + No frozen strategy is used; the node uses rc5's built-in ExecTester. + """ + spec = importlib.util.spec_from_file_location("ibkr_frozen_readonly", FROZEN_RUN) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def sha256(path): + return hashlib.sha256(Path(path).read_bytes()).hexdigest() + + +def decimal(value): + result = Decimal(str(value)) + if not result.is_finite(): + raise ValueError("nonfinite numeric input") + return result + + +def load_plan(path=PLAN_PATH): + def unique(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError("duplicate plan key") + result[key] = value + return result + return json.loads(Path(path).read_text(), object_pairs_hook=unique) + + +def validate_plan(plan, *, port=None, node_client_id=None, check_client_id=None): + """All static bounds are checked without importing a client or connecting.""" + errors = [] + + def need(condition, reason): + if not condition: + errors.append(reason) + + try: + need(type(plan["schema_version"]) is int and plan["schema_version"] == 1 + and plan["kind"] == "ibkr_paper_orders_plan_rc5", "plan schema") + need(plan["engine"]["version"] == "2.0.0rc5" and plan["engine"]["ibapi_version"] == "10.45.1", "runtime pins") + need(plan["engine"]["upstream_commit"] == UPSTREAM_COMMIT, "upstream pin") + need(plan["host"] == "127.0.0.1", "loopback host required") + ports = plan["paper_ports"] + need(isinstance(ports, list) and bool(ports) and all(type(p) is int and p in (4002, 7497) for p in ports), "paper ports") + need(plan["default_port"] in ports and type(plan["default_port"]) is int, "default paper port") + need(port is None or type(port) is int and port in ports and port in (4002, 7497), "selected paper port") + need(plan["account_prefix"] == "DU" and plan["require_single_managed_account"] is True, "single DU account") + need(plan["client_ids"] == {"node": 91, "check": 92} + and all(type(v) is int for v in plan["client_ids"].values()), "client ids 91/92") + need(node_client_id is None or type(node_client_id) is int and node_client_id == 91, "node client id 91") + need(check_client_id is None or type(check_client_id) is int and check_client_id == 92, "check client id 92") + inst = plan["instrument"] + need(tuple(inst[k] for k in ("symbol", "sec_type", "exchange", "primary_exchange", "currency")) + == ("SPY", "STK", "SMART", "ARCA", "USD"), "SPY STK SMART/ARCA USD only") + need(inst["nautilus_instrument_id"] == "SPY=STK.SMART" and inst["expected_price_increment"] == "0.01", "RAW stock identity and tick") + b = plan["bounds"] + need(type(b["max_orders"]) is int and 3 <= b["max_orders"] <= 6, "order budget 3..6") + need(type(b["max_quantity_per_order"]) is int and b["max_quantity_per_order"] == 1, "quantity 1") + for name, ceiling in (("max_notional_per_order_usd", 1000), ("max_roundtrip_loss_usd", 5)): + need(not isinstance(b[name], bool) and 0 < decimal(b[name]) <= ceiling, name) + need(decimal(b["commission_allowance_per_order_usd"]) >= 0, "commission allowance") + s = plan["session"] + need((s["timezone"], s["open"], s["close"]) == ("America/New_York", "09:30", "16:00"), "regular trading hours") + need(s["weekdays_only"] is True and s["whole_run_inside_window"] is True + and s["use_contract_liquid_hours"] is True, "whole run with broker liquid hours") + need(type(s["close_buffer_minutes"]) is int and s["close_buffer_minutes"] >= 10, "10 minute close buffer") + expected_times = {"check_seconds": 30, "node_start_seconds": 60, "per_step_seconds": 45, + "cleanup_seconds": 45, "node_stop_allowance_seconds": 15, + "post_check_reserve_seconds": 60, "overall_deadline_seconds": 420} + need(plan["timeouts"] == expected_times and all(type(v) is int for v in plan["timeouts"].values()), + "frozen timeouts and hard deadline") + r = plan["risk"] + need(r["bypass"] is False and r["require_enforced_notional_before_connection"] is False, + "native risk configured with runner quote admission") + need(not isinstance(r["notional_headroom_usd"], bool) + and decimal(r["notional_headroom_usd"]) == 10, "USD 10 quote admission headroom") + need(r["max_order_submit_rate"] == f"{b['max_orders']}/00:07:00", "whole-run native submit throttle") + # No configurable strategy knobs can quietly turn this into another trial. + need(plan["exec_tester"] == TESTER_SETTINGS + and all(type(plan["exec_tester"][k]) is type(v) for k, v in TESTER_SETTINGS.items()), + "frozen ExecTester settings") + need(plan["data"] == {"market_data_type": "REALTIME", "quote_max_age_seconds": 10, + "clock_resolution_tolerance_seconds": 2, "batch_quotes": False}, "real-time quote bounds") + need(plan["cases"] == CASE_DEFINITIONS, "frozen cases C1..C4") + need(plan["end_disposition"] == {"positions": 0, "open_orders": 0}, "flat disposition") + need(plan["receipt"]["kind"] == KIND and plan["evidence_class"] == "native_paper", "receipt identity") + except (KeyError, TypeError, ValueError, InvalidOperation): + errors.append("missing or malformed plan field") + return errors + + +def runtime_versions(): + def version(package): + try: + return importlib.metadata.version(package) + except importlib.metadata.PackageNotFoundError: + return None + try: + import ibapi + ib_version = ibapi.__version__ + except (ImportError, AttributeError): + ib_version = None + try: + import nautilus_trader + engine_version = nautilus_trader.__version__ + except (ImportError, AttributeError): + engine_version = None + return {"nautilus_trader": engine_version, "nautilus_trader_distribution": version("nautilus_trader"), "ibapi": ib_version, + "nautilus_ibapi_distribution": version("nautilus-ibapi"), + "protobuf": version("protobuf"), "python": platform.python_version()} + + +def runtime_refusal(versions): + if versions.get("nautilus_trader") != "2.0.0rc5" or versions.get("ibapi") != "10.45.1": + return "refused_unpinned_runtime" + if "nautilus_trader_distribution" in versions and versions["nautilus_trader_distribution"] != "2.0.0rc5": + return "refused_unpinned_runtime" + return None + + +def resting_offset_ticks(bid, plan): + """Frozen half-bid placement through upstream ExecTester's supported offset.""" + tick = decimal(plan["instrument"]["expected_price_increment"]) + return int((decimal(bid) * Decimal("0.5") / tick).to_integral_value(rounding=ROUND_FLOOR)) + + +def admit_quote(plan, quote, server_time_epoch, *, elapsed_seconds=0, delayed=False, errors=()): + """Pure admission using IB's timestamped BidAsk and its independent clock.""" + result = {"source": "ibapi.reqTickByTickData.BidAsk/reqCurrentTime", "status": None, + "bid": None, "ask": None, "quote_age_seconds": None} + if delayed or any(code in (10089, 10167) for code in errors): + result["status"] = "refused_delayed_quote" + return result + if errors: + result["status"] = "refused_quote_error" + return result + try: + bid, ask = decimal(quote["bid"]), decimal(quote["ask"]) + age = decimal(server_time_epoch) - decimal(quote["quote_time_epoch"]) + decimal(elapsed_seconds) + result.update(bid=str(bid), ask=str(ask), quote_time_epoch=int(quote["quote_time_epoch"]), + server_time_epoch=int(server_time_epoch), quote_age_seconds=float(max(age, Decimal(0)))) + if bid <= 0 or ask < bid: + result["status"] = "refused_invalid_quote" + elif age < -decimal(plan["data"]["clock_resolution_tolerance_seconds"]) or age > decimal(plan["data"]["quote_max_age_seconds"]): + result["status"] = "refused_stale_quote" + else: + notional = ask * plan["bounds"]["max_quantity_per_order"] + headroom = decimal(plan["risk"]["notional_headroom_usd"]) + result.update(ask_notional_usd=str(notional), headroom_usd=str(headroom), + tob_offset_ticks=resting_offset_ticks(bid, plan)) + result["status"] = "passed" if notional + headroom <= decimal(plan["bounds"]["max_notional_per_order_usd"]) and result["tob_offset_ticks"] > 0 else "refused_quote_notional" + except (KeyError, TypeError, ValueError, InvalidOperation): + result["status"] = "refused_invalid_quote" + return result + + +# Official ibapi 10.45.1 client.py reqTickByTickData and wrapper.py callbacks +# were inspected in the prepared runtime. Delayed enums come from ticktype.py. +DELAYED_TICKS = set(range(66, 77)) | {80, 81, 82, 83, 88, 90, 103, 104} + + +class QuoteState: + """Read-only callback state, kept independent of ibapi for synthetic tests.""" + + def __init__(self): + self.ready = threading.Event() + self.quote_ready = threading.Event() + self.clock_ready = threading.Event() + self.accounts_ready = threading.Event() + self.accounts = [] # Memory only. + self.quote = None + self.server_time = None + self.clock_received = None + self.delayed = False + self.errors = [] + + def nextValidId(self, orderId): + self.ready.set() + + def managedAccounts(self, accountsList): + self.accounts = [value.strip() for value in (accountsList or "").split(",") if value.strip()] + self.accounts_ready.set() + + def tickByTickBidAsk(self, reqId, timestamp, bidPrice, askPrice, bidSize, askSize, tickAttribBidAsk): + if reqId == 9202: + self.quote = {"bid": str(bidPrice), "ask": str(askPrice), "quote_time_epoch": timestamp} + self.quote_ready.set() + + def currentTime(self, timestamp): + self.server_time, self.clock_received = timestamp, time.monotonic() + self.clock_ready.set() + + def error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson=""): + if errorCode in (10089, 10167): + self.delayed = True + if errorCode not in frozen().INFO_CODES or self.delayed: + self.errors.append(errorCode) + self.quote_ready.set() + self.clock_ready.set() + + def marketDataType(self, reqId, marketDataType): + if marketDataType in (3, 4): + self.delayed = True + self.quote_ready.set() + + def delayed_tick(self, tickType): + if tickType in DELAYED_TICKS: + self.delayed = True + self.quote_ready.set() + + def tickPrice(self, reqId, tickType, price, attrib): + self.delayed_tick(tickType) + + def tickSize(self, reqId, tickType, size): + self.delayed_tick(tickType) + + def tickString(self, reqId, tickType, value): + self.delayed_tick(tickType) + + def tickGeneric(self, reqId, tickType, value): + self.delayed_tick(tickType) + + def tickOptionComputation(self, reqId, tickType, *values): + self.delayed_tick(tickType) + + +def build_quote_client(): + from ibapi.client import EClient + from ibapi.wrapper import EWrapper + + class QuoteClient(QuoteState, EWrapper, EClient): + def __init__(self): + QuoteState.__init__(self) + EClient.__init__(self, self) + + return QuoteClient() + + +def quote_check(plan, port, account_id, *, deadline, client_factory=None): + """A separate client-92 read after the flat pre-check, before node creation.""" + client = (client_factory or build_quote_client)() + reader = None + requested = False + + def wait(event): + return event.wait(max(0, deadline - time.monotonic())) + + try: + client.connect(plan["host"], port, plan["client_ids"]["check"]) + if not client.isConnected(): + return {"status": "not_connected"} + reader = threading.Thread(target=client.run, daemon=True) + reader.start() + if not wait(client.ready) or not wait(client.accounts_ready): + return {"status": "not_connected"} + if client.accounts != [account_id] or not account_id.startswith("DU"): + return {"status": "refused_quote_account_scope"} + client.reqMarketDataType(1) + requested = True + client.reqTickByTickData(9202, frozen().spy_contract(), "BidAsk", 0, False) + if not wait(client.quote_ready): + return {"status": "refused_quote_unavailable"} + if client.delayed or client.errors: + return admit_quote(plan, client.quote, client.server_time, + delayed=client.delayed, errors=client.errors) + client.reqCurrentTime() + if not wait(client.clock_ready): + return {"status": "refused_quote_clock"} + elapsed = max(0, time.monotonic() - client.clock_received) if client.clock_received is not None else 0 + result = admit_quote(plan, client.quote, client.server_time, elapsed_seconds=elapsed, + delayed=client.delayed, errors=client.errors) + result["admitted_monotonic"] = time.monotonic() + return result + finally: + try: + if requested: + client.cancelTickByTickData(9202) + finally: + client.disconnect() + if reader is not None: + reader.join(timeout=1) + + +def validate_child_admission(plan, payload, *, monotonic_now=None): + """Recompute age, cap and offset from the pipe; no account in diagnostics.""" + try: + admitted = payload["quote_admission"] + now = time.monotonic() if monotonic_now is None else monotonic_now + elapsed = decimal(now) - decimal(admitted["admitted_monotonic"]) + if admitted["status"] != "passed" or elapsed < 0: + return False + # Stored age already includes time since the independent server callback. + rechecked = admit_quote(plan, admitted, decimal(admitted["quote_time_epoch"]) + decimal(admitted["quote_age_seconds"]), + elapsed_seconds=elapsed) + offset = payload["tob_offset_ticks"] + return (rechecked["status"] == "passed" and type(offset) is int + and offset == resting_offset_ticks(admitted["bid"], plan) + and offset == admitted["tob_offset_ticks"] + and isinstance(payload["account_id"], str) and payload["account_id"].startswith("DU")) + except (KeyError, TypeError, ValueError, InvalidOperation): + return False + + +def sanitize(value): + """Prices/quantities remain structured; all untrusted text uses frozen redaction.""" + if isinstance(value, dict): + return {frozen().redact(k): sanitize(v) for k, v in value.items()} + if isinstance(value, (list, tuple)): + return [sanitize(v) for v in value] + if isinstance(value, str): + # Constant source locators are public evidence, not untrusted error text. + public_sources = {RISK_ROUTE["source"], RISK_ROUTE["issue"], RELEASE_VERIFICATION["source"]} + public_sources.update(issue["url"] for step in BLOCKED_STEPS for issue in step["issues"]) + if value in public_sources: + return frozen().scrub_serialized(value) + return frozen().redact(value) + return value + + +def serialized_receipt(receipt, secret_values=()): + text = json.dumps(sanitize(receipt), indent=2, sort_keys=True, allow_nan=False) + # Final id-shaped-text pass also covers dictionary keys and unexpected nested fields. + return frozen().scrub_serialized(text, secret_values) + "\n" + + +def write_receipt(path, receipt, secret_values=()): + path = Path(path) + protected = {PLAN_PATH.resolve(), Path(__file__).resolve(), FROZEN_RUN.resolve()} + if path.resolve() in protected or frozen().is_gate_receipt(path): + raise ValueError("protected receipt destination") + raw = serialized_receipt(receipt, secret_values) + path.parent.mkdir(parents=True, exist_ok=True) + temporary = None + try: + with tempfile.NamedTemporaryFile(mode="w", dir=path.parent, prefix=".paper-receipt-", delete=False) as stream: + temporary = Path(stream.name) + stream.write(raw) + os.replace(temporary, path) + finally: + if temporary is not None: + temporary.unlink(missing_ok=True) + + +def case_outcomes(events, *, stop_ns=None, flat_proof=None): + """Pure mapping of sanitized native cache events; no strings imply acceptance. + + Each event carries an opaque local order alias and its type/side/quantity/price. + C2 must cancel C1, and C4 must fill after stop AND have an independent flat proof. + Duplicate native event ids are removed in snapshot_events, before this mapping. + """ + definitions = (("LIMIT", "BUY"), ("LIMIT", "BUY"), ("MARKET", "BUY"), ("MARKET", "SELL")) + result = {cid: {"id": cid, "order_type": typ, "side": side, "price": None, + "order": None, "events": [], "outcome": "not_run", "reason": None} + for cid, (typ, side) in zip(CASE_IDS, definitions)} + aliases = {} + for event in events: + aliases.setdefault(event["order"], event) + chosen = {} + for cid, shape in (("C1", ("LIMIT", "BUY")), ("C3", ("MARKET", "BUY")), ("C4", ("MARKET", "SELL"))): + candidates = [key for key, e in aliases.items() if (e["order_type"], e["side"]) == shape] + if candidates: + chosen[cid] = candidates[0] + if "C1" in chosen: + chosen["C2"] = chosen["C1"] + for cid, alias in chosen.items(): + case = result[cid] + seq = [e for e in events if e["order"] == alias] + case.update(order=alias, price=seq[0].get("price"), events=seq, outcome="incomplete") + confirmed = [e for e in seq if not e.get("reconciliation", False)] + bad = [e for e in seq if e["type"] in TERMINAL_FAILURES or e.get("reconciliation", False)] + if bad: + case.update(outcome="failed", reason=f"{cid}_unconfirmed_or_rejected") + continue + if decimal(seq[0]["quantity"]) != 1: + case.update(outcome="failed", reason=f"{cid}_quantity_bound") + continue + if cid == "C1": + if any(e["type"] == "OrderFilled" for e in confirmed): + case.update(outcome="failed", reason="C1_resting_order_filled") + elif any(e["type"] == "OrderAccepted" for e in confirmed): + case["outcome"] = "passed" + elif cid == "C2": + accepted = [e for e in confirmed if e["type"] == "OrderAccepted"] + canceled = [e for e in confirmed if e["type"] == "OrderCanceled"] + if stop_ns is not None and accepted and any(e["ts_init"] >= stop_ns and e["ts_init"] >= accepted[0]["ts_init"] for e in canceled): + case["outcome"] = "passed" + else: + fills = [e for e in confirmed if e["type"] == "OrderFilled"] + filled = sum((decimal(e["fill_quantity"]) for e in fills), Decimal(0)) + if filled > 1: + case.update(outcome="failed", reason=f"{cid}_overfill") + elif filled == 1: + if cid == "C3": + case["outcome"] = "passed" + elif stop_ns is not None and all(e["ts_init"] >= stop_ns for e in fills): + case["outcome"] = "filled_unproven" + if flat_proof and flat_proof.get("status") == "passed": + observed = flat_proof.get("observed", {}) + if observed.get("positions") == 0 and observed.get("open_orders") == 0: + case["outcome"] = "passed" + if case["outcome"] == "incomplete": + case["reason"] = f"{cid}_expected_event_missing" + return result + + +def roundtrip(events, loss_bound): + fills = [e for e in events if e["type"] == "OrderFilled"] + if not fills: + return None + buys = sum((decimal(e["fill_quantity"]) for e in fills if e["side"] == "BUY"), Decimal(0)) + sells = sum((decimal(e["fill_quantity"]) for e in fills if e["side"] == "SELL"), Decimal(0)) + gross = sum((decimal(e["fill_price"]) * decimal(e["fill_quantity"]) * (1 if e["side"] == "SELL" else -1) for e in fills), Decimal(0)) + unresolved = any(e.get("commission") is None or decimal(e["commission"]) <= 0 + or e.get("commission_currency") != "USD" or e.get("currency") != "USD" for e in fills) + fees = None if unresolved else sum((decimal(e["commission"]) for e in fills), Decimal(0)) + closed = buys == sells == 1 + net = gross - fees if fees is not None and closed else None + return {"buy_quantity": str(buys), "sell_quantity": str(sells), "closed": closed, + "gross_usd": str(gross) if buys == sells else None, + "commissions_usd": str(fees) if fees is not None else None, + "net_usd": str(net) if net is not None else None, + "commission_unresolved": unresolved, + "loss_bound_breach": net < -decimal(loss_bound) if net is not None else None} + + +def new_receipt(plan, plan_path, versions): + if not isinstance(plan, dict): + plan = {} + bounds = plan.get("bounds") if isinstance(plan.get("bounds"), dict) else {} + try: + cap = str(decimal(bounds["max_notional_per_order_usd"])) + except (KeyError, ValueError, InvalidOperation): + cap = None + try: + plan_hash = sha256(plan_path) + except OSError: + plan_hash = None + return {"schema_version": 1, "kind": KIND, "evidence_class": "native_paper", + "started_at": datetime.now(timezone.utc).isoformat(), "versions": versions, + "plan_sha256": plan_hash, "harness_sha256": sha256(__file__), + "source_hashes": {"frozen_checker_sha256": sha256(FROZEN_RUN)}, + "upstream_commit": UPSTREAM_COMMIT, "pre_check": None, "quote_admission": None, + "cases": case_outcomes([]), "events": [], "fills": [], "roundtrip": None, + "flat_proof": None, "node": {"started": False, "stop_ns": None}, + "risk": {"bypass": False, "max_notional_per_order_usd": cap, + "enforcement": "runner_quantity_and_quote_admission", "note": RISK_NOTE, + "engine_route": RISK_ROUTE, "submit_budget_enforcement": "engine_throttle", + "release_verification": RELEASE_VERIFICATION}, + "blocked_steps": BLOCKED_STEPS, "failures": [], + "status": "cleanup_required", "exit_code": 3} + + +def finish(receipt, status, *, reason=None, step=2, cases=CASE_IDS): + receipt["status"] = status + receipt["exit_code"] = frozen().exit_code_for(status) + receipt["ended_at"] = datetime.now(timezone.utc).isoformat() + if reason: + receipt["failures"].append({"acceptance_step": step, "cases_blocked": list(cases), "reason": reason}) + return receipt["exit_code"] + + +def native_configs(plan, port, account_id, *, tob_offset_ticks): + """Only upstream configs, with installed rc5 signatures verified offline. + + Source: v2.0.0rc5 examples/live/interactive_brokers/{exec_tester,_common}.py. + Construction does not connect. Actual account_id is supplied by the official + pre-check in memory, never from TWS_ACCOUNT or a command line argument. + """ + from nautilus_trader.adapters import interactive_brokers as ib + from nautilus_trader.config import LiveRiskEngineConfig + from nautilus_trader.model import ClientId, InstrumentId, Quantity, StrategyId, TimeInForce + from nautilus_trader.testkit import ExecTesterConfig + instrument = InstrumentId.from_str(plan["instrument"]["nautilus_instrument_id"]) + provider = ib.InteractiveBrokersInstrumentProviderConfig(symbology_method=ib.SymbologyMethod.RAW, load_ids={instrument}) + common = dict(host=plan["host"], port=port, client_id=plan["client_ids"]["node"], + connection_timeout=10, request_timeout=30, instrument_provider=provider) + tester_values = dict(plan["exec_tester"]) + tester_values["tob_offset_ticks"] = tob_offset_ticks + tester_values["order_qty"] = Quantity.from_str(tester_values["order_qty"]) + tester_values["open_position_on_start_qty"] = Decimal(tester_values["open_position_on_start_qty"]) + for key in ("open_position_time_in_force", "limit_time_in_force"): + tester_values[key] = TimeInForce.from_str(tester_values[key]) + tester = ExecTesterConfig(strategy_id=StrategyId.from_str("EXEC_TESTER-001"), instrument_id=instrument, + client_id=ClientId.from_str("IB"), external_order_instrument_ids=[instrument], + subscribe_quotes=True, subscribe_trades=True, **tester_values) + risk = LiveRiskEngineConfig(bypass=False, max_order_submit_rate=plan["risk"]["max_order_submit_rate"], + max_notional_per_order={str(instrument): str(plan["bounds"]["max_notional_per_order_usd"])}) + return (ib, ib.InteractiveBrokersDataClientConfig(market_data_type=ib.MarketDataType.REALTIME, batch_quotes=False, **common), + ib.InteractiveBrokersExecutionClientConfig(account_id=account_id, **common), risk, tester) + + +def build_node(plan, port, account_id, *, tob_offset_ticks): + from nautilus_trader.common import Environment, LoggerConfig, LogLevel + from nautilus_trader.live import LiveNode + from nautilus_trader.model import TraderId + ib, data, execution, risk, tester = native_configs(plan, port, account_id, tob_offset_ticks=tob_offset_ticks) + node = (LiveNode.builder("IB-EXEC-TESTER-001", TraderId.from_str("TESTER-001"), Environment.LIVE) + .with_reconciliation(True).with_risk_engine_config(risk) + .with_logging(LoggerConfig(stdout_level=LogLevel.OFF, fileout_level=LogLevel.OFF, print_config=False)) + .with_timeout_connection(plan["timeouts"]["node_start_seconds"]) + .with_timeout_reconciliation(5).with_timeout_portfolio(5) + .with_timeout_disconnection_secs(5).with_delay_post_stop_secs(plan["timeouts"]["cleanup_seconds"]) + .add_data_client(None, ib.InteractiveBrokersDataClientFactory(), data) + .add_exec_client(None, ib.InteractiveBrokersExecutionClientFactory(), execution).build()) + node.add_builtin_strategy("ExecTester", tester) + return node + + +def tester_orders(cache): + from nautilus_trader.model import StrategyId + return cache.orders(strategy_id=StrategyId.from_str("EXEC_TESTER-001")) + + +def snapshot_events(cache): + """Read cloned order histories, never serialize account/order/venue/trade ids.""" + orders = tester_orders(cache) + orders.sort(key=lambda order: (order.ts_init, str(order.client_order_id))) + events, seen = [], set() + for index, order in enumerate(orders, 1): + for event in order.events(): + key = str(event.event_id) + if key in seen: + continue + seen.add(key) + price = getattr(order, "price", None) + record = {"order": f"O{index}", "order_type": order.order_type.name, "side": order.side.name, + "instrument_id": str(order.instrument_id), + "quantity": str(order.quantity), "price": str(price) if price is not None else None, + "type": type(event).__name__, "ts_event": int(event.ts_event), "ts_init": int(event.ts_init), + "at": frozen().ns_to_iso(event.ts_event), "reconciliation": bool(getattr(event, "reconciliation", False))} + if record["type"] == "OrderFilled": + commission = event.commission + record.update(fill_price=str(event.last_px), fill_quantity=str(event.last_qty), currency=str(event.currency), + commission=str(commission.as_decimal()) if commission is not None else None, + commission_currency=str(commission.currency) if commission is not None else None) + if hasattr(event, "reason"): + record["reason"] = frozen().redact(event.reason) + events.append(record) + return sorted(events, key=lambda event: (event["ts_init"], event["ts_event"])) + + +def update_observations(receipt, events, plan): + receipt["events"] = events + receipt["cases"] = case_outcomes(events, stop_ns=receipt["node"]["stop_ns"], flat_proof=receipt["flat_proof"]) + receipt["fills"] = [e for e in events if e["type"] == "OrderFilled"] + receipt["roundtrip"] = roundtrip(events, plan["bounds"]["max_roundtrip_loss_usd"]) + orders = {e["order"]: e for e in events} + violations = [] + if len(orders) > plan["bounds"]["max_orders"]: + violations.append(("observed_order_budget_breach", list(CASE_IDS))) + for e in orders.values(): + cases = ["C1", "C2"] if e["order_type"] == "LIMIT" else ["C3" if e["side"] == "BUY" else "C4"] + quantity = decimal(e["quantity"]) + if quantity != 1: + violations.append(("observed_quantity_bound_breach", cases)) + if e.get("instrument_id") != plan["instrument"]["nautilus_instrument_id"]: + violations.append(("observed_instrument_bound_breach", cases)) + if e.get("price") is not None and decimal(e["price"]) * quantity > decimal(plan["bounds"]["max_notional_per_order_usd"]): + violations.append(("observed_notional_bound_breach", cases)) + for e in events: + if e["type"] == "OrderFilled" and decimal(e["fill_price"]) * decimal(e["fill_quantity"]) > decimal(plan["bounds"]["max_notional_per_order_usd"]): + violations.append(("observed_fill_notional_bound_breach", ["C3" if e["side"] == "BUY" else "C4"])) + for reason, cases in violations: + if not any(f["reason"] == reason and f["cases_blocked"] == cases for f in receipt["failures"]): + receipt["failures"].append({"reason": reason, "cases_blocked": cases, "acceptance_step": 2}) + + +def orders_in_flight(orders): + """Upstream close-on-stop runs once; wait for entry resolution before stop.""" + terminal = {"FILLED", "CANCELED", "EXPIRED", "REJECTED", "DENIED"} + return any(order.status.name not in terminal and + (order.status.name == "SUBMITTED" or order.venue_order_id is None) for order in orders) + + +def stop_ready(orders, now, node_stop_at): + return now >= node_stop_at or not orders_in_flight(orders) + + +async def node_phase(plan, port, receipt_path, payload): + """Hosted upstream run: cache + thread-safe handle captured before run_async. + + Source: crates/live/src/python/node.rs at UPSTREAM_COMMIT. The parent owns + the hard deadline; OS process termination cannot strand this awaitable. + """ + if not validate_child_admission(plan, payload): + raise ValueError("refused_child_quote_admission") + receipt, account_id = payload["receipt"], payload["account_id"] + node = build_node(plan, port, account_id, tob_offset_ticks=payload["tob_offset_ticks"]) + # Live view: node.rs py_cache uses PyCache::from_rc(kernel.cache()); + # common/src/python/cache.rs PyCache(Rc>) borrows on each + # orders() call. Individual returned orders are clones; re-read each poll. + cache, handle = node.cache, node.handle() + loop = asyncio.get_running_loop() + pending_stop = None + registered_signals = [] + task = None + + def stop(reason): + if receipt["node"]["stop_ns"] is None: + receipt["node"].update(stop_ns=time.time_ns(), stop_reason=reason) + handle.stop() + + def request_stop(reason): + nonlocal pending_stop + pending_stop = pending_stop or reason + + def record_error(step, exc): + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), + "reason": frozen().redact(f"{step}: {type(exc).__name__}: {exc}")}) + + observed_start = None + try: + for sig in SIGNALS: + loop.add_signal_handler(sig, request_stop, f"signal_{sig.name}") + registered_signals.append(sig) + task = asyncio.create_task(node.run_async()) + try: + while not task.done(): + if handle.is_running: + receipt["node"]["started"] = True + observed_start = observed_start or time.monotonic() + update_observations(receipt, snapshot_events(cache), plan) + cases = receipt["cases"] + now = time.monotonic() + if receipt["failures"] or any(c["outcome"] == "failed" for c in cases.values()): + request_stop("case_failure") + elif cases["C1"]["outcome"] == cases["C3"]["outcome"] == "passed": + request_stop("C1_accepted_C3_filled") + elif observed_start is not None and now - observed_start >= plan["timeouts"]["per_step_seconds"]: + request_stop("case_timeout") + if now >= payload["node_stop_at"]: + stop("deadline") + elif pending_stop: + if stop_ready(tester_orders(cache), now, payload["node_stop_at"]): + stop(pending_stop) + write_receipt(receipt_path, receipt, (account_id,)) + await asyncio.sleep(0.1) + except Exception as exc: + record_error("node_poll", exc) + stop("poll_exception") + await task # Native cancel/close runs even after an observer exception. + except Exception as exc: + record_error("node_run", exc) + stop("node_exception") + if task is not None and not task.done(): + try: + await task + except Exception as task_exc: + record_error("node_shutdown", task_exc) + finally: + try: + update_observations(receipt, snapshot_events(cache), plan) + except Exception as exc: + record_error("final_snapshot", exc) + try: + write_receipt(receipt_path, receipt, (account_id,)) + except Exception as exc: + record_error("final_receipt", exc) + for sig in registered_signals: + try: + loop.remove_signal_handler(sig) + except Exception as exc: + record_error("signal_removal", exc) + try: + node.dispose() + except Exception as exc: + record_error("node_dispose", exc) + # Persist cleanup errors independently, without skipping disposal. + write_receipt(receipt_path, receipt, (account_id,)) + return receipt + + +class CheckTimeout(TimeoutError): + pass + + +def official_check(plan, port, *, with_session, deadline): + """Bound frozen ibapi connect/handshake as well as its request waits.""" + seconds = min(plan["timeouts"]["check_seconds"], max(0, deadline - time.monotonic())) + if seconds <= 0: + return {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}}, None + + def expired(_sig, _frame): + raise CheckTimeout("official_check_deadline") + + previous = signal.signal(signal.SIGALRM, expired) + signal.setitimer(signal.ITIMER_REAL, seconds) + try: + return frozen().run_check(plan, port, with_session=with_session, deadline_s=seconds) + except Exception as exc: + return {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}, + "error": frozen().redact(f"{type(exc).__name__}: {exc}")}, None + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, previous) + + +def official_quote(plan, port, account_id, *, deadline): + seconds = min(plan["timeouts"]["check_seconds"], max(0, deadline - time.monotonic())) + if seconds <= 0: + return {"status": "refused_quote_deadline"} + + def expired(_sig, _frame): + raise CheckTimeout("quote_check_deadline") + + previous = signal.signal(signal.SIGALRM, expired) + signal.setitimer(signal.ITIMER_REAL, seconds) + try: + return quote_check(plan, port, account_id, deadline=time.monotonic() + seconds) + except OSError as exc: + return {"status": "not_connected", "error": frozen().redact(exc)} + except CheckTimeout: + return {"status": "refused_quote_deadline"} + except Exception as exc: + return {"status": "refused_quote_error", "error": frozen().redact(exc)} + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, previous) + + +def final_status(receipt, *, interrupted=False, child_returncode=0): + """Flat-proof failure dominates; observed/case/loss breaches remain failed.""" + if receipt["flat_proof"]["status"] != "passed": + return "cleanup_required", "independent_flat_proof", 4, ("C4",) + rt = receipt["roundtrip"] + if (any(f["reason"].startswith("observed_") for f in receipt["failures"]) + or any(c["outcome"] == "failed" for c in receipt["cases"].values()) + or rt and rt["loss_bound_breach"]): + return "failed", "case_or_loss_bound", 2, CASE_IDS + if interrupted or receipt["failures"] or child_returncode != 0: + return "incomplete", "signal_or_node_failure", 3, CASE_IDS + if all(c["outcome"] == "passed" for c in receipt["cases"].values()) and rt and rt["closed"] and not rt["commission_unresolved"]: + return "passed", None, 2, CASE_IDS + return "incomplete", "cases_or_commissions_unresolved", 3, CASE_IDS + + +def run_trial(args, *, now=None, versions=None): + """Every admission refusal precedes official-check and node construction.""" + started_monotonic = time.monotonic() + try: + plan = load_plan(args.plan) + except (ValueError, OSError): + plan = {} + versions = runtime_versions() if versions is None else versions + receipt = new_receipt(plan, args.plan, versions) + errors = validate_plan(plan, port=args.port, node_client_id=args.node_client_id, check_client_id=args.check_client_id) + status = "refused_plan" if errors else runtime_refusal(versions) + now = now or datetime.now(timezone.utc) + if status is None: + inside, window = frozen().rth_check(now, plan) + receipt["window"] = window + if not inside: + status = "refused_outside_window" + if status: + finish(receipt, status, reason="; ".join(errors) if errors else status) + if args.receipt: + write_receipt(args.receipt, receipt) + return receipt + if args.plan_only: + finish(receipt, "passed") + receipt["evidence_class"] = "structural_validation" + if args.receipt: + write_receipt(args.receipt, receipt) + return receipt + if args.receipt is None: + finish(receipt, "refused_receipt_required", reason="receipt required before node") + return receipt + # Command-center r2 decision: runner quantity + quote admission hold the + # bound; the engine cap remains configured with its route limitation stated. + deadline = started_monotonic + plan["timeouts"]["overall_deadline_seconds"] + port = args.port if args.port is not None else plan["default_port"] + pre, account_id = official_check(plan, port, with_session=True, deadline=deadline) + liquid = pre.pop("liquid_hours", "") + pre.pop("trading_hours", None) + zone = pre.pop("time_zone_id", "America/New_York") + receipt["pre_check"] = pre + if pre["status"] != "passed" or not account_id: + finish(receipt, pre["status"] if pre["status"] != "passed" else "refused_account_scope", reason="pre_check") + write_receipt(args.receipt, receipt, (account_id,)) + return receipt + sessions = frozen().parse_liquid_hours(liquid, zone, now.astimezone(frozen().ZoneInfo("America/New_York")).date()) + if sessions is None or not frozen().rth_check(datetime.now(timezone.utc), plan, sessions, horizon_s=deadline - time.monotonic())[0]: + finish(receipt, "refused_liquid_hours", reason="whole_run_broker_window") + write_receipt(args.receipt, receipt, (account_id,)) + return receipt + admitted = official_quote(plan, port, account_id, deadline=deadline) + receipt["quote_admission"] = admitted + if admitted["status"] != "passed": + finish(receipt, admitted["status"], reason="quote_admission") + write_receipt(args.receipt, receipt, (account_id,)) + return receipt + # Atomic provisional cleanup_required receipt BEFORE creating the child/node. + write_receipt(args.receipt, receipt, (account_id,)) + child = None + interrupted = [] + old_handlers = {} + + def signaled(sig, _frame): + interrupted.append(signal.Signals(sig).name) + if child is not None and child.poll() is None: + child.send_signal(sig) + + try: + for sig in SIGNALS: + old_handlers[sig] = signal.signal(sig, signaled) + stop_at = deadline - plan["timeouts"]["post_check_reserve_seconds"] - plan["timeouts"]["cleanup_seconds"] - plan["timeouts"]["node_stop_allowance_seconds"] + command = [sys.executable, str(Path(__file__).resolve()), "--_node-phase", "--plan", str(Path(args.plan).resolve()), + "--port", str(port), "--receipt", str(Path(args.receipt).resolve())] + child_env = {key: value for key, value in os.environ.items() + if key not in ("RUST_LOG", "NAUTILUS_LOG", "TWS_ACCOUNT")} + child = subprocess.Popen(command, stdin=subprocess.PIPE, text=True, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=child_env) + # Anonymous pipe only: no account in argv, environment, file or receipt. + child.stdin.write(json.dumps({"account_id": account_id, "receipt": receipt, "node_stop_at": stop_at, + "quote_admission": admitted, "tob_offset_ticks": admitted["tob_offset_ticks"]})) + child.stdin.close() + try: + child.wait(timeout=max(0, stop_at - time.monotonic())) + except subprocess.TimeoutExpired: + child.send_signal(signal.SIGTERM) + child.wait(timeout=max(0, deadline - plan["timeouts"]["post_check_reserve_seconds"] - time.monotonic())) + except subprocess.TimeoutExpired: + child.kill() + child.wait() + receipt["failures"].append({"acceptance_step": 4, "cases_blocked": ["C2", "C4"], "reason": "node_hard_deadline"}) + except Exception as exc: + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), "reason": frozen().redact(exc)}) + if child is not None and child.poll() is None: + child.send_signal(signal.SIGTERM) + try: + child.wait(timeout=plan["timeouts"]["node_stop_allowance_seconds"]) + except subprocess.TimeoutExpired: + child.kill() + child.wait() + finally: + for sig, handler in old_handlers.items(): + signal.signal(sig, handler) + saved_failures = receipt["failures"] + try: + receipt = json.loads(Path(args.receipt).read_text()) + receipt["failures"].extend(saved_failures) + except (OSError, ValueError): + pass # Preserve provisional state; a corrupt receipt can never pass. + proof, proof_account = official_check(plan, port, with_session=False, deadline=deadline) + if proof["status"] == "passed" and proof_account != account_id: + proof["status"] = "refused_changed_account" + receipt["flat_proof"] = proof + update_observations(receipt, receipt["events"], plan) + status, reason, step, cases = final_status(receipt, interrupted=bool(interrupted), + child_returncode=child.returncode if child is not None else None) + finish(receipt, status, reason=reason, step=step, cases=cases) + write_receipt(args.receipt, receipt, (account_id, proof_account)) + return receipt + + +class PaperArgumentParser(argparse.ArgumentParser): + def error(self, message): + self.exit(3, "refused_arguments: " + frozen().redact(message) + "\n") + + +def parser(): + result = PaperArgumentParser(description=__doc__) + result.add_argument("--plan", type=Path, default=PLAN_PATH) + result.add_argument("--receipt", type=Path, help="sanitized, atomically replaced receipt; required for an order run") + result.add_argument("--plan-only", action="store_true", help="validate plan, pins and window WITHOUT connecting; quote admission is not exercised") + result.add_argument("--port", type=int, help="4002 (default) or 7497 only") + result.add_argument("--node-client-id", type=int, help="91 only") + result.add_argument("--check-client-id", type=int, help="92 only") + result.add_argument("--_node-phase", action="store_true", help=argparse.SUPPRESS) + return result + + +def main(argv=None): + args = parser().parse_args(argv) + try: + if args._node_phase: + # The pipe supplies the admitted quote and derived offset; validate + # before node creation. This entry never loads an account from env. + plan = load_plan(args.plan) + if validate_plan(plan, port=args.port) or runtime_refusal(runtime_versions()): + return 3 + payload = json.load(sys.stdin) + if not frozen().rth_check(datetime.now(timezone.utc), plan, horizon_s=max(0, payload["node_stop_at"] - time.monotonic()))[0] or not validate_child_admission(plan, payload): + return 3 + receipt = asyncio.run(node_phase(plan, args.port, args.receipt, payload)) + return 1 if receipt["failures"] else 0 + receipt = run_trial(args) + except Exception as exc: + print(frozen().scrub_serialized(json.dumps({"status": "incomplete", "exit_code": 1, "error": frozen().redact(exc)}))) + return 1 + print(frozen().scrub_serialized(json.dumps({"status": receipt["status"], "exit_code": receipt["exit_code"], "window": receipt.get("window"), + "failures": sanitize(receipt["failures"])}))) + return receipt["exit_code"] + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.5a061ff06f41 b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.5a061ff06f41 new file mode 100644 index 000000000..f1a0c5753 --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.5a061ff06f41 @@ -0,0 +1,1239 @@ +#!/usr/bin/env python3 +"""Bounded rc5 paper trial using upstream's built-in ExecTester. + +The rc5 engine cap is configured; quote admission and quantity one hold this +trial's runner bound on the SMART/IB route affected by #4946. --plan-only and +--help never connect. Synthetic tests do not establish broker acceptance. +""" +from __future__ import annotations + +import argparse +import asyncio +import copy +import hashlib +import importlib.metadata +import importlib.util +import json +import os +import platform +import signal +import subprocess +import sys +import tempfile +import threading +import time +from datetime import datetime, timezone +from decimal import Decimal, InvalidOperation, ROUND_FLOOR +from functools import lru_cache +from pathlib import Path + +HERE = Path(__file__).resolve().parent +PLAN_PATH = HERE / "plan.json" +FROZEN_RUN = HERE.parent / "ibkr-paper-orders" / "run.py" +KIND = "ibkr_paper_orders_nautilus_2_0_0rc5" +UPSTREAM_COMMIT = "1b0a49d2792a9432a3aca3fcb617ce7a630d905e" +SOURCE = f"https://github.com/nautechsystems/nautilus_trader/blob/{UPSTREAM_COMMIT}" +CASE_IDS = ("C1", "C2", "C3", "C4") +SIGNALS = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP) +TERMINAL_FAILURES = {"OrderDenied", "OrderRejected", "OrderExpired", "OrderCancelRejected"} +TESTER_SETTINGS = { + "order_qty": "1", "open_position_on_start_qty": "1", "open_position_on_first_quote": True, + "open_position_time_in_force": "IOC", "enable_limit_buys": True, "enable_limit_sells": False, + "enable_stop_buys": False, "enable_stop_sells": False, "enable_brackets": False, + "tob_offset_ticks": "floor(0.5 * admitted_bid / 0.01)", "limit_time_in_force": "GTD", "order_expire_time_delta_mins": 7, + "modify_orders_to_maintain_tob_offset": False, "cancel_replace_orders_to_maintain_tob_offset": False, + "use_post_only": False, "use_individual_cancels_on_stop": True, + "cancel_orders_on_stop": True, "close_positions_on_stop": True, + "reduce_only_on_stop": False, "dry_run": False, "log_data": False, +} +CASE_DEFINITIONS = [ + {"id": "C1", "name": "accept_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderAccepted"}, + {"id": "C2", "name": "cancel_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderCanceled for C1 after stop"}, + {"id": "C3", "name": "fill_buy", "order_type": "MARKET", "side": "BUY", "expect": "OrderFilled quantity 1"}, + {"id": "C4", "name": "flatten", "order_type": "MARKET", "side": "SELL", "expect": "OrderFilled quantity 1 after stop and independent flat proof"}, +] + +# Original sources, read at v2.0.0rc5 (UPSTREAM_COMMIT): +# examples/live/interactive_brokers/exec_tester.py: builder + ExecTester config. +# examples/live/interactive_brokers/_common.py: bounded node timeouts. +# crates/testkit/src/testers/exec/strategy.rs: native entry/cancel/close paths. +# crates/risk/src/engine/mod.rs: check_orders_risk_for_account returns true +# when account_for_venue(SMART) is absent, before checking max_notional. The +# IB account is registered under IB. Config support is not enforcement. +RISK_NOTE = "engine notional cap configured, not enforced on this route (#4946, fixed on develop, unreleased); bound held by qty=1 and quote admission" +RISK_ROUTE = { + "issue": "https://github.com/nautechsystems/nautilus_trader/issues/4946", + "issue_state": "closed", + "pin_fix_present": False, + "upstream_fix": "fixed on develop; included in v2.0.0rc6; absent from the selected rc5 pin", + "source": SOURCE + "/crates/risk/src/engine/mod.rs#L1218", + "reason": "rc5 stock orders can skip account-scoped max_notional_per_order on the SMART/IB venue path", +} +RELEASE_VERIFICATION = { + "checked_at": "2026-10-05", + "fix_commit": "ed6fc8bf47fd37dda97d63b9b2df160719ee2bac", + "released_in": "v2.0.0rc6", + "published_at": "2026-10-05T03:03:18Z", + "source": "https://github.com/nautechsystems/nautilus_trader/releases/tag/v2.0.0rc6", + "correction": "The requested note's unreleased clause reflects earlier metadata; the fix is now released in rc6, while this trial stays pinned to rc5.", +} +BLOCKED_STEPS = [ + {"acceptance_step": 3, "operation": "reconnect_with_open_order", "exercised": False, + "issues": [{"number": 5057, "state": "open"}, {"number": 5060, "state": "open"}]}, + {"acceptance_step": 3, "operation": "restart_reconciliation", "exercised": False, + "issues": [{"number": 5007, "state": "open"}, {"number": 5057, "state": "open"}, + {"number": 5060, "state": "open"}]}, + {"acceptance_step": 4, "operation": "kill_switch", "exercised": False, + "issues": [{"number": 5060, "state": "open"}]}, +] +for _step in BLOCKED_STEPS: + _step["issue_states_checked_at"] = "2026-10-05" + for _issue in _step["issues"]: + _issue["url"] = f"https://github.com/nautechsystems/nautilus_trader/issues/{_issue['number']}" + + +@lru_cache(maxsize=1) +def frozen(): + """Reuse the frozen official-ibapi checker, window parser and redaction. + + Source: ../ibkr-paper-orders/run.py, bound by source_hashes in the receipt. + No frozen strategy is used; the node uses rc5's built-in ExecTester. + """ + spec = importlib.util.spec_from_file_location("ibkr_frozen_readonly", FROZEN_RUN) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def sha256(path): + return hashlib.sha256(Path(path).read_bytes()).hexdigest() + + +def decimal(value): + result = Decimal(str(value)) + if not result.is_finite(): + raise ValueError("nonfinite numeric input") + return result + + +def load_plan(path=PLAN_PATH): + def unique(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError("duplicate plan key") + result[key] = value + return result + return json.loads(Path(path).read_text(), object_pairs_hook=unique) + + +def validate_plan(plan, *, port=None, node_client_id=None, check_client_id=None): + """All static bounds are checked without importing a client or connecting.""" + errors = [] + + def need(condition, reason): + if not condition: + errors.append(reason) + + try: + need(type(plan["schema_version"]) is int and plan["schema_version"] == 1 + and plan["kind"] == "ibkr_paper_orders_plan_rc5", "plan schema") + need(plan["engine"]["version"] == "2.0.0rc5" and plan["engine"]["ibapi_version"] == "10.45.1", "runtime pins") + need(plan["engine"]["upstream_commit"] == UPSTREAM_COMMIT, "upstream pin") + need(plan["host"] == "127.0.0.1", "loopback host required") + ports = plan["paper_ports"] + need(isinstance(ports, list) and bool(ports) and all(type(p) is int and p in (4002, 7497) for p in ports), "paper ports") + need(plan["default_port"] in ports and type(plan["default_port"]) is int, "default paper port") + need(port is None or type(port) is int and port in ports and port in (4002, 7497), "selected paper port") + need(plan["account_prefix"] == "DU" and plan["require_single_managed_account"] is True, "single DU account") + need(plan["client_ids"] == {"node": 91, "check": 92} + and all(type(v) is int for v in plan["client_ids"].values()), "client ids 91/92") + need(node_client_id is None or type(node_client_id) is int and node_client_id == 91, "node client id 91") + need(check_client_id is None or type(check_client_id) is int and check_client_id == 92, "check client id 92") + inst = plan["instrument"] + need(tuple(inst[k] for k in ("symbol", "sec_type", "exchange", "primary_exchange", "currency")) + == ("SPY", "STK", "SMART", "ARCA", "USD"), "SPY STK SMART/ARCA USD only") + need(inst["nautilus_instrument_id"] == "SPY=STK.SMART" and inst["expected_price_increment"] == "0.01", "RAW stock identity and tick") + b = plan["bounds"] + need(type(b["max_orders"]) is int and 3 <= b["max_orders"] <= 6, "order budget 3..6") + need(type(b["max_quantity_per_order"]) is int and b["max_quantity_per_order"] == 1, "quantity 1") + for name, ceiling in (("max_notional_per_order_usd", 1000), ("max_roundtrip_loss_usd", 5)): + need(not isinstance(b[name], bool) and 0 < decimal(b[name]) <= ceiling, name) + need(decimal(b["commission_allowance_per_order_usd"]) >= 0, "commission allowance") + s = plan["session"] + need((s["timezone"], s["open"], s["close"]) == ("America/New_York", "09:30", "16:00"), "regular trading hours") + need(s["weekdays_only"] is True and s["whole_run_inside_window"] is True + and s["use_contract_liquid_hours"] is True, "whole run with broker liquid hours") + need(type(s["close_buffer_minutes"]) is int and s["close_buffer_minutes"] >= 10, "10 minute close buffer") + expected_times = {"check_seconds": 30, "node_start_seconds": 60, "per_step_seconds": 45, + "cleanup_seconds": 45, "node_stop_allowance_seconds": 15, + "post_check_reserve_seconds": 60, "overall_deadline_seconds": 420} + need(plan["timeouts"] == expected_times and all(type(v) is int for v in plan["timeouts"].values()), + "frozen timeouts and hard deadline") + r = plan["risk"] + need(r["bypass"] is False and r["require_enforced_notional_before_connection"] is False, + "native risk configured with runner quote admission") + need(not isinstance(r["notional_headroom_usd"], bool) + and decimal(r["notional_headroom_usd"]) == 10, "USD 10 quote admission headroom") + need(r["max_order_submit_rate"] == f"{b['max_orders']}/00:07:00", "whole-run native submit throttle") + # No configurable strategy knobs can quietly turn this into another trial. + need(plan["exec_tester"] == TESTER_SETTINGS + and all(type(plan["exec_tester"][k]) is type(v) for k, v in TESTER_SETTINGS.items()), + "frozen ExecTester settings") + need(plan["data"] == {"market_data_type": "REALTIME", "quote_max_age_seconds": 10, + "clock_resolution_tolerance_seconds": 2, "batch_quotes": False}, "real-time quote bounds") + need(plan["cases"] == CASE_DEFINITIONS, "frozen cases C1..C4") + need(plan["end_disposition"] == {"positions": 0, "open_orders": 0}, "flat disposition") + need(plan["receipt"]["kind"] == KIND and plan["evidence_class"] == "native_paper", "receipt identity") + except (KeyError, TypeError, ValueError, InvalidOperation): + errors.append("missing or malformed plan field") + return errors + + +def runtime_versions(): + def version(package): + try: + return importlib.metadata.version(package) + except importlib.metadata.PackageNotFoundError: + return None + try: + import ibapi + ib_version = ibapi.__version__ + except (ImportError, AttributeError): + ib_version = None + try: + import nautilus_trader + engine_version = nautilus_trader.__version__ + except (ImportError, AttributeError): + engine_version = None + return {"nautilus_trader": engine_version, "nautilus_trader_distribution": version("nautilus_trader"), "ibapi": ib_version, + "nautilus_ibapi_distribution": version("nautilus-ibapi"), + "protobuf": version("protobuf"), "python": platform.python_version()} + + +def runtime_refusal(versions): + if versions.get("nautilus_trader") != "2.0.0rc5" or versions.get("ibapi") != "10.45.1": + return "refused_unpinned_runtime" + if "nautilus_trader_distribution" in versions and versions["nautilus_trader_distribution"] != "2.0.0rc5": + return "refused_unpinned_runtime" + return None + + +def resting_offset_ticks(bid, plan): + """Frozen half-bid placement through upstream ExecTester's supported offset.""" + tick = decimal(plan["instrument"]["expected_price_increment"]) + return int((decimal(bid) * Decimal("0.5") / tick).to_integral_value(rounding=ROUND_FLOOR)) + + +def admit_quote(plan, quote, server_time_epoch, *, elapsed_seconds=0, delayed=False, errors=()): + """Pure admission using IB's timestamped BidAsk and its independent clock.""" + result = {"source": "ibapi.reqTickByTickData.BidAsk/reqCurrentTime", "status": None, + "bid": None, "ask": None, "quote_age_seconds": None} + if delayed or any(code in (10089, 10167) for code in errors): + result["status"] = "refused_delayed_quote" + return result + if errors: + result["status"] = "refused_quote_error" + return result + try: + bid, ask = decimal(quote["bid"]), decimal(quote["ask"]) + age = decimal(server_time_epoch) - decimal(quote["quote_time_epoch"]) + decimal(elapsed_seconds) + result.update(bid=str(bid), ask=str(ask), quote_time_epoch=int(quote["quote_time_epoch"]), + server_time_epoch=int(server_time_epoch), quote_age_seconds=float(max(age, Decimal(0)))) + if bid <= 0 or ask < bid: + result["status"] = "refused_invalid_quote" + elif age < -decimal(plan["data"]["clock_resolution_tolerance_seconds"]) or age > decimal(plan["data"]["quote_max_age_seconds"]): + result["status"] = "refused_stale_quote" + else: + notional = ask * plan["bounds"]["max_quantity_per_order"] + headroom = decimal(plan["risk"]["notional_headroom_usd"]) + result.update(ask_notional_usd=str(notional), headroom_usd=str(headroom), + tob_offset_ticks=resting_offset_ticks(bid, plan)) + result["status"] = "passed" if notional + headroom <= decimal(plan["bounds"]["max_notional_per_order_usd"]) and result["tob_offset_ticks"] > 0 else "refused_quote_notional" + except (KeyError, TypeError, ValueError, InvalidOperation): + result["status"] = "refused_invalid_quote" + return result + + +# Official ibapi 10.45.1 client.py reqTickByTickData and wrapper.py callbacks +# were inspected in the prepared runtime. Delayed enums come from ticktype.py. +DELAYED_TICKS = set(range(66, 77)) | {80, 81, 82, 83, 88, 90, 103, 104} +CONNECTIVITY_CODES = {326, 502, 503, 504, 507, 1100, 1300, 2110} + + +class QuoteState: + """Read-only callback state, kept independent of ibapi for synthetic tests.""" + + def __init__(self): + self.ready = threading.Event() + self.quote_ready = threading.Event() + self.clock_ready = threading.Event() + self.accounts_ready = threading.Event() + self.accounts = [] # Memory only. + self.quote = None + self.server_time = None + self.clock_received = None + self.delayed = False + self.errors = [] + self.info = [] + self.started_monotonic = time.monotonic() + self.last_stage = "initializing" + self.stage_elapsed_seconds = {} + self.stage_duration_seconds = {} + self.stage_started = self.started_monotonic + self.quote_phase = False + + def mark(self, stage): + if stage not in self.stage_elapsed_seconds: + now = time.monotonic() + self.stage_elapsed_seconds[stage] = round(now - self.started_monotonic, 4) + self.stage_duration_seconds[stage] = round(now - self.stage_started, 4) + self.stage_started, self.last_stage = now, stage + + def diagnostics(self): + return {"last_stage": self.last_stage, "elapsed_seconds": round(time.monotonic() - self.started_monotonic, 4), + "last_stage_elapsed_seconds": round(time.monotonic() - self.stage_started, 4), + "stage_elapsed_seconds": dict(self.stage_elapsed_seconds), + "stage_duration_seconds": dict(self.stage_duration_seconds), + "errors": list(self.errors), "info": list(self.info)} + + def nextValidId(self, orderId): + self.mark("nextValidId") + self.ready.set() + + def managedAccounts(self, accountsList): + self.accounts = [value.strip() for value in (accountsList or "").split(",") if value.strip()] + self.mark("accounts") + self.accounts_ready.set() + + def tickByTickBidAsk(self, reqId, timestamp, bidPrice, askPrice, bidSize, askSize, tickAttribBidAsk): + if reqId == 9202: + self.quote = {"bid": str(bidPrice), "ask": str(askPrice), "quote_time_epoch": timestamp} + self.mark("quote") + self.quote_ready.set() + + def currentTime(self, timestamp): + self.server_time, self.clock_received = timestamp, time.monotonic() + if self.quote_phase: + self.mark("clock") + self.clock_ready.set() + + def error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson=""): + if errorCode in (10089, 10167): + self.delayed = True + entry = {"reqId": reqId, "code": errorCode, + "text": frozen().scrub_serialized(frozen().redact(errorString)[:160], self.accounts)} + fatal = reqId == 9202 or errorCode in CONNECTIVITY_CODES or errorCode in (10089, 10167) + (self.errors if fatal else self.info).append(entry) + if fatal: + self.quote_ready.set() + self.clock_ready.set() + if errorCode in CONNECTIVITY_CODES: + self.ready.set() + self.accounts_ready.set() + + def marketDataType(self, reqId, marketDataType): + if marketDataType in (3, 4): + self.delayed = True + self.quote_ready.set() + + def delayed_tick(self, tickType): + if tickType in DELAYED_TICKS: + self.delayed = True + self.quote_ready.set() + + def tickPrice(self, reqId, tickType, price, attrib): + self.delayed_tick(tickType) + + def tickSize(self, reqId, tickType, size): + self.delayed_tick(tickType) + + def tickString(self, reqId, tickType, value): + self.delayed_tick(tickType) + + def tickGeneric(self, reqId, tickType, value): + self.delayed_tick(tickType) + + def tickOptionComputation(self, reqId, tickType, *values): + self.delayed_tick(tickType) + + +class AdmissionState(frozen().CheckState, QuoteState): + """Reuse frozen flat-check callbacks and quote callbacks on one session.""" + + def __init__(self): + frozen().CheckState.__init__(self) + QuoteState.__init__(self) + + def managedAccounts(self, accountsList): + frozen().CheckState.managedAccounts(self, accountsList) + QuoteState.managedAccounts(self, accountsList) + + def currentTime(self, timestamp): + frozen().CheckState.currentTime(self, timestamp) + QuoteState.currentTime(self, timestamp) + + def error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson=""): + frozen().CheckState.error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson) + QuoteState.error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson) + + +def build_admission_client(): + from ibapi.client import EClient + from ibapi.wrapper import EWrapper + + class QuoteClient(AdmissionState, EWrapper, EClient): + def __init__(self): + AdmissionState.__init__(self) + EClient.__init__(self, self) + + return QuoteClient() + + +class CheckSession: + """Hold the frozen check's session until quote admission and join its reader. + + Official ibapi 10.45.1 client.py: connect creates EReader and startApi; + run drains the shared message queue; disconnect closes/reset but does not + join those threads. Intercept only the frozen harness's final disconnect, + leaving EClient's own disconnect on connection loss unchanged. + """ + + def __init__(self, client): + self.client, self.runner = client, None + + def __getattr__(self, name): + return getattr(self.client, name) + + def connect(self, host, port, client_id): + self.client.connect(host, port, client_id) + if self.client.isConnected(): + self.client.mark("connected") + + def run(self): + self.runner = threading.current_thread() + self.client.run() + + def disconnect(self): + pass # The outer admission scope owns teardown after both read phases. + + def close(self, deadline): + reader = getattr(self.client, "reader", None) + try: + self.client.disconnect() + finally: + for thread in (self.runner, reader): + if thread is not None and thread is not threading.current_thread(): + thread.join(timeout=max(0, deadline - time.monotonic())) + + +def quote_check(plan, account_id, *, deadline, client): + """Continue the already-connected flat-check client; never reconnect.""" + requested = False + + def wait(event): + while not event.is_set() and time.monotonic() < deadline: + if client.errors or not client.isConnected(): + return False + event.wait(min(0.05, max(0, deadline - time.monotonic()))) + return event.is_set() + + def result(value): + return {**value, **client.diagnostics()} + + def failed_wait(status): + if client.errors or client.delayed: + return result(admit_quote(plan, client.quote, client.server_time, + delayed=client.delayed, errors=[entry["code"] for entry in client.errors])) + return result({"status": status, "error": "quote_check_deadline" if time.monotonic() >= deadline else "connection_lost"}) + + try: + if not client.isConnected(): + return result({"status": "not_connected"}) + if not wait(client.ready) or not wait(client.accounts_ready): + return failed_wait("not_connected") + if client.errors: + return failed_wait("not_connected") + if client.accounts != [account_id] or not account_id.startswith("DU"): + return result({"status": "refused_quote_account_scope"}) + client.quote_phase = True + client.reqMarketDataType(1) + requested = True + client.mark("tick requested") + client.reqTickByTickData(9202, frozen().spy_contract(), "BidAsk", 0, False) + if not wait(client.quote_ready): + return failed_wait("refused_quote_unavailable") + if client.delayed or client.errors: + return failed_wait("refused_quote_error") + client.clock_ready.clear() # Discard the clock from the flat pre-check. + client.server_time, client.clock_received = None, None + client.reqCurrentTime() + if not wait(client.clock_ready): + return failed_wait("refused_quote_clock") + elapsed = max(0, time.monotonic() - client.clock_received) if client.clock_received is not None else 0 + admitted = admit_quote(plan, client.quote, client.server_time, elapsed_seconds=elapsed, + delayed=client.delayed, errors=[entry["code"] for entry in client.errors]) + admitted["admitted_monotonic"] = time.monotonic() + return result(admitted) + finally: + if requested: + client.cancelTickByTickData(9202) + + +def validate_child_admission(plan, payload, *, monotonic_now=None): + """Recompute age, cap and offset from the pipe; no account in diagnostics.""" + try: + admitted = payload["quote_admission"] + now = time.monotonic() if monotonic_now is None else monotonic_now + elapsed = decimal(now) - decimal(admitted["admitted_monotonic"]) + if admitted["status"] != "passed" or elapsed < 0: + return False + # Stored age already includes time since the independent server callback. + rechecked = admit_quote(plan, admitted, decimal(admitted["quote_time_epoch"]) + decimal(admitted["quote_age_seconds"]), + elapsed_seconds=elapsed) + offset = payload["tob_offset_ticks"] + return (rechecked["status"] == "passed" and type(offset) is int + and offset == resting_offset_ticks(admitted["bid"], plan) + and offset == admitted["tob_offset_ticks"] + and isinstance(payload["account_id"], str) and payload["account_id"].startswith("DU")) + except (KeyError, TypeError, ValueError, InvalidOperation): + return False + + +def sanitize(value): + """Prices/quantities remain structured; all untrusted text uses frozen redaction.""" + if isinstance(value, dict): + return {frozen().redact(k): sanitize(v) for k, v in value.items()} + if isinstance(value, (list, tuple)): + return [sanitize(v) for v in value] + if isinstance(value, str): + # Constant source locators are public evidence, not untrusted error text. + public_sources = {RISK_ROUTE["source"], RISK_ROUTE["issue"], RELEASE_VERIFICATION["source"]} + public_sources.update(issue["url"] for step in BLOCKED_STEPS for issue in step["issues"]) + if value in public_sources: + return frozen().scrub_serialized(value) + return frozen().redact(value) + return value + + +def serialized_receipt(receipt, secret_values=()): + text = json.dumps(sanitize(receipt), indent=2, sort_keys=True, allow_nan=False) + # Final id-shaped-text pass also covers dictionary keys and unexpected nested fields. + return frozen().scrub_serialized(text, secret_values) + "\n" + + +def write_receipt(path, receipt, secret_values=()): + path = Path(path) + protected = {PLAN_PATH.resolve(), Path(__file__).resolve(), FROZEN_RUN.resolve()} + if path.resolve() in protected or frozen().is_gate_receipt(path): + raise ValueError("protected receipt destination") + raw = serialized_receipt(receipt, secret_values) + path.parent.mkdir(parents=True, exist_ok=True) + temporary = None + try: + with tempfile.NamedTemporaryFile(mode="w", dir=path.parent, prefix=".paper-receipt-", delete=False) as stream: + temporary = Path(stream.name) + stream.write(raw) + os.replace(temporary, path) + finally: + if temporary is not None: + temporary.unlink(missing_ok=True) + + +def case_outcomes(events, *, stop_ns=None, flat_proof=None): + """Pure mapping of sanitized native cache events; no strings imply acceptance. + + Each event carries an opaque local order alias and its type/side/quantity/price. + C2 must cancel C1, and C4 must fill after stop AND have an independent flat proof. + Duplicate native event ids are removed in snapshot_events, before this mapping. + """ + definitions = (("LIMIT", "BUY"), ("LIMIT", "BUY"), ("MARKET", "BUY"), ("MARKET", "SELL")) + result = {cid: {"id": cid, "order_type": typ, "side": side, "price": None, + "order": None, "events": [], "outcome": "not_run", "reason": None} + for cid, (typ, side) in zip(CASE_IDS, definitions)} + aliases = {} + for event in events: + aliases.setdefault(event["order"], event) + chosen = {} + for cid, shape in (("C1", ("LIMIT", "BUY")), ("C3", ("MARKET", "BUY")), ("C4", ("MARKET", "SELL"))): + candidates = [key for key, e in aliases.items() if (e["order_type"], e["side"]) == shape] + if candidates: + chosen[cid] = candidates[0] + if "C1" in chosen: + chosen["C2"] = chosen["C1"] + for cid, alias in chosen.items(): + case = result[cid] + seq = [e for e in events if e["order"] == alias] + case.update(order=alias, price=seq[0].get("price"), events=seq, outcome="incomplete") + confirmed = [e for e in seq if not e.get("reconciliation", False)] + bad = [e for e in seq if e["type"] in TERMINAL_FAILURES or e.get("reconciliation", False)] + if bad: + case.update(outcome="failed", reason=f"{cid}_unconfirmed_or_rejected") + continue + if decimal(seq[0]["quantity"]) != 1: + case.update(outcome="failed", reason=f"{cid}_quantity_bound") + continue + if cid == "C1": + if any(e["type"] == "OrderFilled" for e in confirmed): + case.update(outcome="failed", reason="C1_resting_order_filled") + elif any(e["type"] == "OrderAccepted" for e in confirmed): + case["outcome"] = "passed" + elif cid == "C2": + accepted = [e for e in confirmed if e["type"] == "OrderAccepted"] + canceled = [e for e in confirmed if e["type"] == "OrderCanceled"] + if stop_ns is not None and accepted and any(e["ts_init"] >= stop_ns and e["ts_init"] >= accepted[0]["ts_init"] for e in canceled): + case["outcome"] = "passed" + else: + fills = [e for e in confirmed if e["type"] == "OrderFilled"] + filled = sum((decimal(e["fill_quantity"]) for e in fills), Decimal(0)) + if filled > 1: + case.update(outcome="failed", reason=f"{cid}_overfill") + elif filled == 1: + if cid == "C3": + case["outcome"] = "passed" + elif stop_ns is not None and all(e["ts_init"] >= stop_ns for e in fills): + case["outcome"] = "filled_unproven" + if flat_proof and flat_proof.get("status") == "passed": + observed = flat_proof.get("observed", {}) + if observed.get("positions") == 0 and observed.get("open_orders") == 0: + case["outcome"] = "passed" + if case["outcome"] == "incomplete": + case["reason"] = f"{cid}_expected_event_missing" + return result + + +def roundtrip(events, loss_bound): + fills = [e for e in events if e["type"] == "OrderFilled"] + if not fills: + return None + buys = sum((decimal(e["fill_quantity"]) for e in fills if e["side"] == "BUY"), Decimal(0)) + sells = sum((decimal(e["fill_quantity"]) for e in fills if e["side"] == "SELL"), Decimal(0)) + gross = sum((decimal(e["fill_price"]) * decimal(e["fill_quantity"]) * (1 if e["side"] == "SELL" else -1) for e in fills), Decimal(0)) + unresolved = any(e.get("commission") is None or decimal(e["commission"]) <= 0 + or e.get("commission_currency") != "USD" or e.get("currency") != "USD" for e in fills) + fees = None if unresolved else sum((decimal(e["commission"]) for e in fills), Decimal(0)) + closed = buys == sells == 1 + net = gross - fees if fees is not None and closed else None + return {"buy_quantity": str(buys), "sell_quantity": str(sells), "closed": closed, + "gross_usd": str(gross) if buys == sells else None, + "commissions_usd": str(fees) if fees is not None else None, + "net_usd": str(net) if net is not None else None, + "commission_unresolved": unresolved, + "loss_bound_breach": net < -decimal(loss_bound) if net is not None else None} + + +def new_receipt(plan, plan_path, versions): + if not isinstance(plan, dict): + plan = {} + bounds = plan.get("bounds") if isinstance(plan.get("bounds"), dict) else {} + try: + cap = str(decimal(bounds["max_notional_per_order_usd"])) + except (KeyError, ValueError, InvalidOperation): + cap = None + try: + plan_hash = sha256(plan_path) + except OSError: + plan_hash = None + return {"schema_version": 1, "kind": KIND, "evidence_class": "native_paper", + "started_at": datetime.now(timezone.utc).isoformat(), "versions": versions, + "plan_sha256": plan_hash, "harness_sha256": sha256(__file__), + "source_hashes": {"frozen_checker_sha256": sha256(FROZEN_RUN)}, + "upstream_commit": UPSTREAM_COMMIT, "pre_check": None, "quote_admission": None, + "cases": case_outcomes([]), "events": [], "reconciled_external": [], "fills": [], "roundtrip": None, + "flat_proof": None, "node": {"started": False, "stop_ns": None}, + "risk": {"bypass": False, "max_notional_per_order_usd": cap, + "enforcement": "runner_quantity_and_quote_admission", "note": RISK_NOTE, + "engine_route": RISK_ROUTE, "submit_budget_enforcement": "engine_throttle", + "release_verification": RELEASE_VERIFICATION}, + "blocked_steps": BLOCKED_STEPS, "failures": [], + "status": "cleanup_required", "exit_code": 3} + + +def finish(receipt, status, *, reason=None, step=2, cases=CASE_IDS): + receipt["status"] = status + receipt["exit_code"] = frozen().exit_code_for(status) + receipt["ended_at"] = datetime.now(timezone.utc).isoformat() + if reason: + receipt["failures"].append({"acceptance_step": step, "cases_blocked": list(cases), "reason": reason}) + return receipt["exit_code"] + + +def native_configs(plan, port, account_id, *, tob_offset_ticks): + """Only upstream configs, with installed rc5 signatures verified offline. + + Source: v2.0.0rc5 examples/live/interactive_brokers/{exec_tester,_common}.py. + Construction does not connect. Actual account_id is supplied by the official + pre-check in memory, never from TWS_ACCOUNT or a command line argument. + """ + from nautilus_trader.adapters import interactive_brokers as ib + from nautilus_trader.config import LiveRiskEngineConfig + from nautilus_trader.model import ClientId, InstrumentId, Quantity, StrategyId, TimeInForce + from nautilus_trader.testkit import ExecTesterConfig + instrument = InstrumentId.from_str(plan["instrument"]["nautilus_instrument_id"]) + provider = ib.InteractiveBrokersInstrumentProviderConfig(symbology_method=ib.SymbologyMethod.RAW, load_ids={instrument}) + common = dict(host=plan["host"], port=port, client_id=plan["client_ids"]["node"], + connection_timeout=10, request_timeout=30, instrument_provider=provider) + tester_values = dict(plan["exec_tester"]) + tester_values["tob_offset_ticks"] = tob_offset_ticks + tester_values["order_qty"] = Quantity.from_str(tester_values["order_qty"]) + tester_values["open_position_on_start_qty"] = Decimal(tester_values["open_position_on_start_qty"]) + for key in ("open_position_time_in_force", "limit_time_in_force"): + tester_values[key] = TimeInForce.from_str(tester_values[key]) + tester = ExecTesterConfig(strategy_id=StrategyId.from_str("EXEC_TESTER-001"), instrument_id=instrument, + client_id=ClientId.from_str("IB"), external_order_instrument_ids=[instrument], + subscribe_quotes=True, subscribe_trades=True, **tester_values) + risk = LiveRiskEngineConfig(bypass=False, max_order_submit_rate=plan["risk"]["max_order_submit_rate"], + max_notional_per_order={str(instrument): str(plan["bounds"]["max_notional_per_order_usd"])}) + return (ib, ib.InteractiveBrokersDataClientConfig(market_data_type=ib.MarketDataType.REALTIME, batch_quotes=False, **common), + ib.InteractiveBrokersExecutionClientConfig(account_id=account_id, **common), risk, tester) + + +def build_node(plan, port, account_id, *, tob_offset_ticks): + from nautilus_trader.common import Environment, LoggerConfig, LogLevel + from nautilus_trader.live import LiveNode + from nautilus_trader.model import TraderId + ib, data, execution, risk, tester = native_configs(plan, port, account_id, tob_offset_ticks=tob_offset_ticks) + node = (LiveNode.builder("IB-EXEC-TESTER-001", TraderId.from_str("TESTER-001"), Environment.LIVE) + .with_reconciliation(True).with_risk_engine_config(risk) + .with_logging(LoggerConfig(stdout_level=LogLevel.OFF, fileout_level=LogLevel.OFF, print_config=False)) + .with_timeout_connection(plan["timeouts"]["node_start_seconds"]) + .with_timeout_reconciliation(5).with_timeout_portfolio(5) + .with_timeout_disconnection_secs(5).with_delay_post_stop_secs(plan["timeouts"]["cleanup_seconds"]) + .add_data_client(None, ib.InteractiveBrokersDataClientFactory(), data) + .add_exec_client(None, ib.InteractiveBrokersExecutionClientFactory(), execution).build()) + node.add_builtin_strategy("ExecTester", tester) + return node + + +class RunOrderScope: + """In-memory client-order-id lineage for this ExecTester's submissions. + + Pinned upstream: crates/execution/src/engine/mod.rs materializes and claims + external orders using this strategy id and the current initialization time. + Neither alone proves ownership. IB execution/core_orders.rs emits a local + OrderSubmitted with the original client_order_id on this run's submit path. + Require its matching non-reconciled initialization after run_async launch; + retain ownership if later events reconcile, so the never-pass rule applies. + Raw identifiers stay here; receipts use stable O/X aliases only. + """ + + def __init__(self, started_ns): + self.started_ns = started_ns + self.owned = {} + self.external = {} + + def submitted_here(self, order): + if str(order.strategy_id) != "EXEC_TESTER-001" or str(order.trader_id) != "TESTER-001": + return False + initialized = submitted = False + for event in order.events(): + if (str(event.client_order_id) != str(order.client_order_id) + or str(event.strategy_id) != str(order.strategy_id) + or str(event.trader_id) != str(order.trader_id) + or event.ts_init < self.started_ns + or getattr(event, "reconciliation", False)): + continue + initialized |= type(event).__name__ == "OrderInitialized" + submitted |= type(event).__name__ == "OrderSubmitted" + return initialized and submitted + + def partition(self, orders): + own, other = [], [] + for order in sorted(orders, key=lambda order: (order.ts_init, str(order.client_order_id))): + key = str(order.client_order_id) + if key not in self.owned and self.submitted_here(order): + self.owned[key] = f"O{len(self.owned) + 1}" + if key in self.owned: + own.append(order) + else: + self.external.setdefault(key, f"X{len(self.external) + 1}") + other.append(order) + return own, other + + +def tester_orders(cache, scope): + return scope.partition(cache.orders())[0] + + +def snapshot_events(cache, scope, *, external=False): + """Read cloned order histories, never serialize account/order/venue/trade ids.""" + orders = scope.partition(cache.orders())[1 if external else 0] + orders.sort(key=lambda order: (order.ts_init, str(order.client_order_id))) + events, seen = [], set() + for order in orders: + for event in order.events(): + key = str(event.event_id) + if key in seen: + continue + seen.add(key) + price = getattr(order, "price", None) + alias = (scope.external if external else scope.owned)[str(order.client_order_id)] + record = {"order": alias, "order_type": order.order_type.name, "side": order.side.name, + "instrument_id": str(order.instrument_id), + "quantity": str(order.quantity), "price": str(price) if price is not None else None, + "type": type(event).__name__, "ts_event": int(event.ts_event), "ts_init": int(event.ts_init), + "at": frozen().ns_to_iso(event.ts_event), "reconciliation": bool(getattr(event, "reconciliation", False))} + if record["type"] == "OrderFilled": + commission = event.commission + record.update(fill_price=str(event.last_px), fill_quantity=str(event.last_qty), currency=str(event.currency), + commission=str(commission.as_decimal()) if commission is not None else None, + commission_currency=str(commission.currency) if commission is not None else None) + if external: + record.pop("instrument_id") + if not external and hasattr(event, "reason"): + record["reason"] = frozen().redact(event.reason) + events.append(record) + return sorted(events, key=lambda event: (event["ts_init"], event["ts_event"])) + + +def update_observations(receipt, events, plan): + receipt["events"] = events + receipt["cases"] = case_outcomes(events, stop_ns=receipt["node"]["stop_ns"], flat_proof=receipt["flat_proof"]) + receipt["fills"] = [e for e in events if e["type"] == "OrderFilled"] + receipt["roundtrip"] = roundtrip(events, plan["bounds"]["max_roundtrip_loss_usd"]) + orders = {e["order"]: e for e in events} + violations = [] + if len(orders) > plan["bounds"]["max_orders"]: + violations.append(("observed_order_budget_breach", list(CASE_IDS))) + for e in orders.values(): + cases = ["C1", "C2"] if e["order_type"] == "LIMIT" else ["C3" if e["side"] == "BUY" else "C4"] + quantity = decimal(e["quantity"]) + if quantity != 1: + violations.append(("observed_quantity_bound_breach", cases)) + if e.get("instrument_id") != plan["instrument"]["nautilus_instrument_id"]: + violations.append(("observed_instrument_bound_breach", cases)) + if e.get("price") is not None and decimal(e["price"]) * quantity > decimal(plan["bounds"]["max_notional_per_order_usd"]): + violations.append(("observed_notional_bound_breach", cases)) + for e in events: + if e["type"] == "OrderFilled" and decimal(e["fill_price"]) * decimal(e["fill_quantity"]) > decimal(plan["bounds"]["max_notional_per_order_usd"]): + violations.append(("observed_fill_notional_bound_breach", ["C3" if e["side"] == "BUY" else "C4"])) + for reason, cases in violations: + if not any(f["reason"] == reason and f["cases_blocked"] == cases for f in receipt["failures"]): + receipt["failures"].append({"reason": reason, "cases_blocked": cases, "acceptance_step": 2}) + + +def orders_in_flight(orders): + """Upstream close-on-stop runs once; wait for entry resolution before stop.""" + terminal = {"FILLED", "CANCELED", "EXPIRED", "REJECTED", "DENIED"} + return any(order.status.name not in terminal and + (order.order_type.name == "MARKET" or order.time_in_force.name in ("IOC", "FOK") + or order.status.name == "SUBMITTED" or order.venue_order_id is None) for order in orders) + + +def stop_ready(orders, now, node_stop_at): + return now >= node_stop_at or not orders_in_flight(orders) + + +async def node_phase(plan, port, receipt_path, payload): + """Hosted upstream run: cache + thread-safe handle captured before run_async. + + Source: crates/live/src/python/node.rs at UPSTREAM_COMMIT. The parent owns + the hard deadline; OS process termination cannot strand this awaitable. + """ + if not validate_child_admission(plan, payload): + return persist_child_refusal(receipt_path, payload, "refused_child_quote_admission") + receipt, account_id = payload["receipt"], payload["account_id"] + node = build_node(plan, port, account_id, tob_offset_ticks=payload["tob_offset_ticks"]) + # Live view: node.rs py_cache uses PyCache::from_rc(kernel.cache()); + # common/src/python/cache.rs PyCache(Rc>) borrows on each + # orders() call. Individual returned orders are clones; re-read each poll. + cache, handle = node.cache, node.handle() + loop = asyncio.get_running_loop() + pending_stop = None + registered_signals = [] + task = None + scope = RunOrderScope(time.time_ns()) + + def observe(): + update_observations(receipt, snapshot_events(cache, scope), plan) + receipt["reconciled_external"] = snapshot_events(cache, scope, external=True) + + def stop(reason): + if receipt["node"]["stop_ns"] is None: + receipt["node"].update(stop_ns=time.time_ns(), stop_reason=reason) + handle.stop() + + def request_stop(reason): + nonlocal pending_stop + pending_stop = pending_stop or reason + + def record_error(step, exc): + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), + "reason": frozen().redact(f"{step}: {type(exc).__name__}: {exc}")}) + + observed_start = None + try: + for sig in SIGNALS: + loop.add_signal_handler(sig, request_stop, f"signal_{sig.name}") + registered_signals.append(sig) + task = asyncio.create_task(node.run_async()) + try: + while not task.done(): + if handle.is_running: + receipt["node"]["started"] = True + observed_start = observed_start or time.monotonic() + observe() + cases = receipt["cases"] + now = time.monotonic() + if receipt["failures"] or any(c["outcome"] == "failed" for c in cases.values()): + request_stop("case_failure") + elif cases["C1"]["outcome"] == cases["C3"]["outcome"] == "passed": + request_stop("C1_accepted_C3_filled") + elif observed_start is not None and now - observed_start >= plan["timeouts"]["per_step_seconds"]: + request_stop("case_timeout") + if now >= payload["node_stop_at"]: + stop("deadline") + elif pending_stop: + if stop_ready(tester_orders(cache, scope), now, payload["node_stop_at"]): + stop(pending_stop) + write_receipt(receipt_path, receipt, (account_id,)) + await asyncio.sleep(0.1) + except Exception as exc: + record_error("node_poll", exc) + # An accepted MARKET IOC can still fill; preserve close-on-stop's + # one opportunity to see its position even if observation failed. + while not task.done() and time.monotonic() < payload["node_stop_at"]: + try: + if stop_ready(tester_orders(cache, scope), time.monotonic(), payload["node_stop_at"]): + break + except Exception: + pass # Unknown state waits only to the bounded deadline. + await asyncio.sleep(0.1) + stop("poll_exception") + await task # Native cancel/close runs even after an observer exception. + except Exception as exc: + record_error("node_run", exc) + stop("node_exception") + if task is not None and not task.done(): + try: + await task + except Exception as task_exc: + record_error("node_shutdown", task_exc) + finally: + try: + observe() + except Exception as exc: + record_error("final_snapshot", exc) + try: + write_receipt(receipt_path, receipt, (account_id,)) + except Exception as exc: + record_error("final_receipt", exc) + for sig in registered_signals: + try: + loop.remove_signal_handler(sig) + except Exception as exc: + record_error("signal_removal", exc) + try: + node.dispose() + except Exception as exc: + record_error("node_dispose", exc) + # Persist cleanup errors independently, without skipping disposal. + write_receipt(receipt_path, receipt, (account_id,)) + return receipt + + +class CheckTimeout(TimeoutError): + pass + + +def official_check(plan, port, *, with_session, deadline): + """Bound frozen ibapi connect/handshake as well as its request waits.""" + seconds = min(plan["timeouts"]["check_seconds"], max(0, deadline - time.monotonic())) + if seconds <= 0: + return {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}}, None + + def expired(_sig, _frame): + raise CheckTimeout("official_check_deadline") + + previous = signal.signal(signal.SIGALRM, expired) + signal.setitimer(signal.ITIMER_REAL, seconds) + try: + return frozen().run_check(plan, port, with_session=with_session, deadline_s=seconds) + except Exception as exc: + return {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}, + "error": frozen().redact(f"{type(exc).__name__}: {exc}")}, None + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, previous) + + +def official_admission(plan, port, *, deadline, client_factory=None): + """Flat pre-check and quote admission share one official client-92 session.""" + seconds = min(plan["timeouts"]["check_seconds"], max(0, deadline - time.monotonic())) + client = (client_factory or build_admission_client)() + session = CheckSession(client) + check_deadline = time.monotonic() + seconds + pre = {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}} + account_id = None + phase = "pre_check" + pre_timed_out = False + + def failure(exc): + status = "not_connected" + if "tick requested" in client.stage_elapsed_seconds: + status = "refused_quote_clock" if "quote" in client.stage_elapsed_seconds else "refused_quote_unavailable" + if client.delayed: + status = "refused_delayed_quote" + elif client.errors and status != "not_connected": + status = "refused_quote_error" + return {"status": status, "error": frozen().redact(exc), **client.diagnostics()} + + if seconds <= 0: + pre["error"] = "CheckTimeout: pre_check_deadline" + return pre, None, {"status": "not_run", **client.diagnostics()} + + def expired(_sig, _frame): + nonlocal pre_timed_out + pre_timed_out = phase == "pre_check" + raise CheckTimeout(f"{phase}_deadline") + + previous = signal.signal(signal.SIGALRM, expired) + signal.setitimer(signal.ITIMER_REAL, seconds) + try: + pre, account_id = frozen().run_check(plan, port, with_session=True, + client_factory=lambda: session, deadline_s=seconds) + pre["observed"] = copy.deepcopy(pre.get("observed", {})) + # ibapi.connect / frozen run_check can catch TimeoutError as OSError. + # Preserve the alarm's phase even when that path returns normally. + if pre_timed_out: + pre.update(status="incomplete", error="CheckTimeout: pre_check_deadline") + return pre, None, {"status": "not_run", **client.diagnostics()} + phase = "quote_check" + if pre["status"] != "passed" or not account_id: + return pre, account_id, {"status": "not_run", **client.diagnostics()} + admitted = quote_check(plan, account_id, deadline=check_deadline, client=client) + return pre, account_id, admitted + except Exception as exc: + if phase == "pre_check": + pre.update(status="incomplete", observed=copy.deepcopy(client.r), requests_completed=client.completed(), + error=frozen().redact(f"pre_check: {type(exc).__name__}: {exc}")) + return pre, None, {"status": "not_run", **client.diagnostics()} + return pre, account_id, failure(exc) + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, previous) + session.close(check_deadline) + + +def final_status(receipt, *, interrupted=False, child_returncode=0): + """Flat-proof failure dominates; observed/case/loss breaches remain failed.""" + if receipt["flat_proof"]["status"] != "passed": + return "cleanup_required", "independent_flat_proof", 4, ("C4",) + child_refusal = receipt["node"].get("pre_node_refusal") + if not receipt["node"]["started"] and child_refusal: + return child_refusal, child_refusal, 2, CASE_IDS + rt = receipt["roundtrip"] + if (any(f["reason"].startswith("observed_") for f in receipt["failures"]) + or any(c["outcome"] == "failed" for c in receipt["cases"].values()) + or rt and rt["loss_bound_breach"]): + return "failed", "case_or_loss_bound", 2, CASE_IDS + if interrupted or receipt["failures"] or child_returncode != 0: + return "incomplete", "signal_or_node_failure", 3, CASE_IDS + if all(c["outcome"] == "passed" for c in receipt["cases"].values()) and rt and rt["closed"] and not rt["commission_unresolved"]: + return "passed", None, 2, CASE_IDS + return "incomplete", "cases_or_commissions_unresolved", 3, CASE_IDS + + +def run_trial(args, *, now=None, versions=None): + """Every admission refusal precedes official-check and node construction.""" + started_monotonic = time.monotonic() + try: + plan = load_plan(args.plan) + except (ValueError, OSError): + plan = {} + versions = runtime_versions() if versions is None else versions + receipt = new_receipt(plan, args.plan, versions) + errors = validate_plan(plan, port=args.port, node_client_id=args.node_client_id, check_client_id=args.check_client_id) + status = "refused_plan" if errors else runtime_refusal(versions) + now = now or datetime.now(timezone.utc) + if status is None: + inside, window = frozen().rth_check(now, plan) + receipt["window"] = window + if not inside: + status = "refused_outside_window" + if status: + finish(receipt, status, reason="; ".join(errors) if errors else status) + if args.receipt: + write_receipt(args.receipt, receipt) + return receipt + if args.plan_only: + finish(receipt, "passed") + receipt["evidence_class"] = "structural_validation" + if args.receipt: + write_receipt(args.receipt, receipt) + return receipt + if args.receipt is None: + finish(receipt, "refused_receipt_required", reason="receipt required before node") + return receipt + # Command-center r2 decision: runner quantity + quote admission hold the + # bound; the engine cap remains configured with its route limitation stated. + deadline = started_monotonic + plan["timeouts"]["overall_deadline_seconds"] + port = args.port if args.port is not None else plan["default_port"] + pre, account_id, admitted = official_admission(plan, port, deadline=deadline) + liquid = pre.pop("liquid_hours", "") + pre.pop("trading_hours", None) + zone = pre.pop("time_zone_id", "America/New_York") + receipt["pre_check"] = pre + receipt["quote_admission"] = admitted + if pre["status"] != "passed" or not account_id: + finish(receipt, pre["status"] if pre["status"] != "passed" else "refused_account_scope", reason="pre_check") + write_receipt(args.receipt, receipt, (account_id,)) + return receipt + sessions = frozen().parse_liquid_hours(liquid, zone, now.astimezone(frozen().ZoneInfo("America/New_York")).date()) + if sessions is None or not frozen().rth_check(datetime.now(timezone.utc), plan, sessions, horizon_s=deadline - time.monotonic())[0]: + finish(receipt, "refused_liquid_hours", reason="whole_run_broker_window") + write_receipt(args.receipt, receipt, (account_id,)) + return receipt + if admitted["status"] != "passed": + finish(receipt, admitted["status"], reason="quote_admission") + write_receipt(args.receipt, receipt, (account_id,)) + return receipt + # Atomic provisional cleanup_required receipt BEFORE creating the child/node. + write_receipt(args.receipt, receipt, (account_id,)) + child = None + interrupted = [] + old_handlers = {} + + def signaled(sig, _frame): + interrupted.append(signal.Signals(sig).name) + if child is not None and child.poll() is None: + child.send_signal(sig) + + try: + for sig in SIGNALS: + old_handlers[sig] = signal.signal(sig, signaled) + stop_at = deadline - plan["timeouts"]["post_check_reserve_seconds"] - plan["timeouts"]["cleanup_seconds"] - plan["timeouts"]["node_stop_allowance_seconds"] + command = [sys.executable, str(Path(__file__).resolve()), "--_node-phase", "--plan", str(Path(args.plan).resolve()), + "--port", str(port), "--receipt", str(Path(args.receipt).resolve())] + child_env = {key: value for key, value in os.environ.items() + if key not in ("RUST_LOG", "NAUTILUS_LOG", "TWS_ACCOUNT")} + child = subprocess.Popen(command, stdin=subprocess.PIPE, text=True, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=child_env) + # Anonymous pipe only: no account in argv, environment, file or receipt. + child.stdin.write(json.dumps({"account_id": account_id, "receipt": receipt, "node_stop_at": stop_at, + "quote_admission": admitted, "tob_offset_ticks": admitted["tob_offset_ticks"]})) + child.stdin.close() + try: + child.wait(timeout=max(0, stop_at - time.monotonic())) + except subprocess.TimeoutExpired: + child.send_signal(signal.SIGTERM) + child.wait(timeout=max(0, deadline - plan["timeouts"]["post_check_reserve_seconds"] - time.monotonic())) + except subprocess.TimeoutExpired: + child.kill() + child.wait() + receipt["failures"].append({"acceptance_step": 4, "cases_blocked": ["C2", "C4"], "reason": "node_hard_deadline"}) + except Exception as exc: + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), "reason": frozen().redact(exc)}) + if child is not None and child.poll() is None: + child.send_signal(signal.SIGTERM) + try: + child.wait(timeout=plan["timeouts"]["node_stop_allowance_seconds"]) + except subprocess.TimeoutExpired: + child.kill() + child.wait() + finally: + for sig, handler in old_handlers.items(): + signal.signal(sig, handler) + saved_failures = receipt["failures"] + try: + receipt = json.loads(Path(args.receipt).read_text()) + receipt["failures"].extend(saved_failures) + except (OSError, ValueError): + pass # Preserve provisional state; a corrupt receipt can never pass. + proof, proof_account = official_check(plan, port, with_session=False, deadline=deadline) + if proof["status"] == "passed" and proof_account != account_id: + proof["status"] = "refused_changed_account" + receipt["flat_proof"] = proof + update_observations(receipt, receipt["events"], plan) + status, reason, step, cases = final_status(receipt, interrupted=bool(interrupted), + child_returncode=child.returncode if child is not None else None) + finish(receipt, status, reason=reason, step=step, cases=cases) + write_receipt(args.receipt, receipt, (account_id, proof_account)) + return receipt + + +class PaperArgumentParser(argparse.ArgumentParser): + def error(self, message): + self.exit(3, "refused_arguments: " + frozen().redact(message) + "\n") + + +def parser(): + result = PaperArgumentParser(description=__doc__) + result.add_argument("--plan", type=Path, default=PLAN_PATH) + result.add_argument("--receipt", type=Path, help="sanitized, atomically replaced receipt; required for an order run") + result.add_argument("--plan-only", action="store_true", help="validate plan, pins and window WITHOUT connecting; quote admission is not exercised") + result.add_argument("--port", type=int, help="4002 (default) or 7497 only") + result.add_argument("--node-client-id", type=int, help="91 only") + result.add_argument("--check-client-id", type=int, help="92 only") + result.add_argument("--_node-phase", action="store_true", help=argparse.SUPPRESS) + return result + + +def persist_child_refusal(path, payload, reason): + """Keep a sanitized pre-node reason even though child output is discarded.""" + receipt = payload.get("receipt") if isinstance(payload, dict) else None + if not isinstance(receipt, dict): + try: + receipt = json.loads(Path(path).read_text()) + except (OSError, TypeError, ValueError): + receipt = new_receipt(load_plan(), PLAN_PATH, runtime_versions()) + receipt["node"]["pre_node_refusal"] = reason + finish(receipt, reason, reason=reason) + if path is not None: + account = payload.get("account_id") if isinstance(payload, dict) else None + write_receipt(path, receipt, (account,)) + return receipt + + +def run_child(args): + try: + payload = json.load(sys.stdin) + if not isinstance(payload, dict): + raise ValueError("invalid child payload") + except (ValueError, OSError): + persist_child_refusal(args.receipt, {}, "refused_child_payload") + return 3 + try: + plan = load_plan(args.plan) + reason = "refused_child_plan" if validate_plan(plan, port=args.port) else None + except (ValueError, OSError): + reason = "refused_child_plan" + if reason is None and runtime_refusal(runtime_versions()): + reason = "refused_child_runtime" + if reason is None: + try: + inside = frozen().rth_check(datetime.now(timezone.utc), plan, + horizon_s=max(0, payload["node_stop_at"] - time.monotonic()))[0] + if not inside: + reason = "refused_child_window" + except (KeyError, TypeError, ValueError): + reason = "refused_child_payload" + if reason is None and not validate_child_admission(plan, payload): + reason = "refused_child_quote_admission" + if reason: + persist_child_refusal(args.receipt, payload, reason) + return 3 + receipt = asyncio.run(node_phase(plan, args.port, args.receipt, payload)) + if receipt["node"].get("pre_node_refusal"): + return 3 + return 1 if receipt["failures"] else 0 + + +def main(argv=None): + args = parser().parse_args(argv) + try: + if args._node_phase: + # The pipe supplies the admitted quote and derived offset; validate + # before node creation. This entry never loads an account from env. + return run_child(args) + receipt = run_trial(args) + except Exception as exc: + print(frozen().scrub_serialized(json.dumps({"status": "incomplete", "exit_code": 1, "error": frozen().redact(exc)}))) + return 1 + print(frozen().scrub_serialized(json.dumps({"status": receipt["status"], "exit_code": receipt["exit_code"], "window": receipt.get("window"), + "failures": sanitize(receipt["failures"])}))) + return receipt["exit_code"] + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.67429251370f b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.67429251370f new file mode 100644 index 000000000..ce6ea6fe3 --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.67429251370f @@ -0,0 +1,1169 @@ +#!/usr/bin/env python3 +"""Bounded rc5 paper trial using upstream's built-in ExecTester. + +The rc5 engine cap is configured; quote admission and quantity one hold this +trial's runner bound on the SMART/IB route affected by #4946. --plan-only and +--help never connect. Synthetic tests do not establish broker acceptance. +""" +from __future__ import annotations + +import argparse +import asyncio +import hashlib +import importlib.metadata +import importlib.util +import json +import os +import platform +import signal +import subprocess +import sys +import tempfile +import threading +import time +from datetime import datetime, timezone +from decimal import Decimal, InvalidOperation, ROUND_FLOOR +from functools import lru_cache +from pathlib import Path + +HERE = Path(__file__).resolve().parent +PLAN_PATH = HERE / "plan.json" +FROZEN_RUN = HERE.parent / "ibkr-paper-orders" / "run.py" +KIND = "ibkr_paper_orders_nautilus_2_0_0rc5" +UPSTREAM_COMMIT = "1b0a49d2792a9432a3aca3fcb617ce7a630d905e" +SOURCE = f"https://github.com/nautechsystems/nautilus_trader/blob/{UPSTREAM_COMMIT}" +CASE_IDS = ("C1", "C2", "C3", "C4") +SIGNALS = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP) +TERMINAL_FAILURES = {"OrderDenied", "OrderRejected", "OrderExpired", "OrderCancelRejected"} +TESTER_SETTINGS = { + "order_qty": "1", "open_position_on_start_qty": "1", "open_position_on_first_quote": True, + "open_position_time_in_force": "IOC", "enable_limit_buys": True, "enable_limit_sells": False, + "enable_stop_buys": False, "enable_stop_sells": False, "enable_brackets": False, + "tob_offset_ticks": "floor(0.5 * admitted_bid / 0.01)", "limit_time_in_force": "GTD", "order_expire_time_delta_mins": 7, + "modify_orders_to_maintain_tob_offset": False, "cancel_replace_orders_to_maintain_tob_offset": False, + "use_post_only": False, "use_individual_cancels_on_stop": True, + "cancel_orders_on_stop": True, "close_positions_on_stop": True, + "reduce_only_on_stop": False, "dry_run": False, "log_data": False, +} +CASE_DEFINITIONS = [ + {"id": "C1", "name": "accept_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderAccepted"}, + {"id": "C2", "name": "cancel_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderCanceled for C1 after stop"}, + {"id": "C3", "name": "fill_buy", "order_type": "MARKET", "side": "BUY", "expect": "OrderFilled quantity 1"}, + {"id": "C4", "name": "flatten", "order_type": "MARKET", "side": "SELL", "expect": "OrderFilled quantity 1 after stop and independent flat proof"}, +] + +# Original sources, read at v2.0.0rc5 (UPSTREAM_COMMIT): +# examples/live/interactive_brokers/exec_tester.py: builder + ExecTester config. +# examples/live/interactive_brokers/_common.py: bounded node timeouts. +# crates/testkit/src/testers/exec/strategy.rs: native entry/cancel/close paths. +# crates/risk/src/engine/mod.rs: check_orders_risk_for_account returns true +# when account_for_venue(SMART) is absent, before checking max_notional. The +# IB account is registered under IB. Config support is not enforcement. +RISK_NOTE = "engine notional cap configured, not enforced on this route (#4946, fixed on develop, unreleased); bound held by qty=1 and quote admission" +RISK_ROUTE = { + "issue": "https://github.com/nautechsystems/nautilus_trader/issues/4946", + "issue_state": "closed", + "pin_fix_present": False, + "upstream_fix": "fixed on develop; included in v2.0.0rc6; absent from the selected rc5 pin", + "source": SOURCE + "/crates/risk/src/engine/mod.rs#L1218", + "reason": "rc5 stock orders can skip account-scoped max_notional_per_order on the SMART/IB venue path", +} +RELEASE_VERIFICATION = { + "checked_at": "2026-10-05", + "fix_commit": "ed6fc8bf47fd37dda97d63b9b2df160719ee2bac", + "released_in": "v2.0.0rc6", + "published_at": "2026-10-05T03:03:18Z", + "source": "https://github.com/nautechsystems/nautilus_trader/releases/tag/v2.0.0rc6", + "correction": "The requested note's unreleased clause reflects earlier metadata; the fix is now released in rc6, while this trial stays pinned to rc5.", +} +BLOCKED_STEPS = [ + {"acceptance_step": 3, "operation": "reconnect_with_open_order", "exercised": False, + "issues": [{"number": 5057, "state": "open"}, {"number": 5060, "state": "open"}]}, + {"acceptance_step": 3, "operation": "restart_reconciliation", "exercised": False, + "issues": [{"number": 5007, "state": "open"}, {"number": 5057, "state": "open"}, + {"number": 5060, "state": "open"}]}, + {"acceptance_step": 4, "operation": "kill_switch", "exercised": False, + "issues": [{"number": 5060, "state": "open"}]}, +] +for _step in BLOCKED_STEPS: + _step["issue_states_checked_at"] = "2026-10-05" + for _issue in _step["issues"]: + _issue["url"] = f"https://github.com/nautechsystems/nautilus_trader/issues/{_issue['number']}" + + +@lru_cache(maxsize=1) +def frozen(): + """Reuse the frozen official-ibapi checker, window parser and redaction. + + Source: ../ibkr-paper-orders/run.py, bound by source_hashes in the receipt. + No frozen strategy is used; the node uses rc5's built-in ExecTester. + """ + spec = importlib.util.spec_from_file_location("ibkr_frozen_readonly", FROZEN_RUN) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def sha256(path): + return hashlib.sha256(Path(path).read_bytes()).hexdigest() + + +def decimal(value): + result = Decimal(str(value)) + if not result.is_finite(): + raise ValueError("nonfinite numeric input") + return result + + +def load_plan(path=PLAN_PATH): + def unique(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError("duplicate plan key") + result[key] = value + return result + return json.loads(Path(path).read_text(), object_pairs_hook=unique) + + +def validate_plan(plan, *, port=None, node_client_id=None, check_client_id=None): + """All static bounds are checked without importing a client or connecting.""" + errors = [] + + def need(condition, reason): + if not condition: + errors.append(reason) + + try: + need(type(plan["schema_version"]) is int and plan["schema_version"] == 1 + and plan["kind"] == "ibkr_paper_orders_plan_rc5", "plan schema") + need(plan["engine"]["version"] == "2.0.0rc5" and plan["engine"]["ibapi_version"] == "10.45.1", "runtime pins") + need(plan["engine"]["upstream_commit"] == UPSTREAM_COMMIT, "upstream pin") + need(plan["host"] == "127.0.0.1", "loopback host required") + ports = plan["paper_ports"] + need(isinstance(ports, list) and bool(ports) and all(type(p) is int and p in (4002, 7497) for p in ports), "paper ports") + need(plan["default_port"] in ports and type(plan["default_port"]) is int, "default paper port") + need(port is None or type(port) is int and port in ports and port in (4002, 7497), "selected paper port") + need(plan["account_prefix"] == "DU" and plan["require_single_managed_account"] is True, "single DU account") + need(plan["client_ids"] == {"node": 91, "check": 92} + and all(type(v) is int for v in plan["client_ids"].values()), "client ids 91/92") + need(node_client_id is None or type(node_client_id) is int and node_client_id == 91, "node client id 91") + need(check_client_id is None or type(check_client_id) is int and check_client_id == 92, "check client id 92") + inst = plan["instrument"] + need(tuple(inst[k] for k in ("symbol", "sec_type", "exchange", "primary_exchange", "currency")) + == ("SPY", "STK", "SMART", "ARCA", "USD"), "SPY STK SMART/ARCA USD only") + need(inst["nautilus_instrument_id"] == "SPY=STK.SMART" and inst["expected_price_increment"] == "0.01", "RAW stock identity and tick") + b = plan["bounds"] + need(type(b["max_orders"]) is int and 3 <= b["max_orders"] <= 6, "order budget 3..6") + need(type(b["max_quantity_per_order"]) is int and b["max_quantity_per_order"] == 1, "quantity 1") + for name, ceiling in (("max_notional_per_order_usd", 1000), ("max_roundtrip_loss_usd", 5)): + need(not isinstance(b[name], bool) and 0 < decimal(b[name]) <= ceiling, name) + need(decimal(b["commission_allowance_per_order_usd"]) >= 0, "commission allowance") + s = plan["session"] + need((s["timezone"], s["open"], s["close"]) == ("America/New_York", "09:30", "16:00"), "regular trading hours") + need(s["weekdays_only"] is True and s["whole_run_inside_window"] is True + and s["use_contract_liquid_hours"] is True, "whole run with broker liquid hours") + need(type(s["close_buffer_minutes"]) is int and s["close_buffer_minutes"] >= 10, "10 minute close buffer") + expected_times = {"check_seconds": 30, "node_start_seconds": 60, "per_step_seconds": 45, + "cleanup_seconds": 45, "node_stop_allowance_seconds": 15, + "post_check_reserve_seconds": 60, "overall_deadline_seconds": 420} + need(plan["timeouts"] == expected_times and all(type(v) is int for v in plan["timeouts"].values()), + "frozen timeouts and hard deadline") + r = plan["risk"] + need(r["bypass"] is False and r["require_enforced_notional_before_connection"] is False, + "native risk configured with runner quote admission") + need(not isinstance(r["notional_headroom_usd"], bool) + and decimal(r["notional_headroom_usd"]) == 10, "USD 10 quote admission headroom") + need(r["max_order_submit_rate"] == f"{b['max_orders']}/00:07:00", "whole-run native submit throttle") + # No configurable strategy knobs can quietly turn this into another trial. + need(plan["exec_tester"] == TESTER_SETTINGS + and all(type(plan["exec_tester"][k]) is type(v) for k, v in TESTER_SETTINGS.items()), + "frozen ExecTester settings") + need(plan["data"] == {"market_data_type": "REALTIME", "quote_max_age_seconds": 10, + "clock_resolution_tolerance_seconds": 2, "batch_quotes": False}, "real-time quote bounds") + need(plan["cases"] == CASE_DEFINITIONS, "frozen cases C1..C4") + need(plan["end_disposition"] == {"positions": 0, "open_orders": 0}, "flat disposition") + need(plan["receipt"]["kind"] == KIND and plan["evidence_class"] == "native_paper", "receipt identity") + except (KeyError, TypeError, ValueError, InvalidOperation): + errors.append("missing or malformed plan field") + return errors + + +def runtime_versions(): + def version(package): + try: + return importlib.metadata.version(package) + except importlib.metadata.PackageNotFoundError: + return None + try: + import ibapi + ib_version = ibapi.__version__ + except (ImportError, AttributeError): + ib_version = None + try: + import nautilus_trader + engine_version = nautilus_trader.__version__ + except (ImportError, AttributeError): + engine_version = None + return {"nautilus_trader": engine_version, "nautilus_trader_distribution": version("nautilus_trader"), "ibapi": ib_version, + "nautilus_ibapi_distribution": version("nautilus-ibapi"), + "protobuf": version("protobuf"), "python": platform.python_version()} + + +def runtime_refusal(versions): + if versions.get("nautilus_trader") != "2.0.0rc5" or versions.get("ibapi") != "10.45.1": + return "refused_unpinned_runtime" + if "nautilus_trader_distribution" in versions and versions["nautilus_trader_distribution"] != "2.0.0rc5": + return "refused_unpinned_runtime" + return None + + +def resting_offset_ticks(bid, plan): + """Frozen half-bid placement through upstream ExecTester's supported offset.""" + tick = decimal(plan["instrument"]["expected_price_increment"]) + return int((decimal(bid) * Decimal("0.5") / tick).to_integral_value(rounding=ROUND_FLOOR)) + + +def admit_quote(plan, quote, server_time_epoch, *, elapsed_seconds=0, delayed=False, errors=()): + """Pure admission using IB's timestamped BidAsk and its independent clock.""" + result = {"source": "ibapi.reqTickByTickData.BidAsk/reqCurrentTime", "status": None, + "bid": None, "ask": None, "quote_age_seconds": None} + if delayed or any(code in (10089, 10167) for code in errors): + result["status"] = "refused_delayed_quote" + return result + if errors: + result["status"] = "refused_quote_error" + return result + try: + bid, ask = decimal(quote["bid"]), decimal(quote["ask"]) + age = decimal(server_time_epoch) - decimal(quote["quote_time_epoch"]) + decimal(elapsed_seconds) + result.update(bid=str(bid), ask=str(ask), quote_time_epoch=int(quote["quote_time_epoch"]), + server_time_epoch=int(server_time_epoch), quote_age_seconds=float(max(age, Decimal(0)))) + if bid <= 0 or ask < bid: + result["status"] = "refused_invalid_quote" + elif age < -decimal(plan["data"]["clock_resolution_tolerance_seconds"]) or age > decimal(plan["data"]["quote_max_age_seconds"]): + result["status"] = "refused_stale_quote" + else: + notional = ask * plan["bounds"]["max_quantity_per_order"] + headroom = decimal(plan["risk"]["notional_headroom_usd"]) + result.update(ask_notional_usd=str(notional), headroom_usd=str(headroom), + tob_offset_ticks=resting_offset_ticks(bid, plan)) + result["status"] = "passed" if notional + headroom <= decimal(plan["bounds"]["max_notional_per_order_usd"]) and result["tob_offset_ticks"] > 0 else "refused_quote_notional" + except (KeyError, TypeError, ValueError, InvalidOperation): + result["status"] = "refused_invalid_quote" + return result + + +# Official ibapi 10.45.1 client.py reqTickByTickData and wrapper.py callbacks +# were inspected in the prepared runtime. Delayed enums come from ticktype.py. +DELAYED_TICKS = set(range(66, 77)) | {80, 81, 82, 83, 88, 90, 103, 104} +CONNECTIVITY_CODES = {326, 502, 503, 504, 507, 1100, 1300, 2110} + + +class QuoteState: + """Read-only callback state, kept independent of ibapi for synthetic tests.""" + + def __init__(self): + self.ready = threading.Event() + self.quote_ready = threading.Event() + self.clock_ready = threading.Event() + self.accounts_ready = threading.Event() + self.accounts = [] # Memory only. + self.quote = None + self.server_time = None + self.clock_received = None + self.delayed = False + self.errors = [] + self.info = [] + self.started_monotonic = time.monotonic() + self.last_stage = "initializing" + self.stage_elapsed_seconds = {} + self.stage_duration_seconds = {} + self.stage_started = self.started_monotonic + self.quote_phase = False + + def mark(self, stage): + if stage not in self.stage_elapsed_seconds: + now = time.monotonic() + self.stage_elapsed_seconds[stage] = round(now - self.started_monotonic, 4) + self.stage_duration_seconds[stage] = round(now - self.stage_started, 4) + self.stage_started, self.last_stage = now, stage + + def diagnostics(self): + return {"last_stage": self.last_stage, "elapsed_seconds": round(time.monotonic() - self.started_monotonic, 4), + "last_stage_elapsed_seconds": round(time.monotonic() - self.stage_started, 4), + "stage_elapsed_seconds": dict(self.stage_elapsed_seconds), + "stage_duration_seconds": dict(self.stage_duration_seconds), + "errors": list(self.errors), "info": list(self.info)} + + def nextValidId(self, orderId): + self.mark("nextValidId") + self.ready.set() + + def managedAccounts(self, accountsList): + self.accounts = [value.strip() for value in (accountsList or "").split(",") if value.strip()] + self.mark("accounts") + self.accounts_ready.set() + + def tickByTickBidAsk(self, reqId, timestamp, bidPrice, askPrice, bidSize, askSize, tickAttribBidAsk): + if reqId == 9202: + self.quote = {"bid": str(bidPrice), "ask": str(askPrice), "quote_time_epoch": timestamp} + self.mark("quote") + self.quote_ready.set() + + def currentTime(self, timestamp): + self.server_time, self.clock_received = timestamp, time.monotonic() + if self.quote_phase: + self.mark("clock") + self.clock_ready.set() + + def error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson=""): + if errorCode in (10089, 10167): + self.delayed = True + entry = {"reqId": reqId, "code": errorCode, + "text": frozen().scrub_serialized(frozen().redact(errorString)[:160], self.accounts)} + fatal = reqId == 9202 or errorCode in CONNECTIVITY_CODES or errorCode in (10089, 10167) + (self.errors if fatal else self.info).append(entry) + if fatal: + self.quote_ready.set() + self.clock_ready.set() + if errorCode in CONNECTIVITY_CODES: + self.ready.set() + self.accounts_ready.set() + + def marketDataType(self, reqId, marketDataType): + if marketDataType in (3, 4): + self.delayed = True + self.quote_ready.set() + + def delayed_tick(self, tickType): + if tickType in DELAYED_TICKS: + self.delayed = True + self.quote_ready.set() + + def tickPrice(self, reqId, tickType, price, attrib): + self.delayed_tick(tickType) + + def tickSize(self, reqId, tickType, size): + self.delayed_tick(tickType) + + def tickString(self, reqId, tickType, value): + self.delayed_tick(tickType) + + def tickGeneric(self, reqId, tickType, value): + self.delayed_tick(tickType) + + def tickOptionComputation(self, reqId, tickType, *values): + self.delayed_tick(tickType) + + +class AdmissionState(frozen().CheckState, QuoteState): + """Reuse frozen flat-check callbacks and quote callbacks on one session.""" + + def __init__(self): + frozen().CheckState.__init__(self) + QuoteState.__init__(self) + + def managedAccounts(self, accountsList): + frozen().CheckState.managedAccounts(self, accountsList) + QuoteState.managedAccounts(self, accountsList) + + def currentTime(self, timestamp): + frozen().CheckState.currentTime(self, timestamp) + QuoteState.currentTime(self, timestamp) + + def error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson=""): + frozen().CheckState.error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson) + QuoteState.error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson) + + +def build_admission_client(): + from ibapi.client import EClient + from ibapi.wrapper import EWrapper + + class QuoteClient(AdmissionState, EWrapper, EClient): + def __init__(self): + AdmissionState.__init__(self) + EClient.__init__(self, self) + + return QuoteClient() + + +class CheckSession: + """Hold the frozen check's session until quote admission and join its reader. + + Official ibapi 10.45.1 client.py: connect creates EReader and startApi; + run drains the shared message queue; disconnect closes/reset but does not + join those threads. Intercept only the frozen harness's final disconnect, + leaving EClient's own disconnect on connection loss unchanged. + """ + + def __init__(self, client): + self.client, self.runner = client, None + + def __getattr__(self, name): + return getattr(self.client, name) + + def connect(self, host, port, client_id): + self.client.connect(host, port, client_id) + if self.client.isConnected(): + self.client.mark("connected") + + def run(self): + self.runner = threading.current_thread() + self.client.run() + + def disconnect(self): + pass # The outer admission scope owns teardown after both read phases. + + def close(self, deadline): + reader = getattr(self.client, "reader", None) + try: + self.client.disconnect() + finally: + for thread in (self.runner, reader): + if thread is not None and thread is not threading.current_thread(): + thread.join(timeout=max(0, deadline - time.monotonic())) + + +def quote_check(plan, account_id, *, deadline, client): + """Continue the already-connected flat-check client; never reconnect.""" + requested = False + + def wait(event): + while not event.is_set() and time.monotonic() < deadline: + if client.errors or not client.isConnected(): + return False + event.wait(min(0.05, max(0, deadline - time.monotonic()))) + return event.is_set() + + def result(value): + return {**value, **client.diagnostics()} + + def failed_wait(status): + if client.errors or client.delayed: + return result(admit_quote(plan, client.quote, client.server_time, + delayed=client.delayed, errors=[entry["code"] for entry in client.errors])) + return result({"status": status, "error": "quote_check_deadline" if time.monotonic() >= deadline else "connection_lost"}) + + try: + if not client.isConnected(): + return result({"status": "not_connected"}) + if not wait(client.ready) or not wait(client.accounts_ready): + return failed_wait("not_connected") + if client.errors: + return failed_wait("not_connected") + if client.accounts != [account_id] or not account_id.startswith("DU"): + return result({"status": "refused_quote_account_scope"}) + client.quote_phase = True + client.reqMarketDataType(1) + requested = True + client.mark("tick requested") + client.reqTickByTickData(9202, frozen().spy_contract(), "BidAsk", 0, False) + if not wait(client.quote_ready): + return failed_wait("refused_quote_unavailable") + if client.delayed or client.errors: + return failed_wait("refused_quote_error") + client.clock_ready.clear() # Discard the clock from the flat pre-check. + client.server_time, client.clock_received = None, None + client.reqCurrentTime() + if not wait(client.clock_ready): + return failed_wait("refused_quote_clock") + elapsed = max(0, time.monotonic() - client.clock_received) if client.clock_received is not None else 0 + admitted = admit_quote(plan, client.quote, client.server_time, elapsed_seconds=elapsed, + delayed=client.delayed, errors=[entry["code"] for entry in client.errors]) + admitted["admitted_monotonic"] = time.monotonic() + return result(admitted) + finally: + if requested: + client.cancelTickByTickData(9202) + + +def validate_child_admission(plan, payload, *, monotonic_now=None): + """Recompute age, cap and offset from the pipe; no account in diagnostics.""" + try: + admitted = payload["quote_admission"] + now = time.monotonic() if monotonic_now is None else monotonic_now + elapsed = decimal(now) - decimal(admitted["admitted_monotonic"]) + if admitted["status"] != "passed" or elapsed < 0: + return False + # Stored age already includes time since the independent server callback. + rechecked = admit_quote(plan, admitted, decimal(admitted["quote_time_epoch"]) + decimal(admitted["quote_age_seconds"]), + elapsed_seconds=elapsed) + offset = payload["tob_offset_ticks"] + return (rechecked["status"] == "passed" and type(offset) is int + and offset == resting_offset_ticks(admitted["bid"], plan) + and offset == admitted["tob_offset_ticks"] + and isinstance(payload["account_id"], str) and payload["account_id"].startswith("DU")) + except (KeyError, TypeError, ValueError, InvalidOperation): + return False + + +def sanitize(value): + """Prices/quantities remain structured; all untrusted text uses frozen redaction.""" + if isinstance(value, dict): + return {frozen().redact(k): sanitize(v) for k, v in value.items()} + if isinstance(value, (list, tuple)): + return [sanitize(v) for v in value] + if isinstance(value, str): + # Constant source locators are public evidence, not untrusted error text. + public_sources = {RISK_ROUTE["source"], RISK_ROUTE["issue"], RELEASE_VERIFICATION["source"]} + public_sources.update(issue["url"] for step in BLOCKED_STEPS for issue in step["issues"]) + if value in public_sources: + return frozen().scrub_serialized(value) + return frozen().redact(value) + return value + + +def serialized_receipt(receipt, secret_values=()): + text = json.dumps(sanitize(receipt), indent=2, sort_keys=True, allow_nan=False) + # Final id-shaped-text pass also covers dictionary keys and unexpected nested fields. + return frozen().scrub_serialized(text, secret_values) + "\n" + + +def write_receipt(path, receipt, secret_values=()): + path = Path(path) + protected = {PLAN_PATH.resolve(), Path(__file__).resolve(), FROZEN_RUN.resolve()} + if path.resolve() in protected or frozen().is_gate_receipt(path): + raise ValueError("protected receipt destination") + raw = serialized_receipt(receipt, secret_values) + path.parent.mkdir(parents=True, exist_ok=True) + temporary = None + try: + with tempfile.NamedTemporaryFile(mode="w", dir=path.parent, prefix=".paper-receipt-", delete=False) as stream: + temporary = Path(stream.name) + stream.write(raw) + os.replace(temporary, path) + finally: + if temporary is not None: + temporary.unlink(missing_ok=True) + + +def case_outcomes(events, *, stop_ns=None, flat_proof=None): + """Pure mapping of sanitized native cache events; no strings imply acceptance. + + Each event carries an opaque local order alias and its type/side/quantity/price. + C2 must cancel C1, and C4 must fill after stop AND have an independent flat proof. + Duplicate native event ids are removed in snapshot_events, before this mapping. + """ + definitions = (("LIMIT", "BUY"), ("LIMIT", "BUY"), ("MARKET", "BUY"), ("MARKET", "SELL")) + result = {cid: {"id": cid, "order_type": typ, "side": side, "price": None, + "order": None, "events": [], "outcome": "not_run", "reason": None} + for cid, (typ, side) in zip(CASE_IDS, definitions)} + aliases = {} + for event in events: + aliases.setdefault(event["order"], event) + chosen = {} + for cid, shape in (("C1", ("LIMIT", "BUY")), ("C3", ("MARKET", "BUY")), ("C4", ("MARKET", "SELL"))): + candidates = [key for key, e in aliases.items() if (e["order_type"], e["side"]) == shape] + if candidates: + chosen[cid] = candidates[0] + if "C1" in chosen: + chosen["C2"] = chosen["C1"] + for cid, alias in chosen.items(): + case = result[cid] + seq = [e for e in events if e["order"] == alias] + case.update(order=alias, price=seq[0].get("price"), events=seq, outcome="incomplete") + confirmed = [e for e in seq if not e.get("reconciliation", False)] + bad = [e for e in seq if e["type"] in TERMINAL_FAILURES or e.get("reconciliation", False)] + if bad: + case.update(outcome="failed", reason=f"{cid}_unconfirmed_or_rejected") + continue + if decimal(seq[0]["quantity"]) != 1: + case.update(outcome="failed", reason=f"{cid}_quantity_bound") + continue + if cid == "C1": + if any(e["type"] == "OrderFilled" for e in confirmed): + case.update(outcome="failed", reason="C1_resting_order_filled") + elif any(e["type"] == "OrderAccepted" for e in confirmed): + case["outcome"] = "passed" + elif cid == "C2": + accepted = [e for e in confirmed if e["type"] == "OrderAccepted"] + canceled = [e for e in confirmed if e["type"] == "OrderCanceled"] + if stop_ns is not None and accepted and any(e["ts_init"] >= stop_ns and e["ts_init"] >= accepted[0]["ts_init"] for e in canceled): + case["outcome"] = "passed" + else: + fills = [e for e in confirmed if e["type"] == "OrderFilled"] + filled = sum((decimal(e["fill_quantity"]) for e in fills), Decimal(0)) + if filled > 1: + case.update(outcome="failed", reason=f"{cid}_overfill") + elif filled == 1: + if cid == "C3": + case["outcome"] = "passed" + elif stop_ns is not None and all(e["ts_init"] >= stop_ns for e in fills): + case["outcome"] = "filled_unproven" + if flat_proof and flat_proof.get("status") == "passed": + observed = flat_proof.get("observed", {}) + if observed.get("positions") == 0 and observed.get("open_orders") == 0: + case["outcome"] = "passed" + if case["outcome"] == "incomplete": + case["reason"] = f"{cid}_expected_event_missing" + return result + + +def roundtrip(events, loss_bound): + fills = [e for e in events if e["type"] == "OrderFilled"] + if not fills: + return None + buys = sum((decimal(e["fill_quantity"]) for e in fills if e["side"] == "BUY"), Decimal(0)) + sells = sum((decimal(e["fill_quantity"]) for e in fills if e["side"] == "SELL"), Decimal(0)) + gross = sum((decimal(e["fill_price"]) * decimal(e["fill_quantity"]) * (1 if e["side"] == "SELL" else -1) for e in fills), Decimal(0)) + unresolved = any(e.get("commission") is None or decimal(e["commission"]) <= 0 + or e.get("commission_currency") != "USD" or e.get("currency") != "USD" for e in fills) + fees = None if unresolved else sum((decimal(e["commission"]) for e in fills), Decimal(0)) + closed = buys == sells == 1 + net = gross - fees if fees is not None and closed else None + return {"buy_quantity": str(buys), "sell_quantity": str(sells), "closed": closed, + "gross_usd": str(gross) if buys == sells else None, + "commissions_usd": str(fees) if fees is not None else None, + "net_usd": str(net) if net is not None else None, + "commission_unresolved": unresolved, + "loss_bound_breach": net < -decimal(loss_bound) if net is not None else None} + + +def new_receipt(plan, plan_path, versions): + if not isinstance(plan, dict): + plan = {} + bounds = plan.get("bounds") if isinstance(plan.get("bounds"), dict) else {} + try: + cap = str(decimal(bounds["max_notional_per_order_usd"])) + except (KeyError, ValueError, InvalidOperation): + cap = None + try: + plan_hash = sha256(plan_path) + except OSError: + plan_hash = None + return {"schema_version": 1, "kind": KIND, "evidence_class": "native_paper", + "started_at": datetime.now(timezone.utc).isoformat(), "versions": versions, + "plan_sha256": plan_hash, "harness_sha256": sha256(__file__), + "source_hashes": {"frozen_checker_sha256": sha256(FROZEN_RUN)}, + "upstream_commit": UPSTREAM_COMMIT, "pre_check": None, "quote_admission": None, + "cases": case_outcomes([]), "events": [], "fills": [], "roundtrip": None, + "flat_proof": None, "node": {"started": False, "stop_ns": None}, + "risk": {"bypass": False, "max_notional_per_order_usd": cap, + "enforcement": "runner_quantity_and_quote_admission", "note": RISK_NOTE, + "engine_route": RISK_ROUTE, "submit_budget_enforcement": "engine_throttle", + "release_verification": RELEASE_VERIFICATION}, + "blocked_steps": BLOCKED_STEPS, "failures": [], + "status": "cleanup_required", "exit_code": 3} + + +def finish(receipt, status, *, reason=None, step=2, cases=CASE_IDS): + receipt["status"] = status + receipt["exit_code"] = frozen().exit_code_for(status) + receipt["ended_at"] = datetime.now(timezone.utc).isoformat() + if reason: + receipt["failures"].append({"acceptance_step": step, "cases_blocked": list(cases), "reason": reason}) + return receipt["exit_code"] + + +def native_configs(plan, port, account_id, *, tob_offset_ticks): + """Only upstream configs, with installed rc5 signatures verified offline. + + Source: v2.0.0rc5 examples/live/interactive_brokers/{exec_tester,_common}.py. + Construction does not connect. Actual account_id is supplied by the official + pre-check in memory, never from TWS_ACCOUNT or a command line argument. + """ + from nautilus_trader.adapters import interactive_brokers as ib + from nautilus_trader.config import LiveRiskEngineConfig + from nautilus_trader.model import ClientId, InstrumentId, Quantity, StrategyId, TimeInForce + from nautilus_trader.testkit import ExecTesterConfig + instrument = InstrumentId.from_str(plan["instrument"]["nautilus_instrument_id"]) + provider = ib.InteractiveBrokersInstrumentProviderConfig(symbology_method=ib.SymbologyMethod.RAW, load_ids={instrument}) + common = dict(host=plan["host"], port=port, client_id=plan["client_ids"]["node"], + connection_timeout=10, request_timeout=30, instrument_provider=provider) + tester_values = dict(plan["exec_tester"]) + tester_values["tob_offset_ticks"] = tob_offset_ticks + tester_values["order_qty"] = Quantity.from_str(tester_values["order_qty"]) + tester_values["open_position_on_start_qty"] = Decimal(tester_values["open_position_on_start_qty"]) + for key in ("open_position_time_in_force", "limit_time_in_force"): + tester_values[key] = TimeInForce.from_str(tester_values[key]) + tester = ExecTesterConfig(strategy_id=StrategyId.from_str("EXEC_TESTER-001"), instrument_id=instrument, + client_id=ClientId.from_str("IB"), external_order_instrument_ids=[instrument], + subscribe_quotes=True, subscribe_trades=True, **tester_values) + risk = LiveRiskEngineConfig(bypass=False, max_order_submit_rate=plan["risk"]["max_order_submit_rate"], + max_notional_per_order={str(instrument): str(plan["bounds"]["max_notional_per_order_usd"])}) + return (ib, ib.InteractiveBrokersDataClientConfig(market_data_type=ib.MarketDataType.REALTIME, batch_quotes=False, **common), + ib.InteractiveBrokersExecutionClientConfig(account_id=account_id, **common), risk, tester) + + +def build_node(plan, port, account_id, *, tob_offset_ticks): + from nautilus_trader.common import Environment, LoggerConfig, LogLevel + from nautilus_trader.live import LiveNode + from nautilus_trader.model import TraderId + ib, data, execution, risk, tester = native_configs(plan, port, account_id, tob_offset_ticks=tob_offset_ticks) + node = (LiveNode.builder("IB-EXEC-TESTER-001", TraderId.from_str("TESTER-001"), Environment.LIVE) + .with_reconciliation(True).with_risk_engine_config(risk) + .with_logging(LoggerConfig(stdout_level=LogLevel.OFF, fileout_level=LogLevel.OFF, print_config=False)) + .with_timeout_connection(plan["timeouts"]["node_start_seconds"]) + .with_timeout_reconciliation(5).with_timeout_portfolio(5) + .with_timeout_disconnection_secs(5).with_delay_post_stop_secs(plan["timeouts"]["cleanup_seconds"]) + .add_data_client(None, ib.InteractiveBrokersDataClientFactory(), data) + .add_exec_client(None, ib.InteractiveBrokersExecutionClientFactory(), execution).build()) + node.add_builtin_strategy("ExecTester", tester) + return node + + +def tester_orders(cache): + from nautilus_trader.model import StrategyId + return cache.orders(strategy_id=StrategyId.from_str("EXEC_TESTER-001")) + + +def snapshot_events(cache): + """Read cloned order histories, never serialize account/order/venue/trade ids.""" + orders = tester_orders(cache) + orders.sort(key=lambda order: (order.ts_init, str(order.client_order_id))) + events, seen = [], set() + for index, order in enumerate(orders, 1): + for event in order.events(): + key = str(event.event_id) + if key in seen: + continue + seen.add(key) + price = getattr(order, "price", None) + record = {"order": f"O{index}", "order_type": order.order_type.name, "side": order.side.name, + "instrument_id": str(order.instrument_id), + "quantity": str(order.quantity), "price": str(price) if price is not None else None, + "type": type(event).__name__, "ts_event": int(event.ts_event), "ts_init": int(event.ts_init), + "at": frozen().ns_to_iso(event.ts_event), "reconciliation": bool(getattr(event, "reconciliation", False))} + if record["type"] == "OrderFilled": + commission = event.commission + record.update(fill_price=str(event.last_px), fill_quantity=str(event.last_qty), currency=str(event.currency), + commission=str(commission.as_decimal()) if commission is not None else None, + commission_currency=str(commission.currency) if commission is not None else None) + if hasattr(event, "reason"): + record["reason"] = frozen().redact(event.reason) + events.append(record) + return sorted(events, key=lambda event: (event["ts_init"], event["ts_event"])) + + +def update_observations(receipt, events, plan): + receipt["events"] = events + receipt["cases"] = case_outcomes(events, stop_ns=receipt["node"]["stop_ns"], flat_proof=receipt["flat_proof"]) + receipt["fills"] = [e for e in events if e["type"] == "OrderFilled"] + receipt["roundtrip"] = roundtrip(events, plan["bounds"]["max_roundtrip_loss_usd"]) + orders = {e["order"]: e for e in events} + violations = [] + if len(orders) > plan["bounds"]["max_orders"]: + violations.append(("observed_order_budget_breach", list(CASE_IDS))) + for e in orders.values(): + cases = ["C1", "C2"] if e["order_type"] == "LIMIT" else ["C3" if e["side"] == "BUY" else "C4"] + quantity = decimal(e["quantity"]) + if quantity != 1: + violations.append(("observed_quantity_bound_breach", cases)) + if e.get("instrument_id") != plan["instrument"]["nautilus_instrument_id"]: + violations.append(("observed_instrument_bound_breach", cases)) + if e.get("price") is not None and decimal(e["price"]) * quantity > decimal(plan["bounds"]["max_notional_per_order_usd"]): + violations.append(("observed_notional_bound_breach", cases)) + for e in events: + if e["type"] == "OrderFilled" and decimal(e["fill_price"]) * decimal(e["fill_quantity"]) > decimal(plan["bounds"]["max_notional_per_order_usd"]): + violations.append(("observed_fill_notional_bound_breach", ["C3" if e["side"] == "BUY" else "C4"])) + for reason, cases in violations: + if not any(f["reason"] == reason and f["cases_blocked"] == cases for f in receipt["failures"]): + receipt["failures"].append({"reason": reason, "cases_blocked": cases, "acceptance_step": 2}) + + +def orders_in_flight(orders): + """Upstream close-on-stop runs once; wait for entry resolution before stop.""" + terminal = {"FILLED", "CANCELED", "EXPIRED", "REJECTED", "DENIED"} + return any(order.status.name not in terminal and + (order.order_type.name == "MARKET" or order.time_in_force.name in ("IOC", "FOK") + or order.status.name == "SUBMITTED" or order.venue_order_id is None) for order in orders) + + +def stop_ready(orders, now, node_stop_at): + return now >= node_stop_at or not orders_in_flight(orders) + + +async def node_phase(plan, port, receipt_path, payload): + """Hosted upstream run: cache + thread-safe handle captured before run_async. + + Source: crates/live/src/python/node.rs at UPSTREAM_COMMIT. The parent owns + the hard deadline; OS process termination cannot strand this awaitable. + """ + if not validate_child_admission(plan, payload): + return persist_child_refusal(receipt_path, payload, "refused_child_quote_admission") + receipt, account_id = payload["receipt"], payload["account_id"] + node = build_node(plan, port, account_id, tob_offset_ticks=payload["tob_offset_ticks"]) + # Live view: node.rs py_cache uses PyCache::from_rc(kernel.cache()); + # common/src/python/cache.rs PyCache(Rc>) borrows on each + # orders() call. Individual returned orders are clones; re-read each poll. + cache, handle = node.cache, node.handle() + loop = asyncio.get_running_loop() + pending_stop = None + registered_signals = [] + task = None + + def stop(reason): + if receipt["node"]["stop_ns"] is None: + receipt["node"].update(stop_ns=time.time_ns(), stop_reason=reason) + handle.stop() + + def request_stop(reason): + nonlocal pending_stop + pending_stop = pending_stop or reason + + def record_error(step, exc): + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), + "reason": frozen().redact(f"{step}: {type(exc).__name__}: {exc}")}) + + observed_start = None + try: + for sig in SIGNALS: + loop.add_signal_handler(sig, request_stop, f"signal_{sig.name}") + registered_signals.append(sig) + task = asyncio.create_task(node.run_async()) + try: + while not task.done(): + if handle.is_running: + receipt["node"]["started"] = True + observed_start = observed_start or time.monotonic() + update_observations(receipt, snapshot_events(cache), plan) + cases = receipt["cases"] + now = time.monotonic() + if receipt["failures"] or any(c["outcome"] == "failed" for c in cases.values()): + request_stop("case_failure") + elif cases["C1"]["outcome"] == cases["C3"]["outcome"] == "passed": + request_stop("C1_accepted_C3_filled") + elif observed_start is not None and now - observed_start >= plan["timeouts"]["per_step_seconds"]: + request_stop("case_timeout") + if now >= payload["node_stop_at"]: + stop("deadline") + elif pending_stop: + if stop_ready(tester_orders(cache), now, payload["node_stop_at"]): + stop(pending_stop) + write_receipt(receipt_path, receipt, (account_id,)) + await asyncio.sleep(0.1) + except Exception as exc: + record_error("node_poll", exc) + # An accepted MARKET IOC can still fill; preserve close-on-stop's + # one opportunity to see its position even if observation failed. + while not task.done() and time.monotonic() < payload["node_stop_at"]: + try: + if stop_ready(tester_orders(cache), time.monotonic(), payload["node_stop_at"]): + break + except Exception: + pass # Unknown state waits only to the bounded deadline. + await asyncio.sleep(0.1) + stop("poll_exception") + await task # Native cancel/close runs even after an observer exception. + except Exception as exc: + record_error("node_run", exc) + stop("node_exception") + if task is not None and not task.done(): + try: + await task + except Exception as task_exc: + record_error("node_shutdown", task_exc) + finally: + try: + update_observations(receipt, snapshot_events(cache), plan) + except Exception as exc: + record_error("final_snapshot", exc) + try: + write_receipt(receipt_path, receipt, (account_id,)) + except Exception as exc: + record_error("final_receipt", exc) + for sig in registered_signals: + try: + loop.remove_signal_handler(sig) + except Exception as exc: + record_error("signal_removal", exc) + try: + node.dispose() + except Exception as exc: + record_error("node_dispose", exc) + # Persist cleanup errors independently, without skipping disposal. + write_receipt(receipt_path, receipt, (account_id,)) + return receipt + + +class CheckTimeout(TimeoutError): + pass + + +def official_check(plan, port, *, with_session, deadline): + """Bound frozen ibapi connect/handshake as well as its request waits.""" + seconds = min(plan["timeouts"]["check_seconds"], max(0, deadline - time.monotonic())) + if seconds <= 0: + return {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}}, None + + def expired(_sig, _frame): + raise CheckTimeout("official_check_deadline") + + previous = signal.signal(signal.SIGALRM, expired) + signal.setitimer(signal.ITIMER_REAL, seconds) + try: + return frozen().run_check(plan, port, with_session=with_session, deadline_s=seconds) + except Exception as exc: + return {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}, + "error": frozen().redact(f"{type(exc).__name__}: {exc}")}, None + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, previous) + + +def official_admission(plan, port, *, deadline, client_factory=None): + """Flat pre-check and quote admission share one official client-92 session.""" + seconds = min(plan["timeouts"]["check_seconds"], max(0, deadline - time.monotonic())) + client = (client_factory or build_admission_client)() + session = CheckSession(client) + check_deadline = time.monotonic() + seconds + pre = {"client": "ibapi", "client_id": 92, "status": "not_connected", "observed": client.r} + account_id = None + + def failure(exc): + status = "not_connected" + if "tick requested" in client.stage_elapsed_seconds: + status = "refused_quote_clock" if "quote" in client.stage_elapsed_seconds else "refused_quote_unavailable" + if client.delayed: + status = "refused_delayed_quote" + elif client.errors and status != "not_connected": + status = "refused_quote_error" + return {"status": status, "error": frozen().redact(exc), **client.diagnostics()} + + if seconds <= 0: + return pre, None, failure("quote_check_deadline") + + def expired(_sig, _frame): + raise CheckTimeout("quote_check_deadline") + + previous = signal.signal(signal.SIGALRM, expired) + signal.setitimer(signal.ITIMER_REAL, seconds) + try: + pre, account_id = frozen().run_check(plan, port, with_session=True, + client_factory=lambda: session, deadline_s=seconds) + if pre["status"] != "passed" or not account_id: + return pre, account_id, {"status": "not_run", **client.diagnostics()} + admitted = quote_check(plan, account_id, deadline=check_deadline, client=client) + return pre, account_id, admitted + except Exception as exc: + return pre, account_id, failure(exc) + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, previous) + session.close(check_deadline) + + +def final_status(receipt, *, interrupted=False, child_returncode=0): + """Flat-proof failure dominates; observed/case/loss breaches remain failed.""" + if receipt["flat_proof"]["status"] != "passed": + return "cleanup_required", "independent_flat_proof", 4, ("C4",) + child_refusal = receipt["node"].get("pre_node_refusal") + if not receipt["node"]["started"] and child_refusal: + return child_refusal, child_refusal, 2, CASE_IDS + rt = receipt["roundtrip"] + if (any(f["reason"].startswith("observed_") for f in receipt["failures"]) + or any(c["outcome"] == "failed" for c in receipt["cases"].values()) + or rt and rt["loss_bound_breach"]): + return "failed", "case_or_loss_bound", 2, CASE_IDS + if interrupted or receipt["failures"] or child_returncode != 0: + return "incomplete", "signal_or_node_failure", 3, CASE_IDS + if all(c["outcome"] == "passed" for c in receipt["cases"].values()) and rt and rt["closed"] and not rt["commission_unresolved"]: + return "passed", None, 2, CASE_IDS + return "incomplete", "cases_or_commissions_unresolved", 3, CASE_IDS + + +def run_trial(args, *, now=None, versions=None): + """Every admission refusal precedes official-check and node construction.""" + started_monotonic = time.monotonic() + try: + plan = load_plan(args.plan) + except (ValueError, OSError): + plan = {} + versions = runtime_versions() if versions is None else versions + receipt = new_receipt(plan, args.plan, versions) + errors = validate_plan(plan, port=args.port, node_client_id=args.node_client_id, check_client_id=args.check_client_id) + status = "refused_plan" if errors else runtime_refusal(versions) + now = now or datetime.now(timezone.utc) + if status is None: + inside, window = frozen().rth_check(now, plan) + receipt["window"] = window + if not inside: + status = "refused_outside_window" + if status: + finish(receipt, status, reason="; ".join(errors) if errors else status) + if args.receipt: + write_receipt(args.receipt, receipt) + return receipt + if args.plan_only: + finish(receipt, "passed") + receipt["evidence_class"] = "structural_validation" + if args.receipt: + write_receipt(args.receipt, receipt) + return receipt + if args.receipt is None: + finish(receipt, "refused_receipt_required", reason="receipt required before node") + return receipt + # Command-center r2 decision: runner quantity + quote admission hold the + # bound; the engine cap remains configured with its route limitation stated. + deadline = started_monotonic + plan["timeouts"]["overall_deadline_seconds"] + port = args.port if args.port is not None else plan["default_port"] + pre, account_id, admitted = official_admission(plan, port, deadline=deadline) + liquid = pre.pop("liquid_hours", "") + pre.pop("trading_hours", None) + zone = pre.pop("time_zone_id", "America/New_York") + receipt["pre_check"] = pre + receipt["quote_admission"] = admitted + if pre["status"] != "passed" or not account_id: + finish(receipt, pre["status"] if pre["status"] != "passed" else "refused_account_scope", reason="pre_check") + write_receipt(args.receipt, receipt, (account_id,)) + return receipt + sessions = frozen().parse_liquid_hours(liquid, zone, now.astimezone(frozen().ZoneInfo("America/New_York")).date()) + if sessions is None or not frozen().rth_check(datetime.now(timezone.utc), plan, sessions, horizon_s=deadline - time.monotonic())[0]: + finish(receipt, "refused_liquid_hours", reason="whole_run_broker_window") + write_receipt(args.receipt, receipt, (account_id,)) + return receipt + if admitted["status"] != "passed": + finish(receipt, admitted["status"], reason="quote_admission") + write_receipt(args.receipt, receipt, (account_id,)) + return receipt + # Atomic provisional cleanup_required receipt BEFORE creating the child/node. + write_receipt(args.receipt, receipt, (account_id,)) + child = None + interrupted = [] + old_handlers = {} + + def signaled(sig, _frame): + interrupted.append(signal.Signals(sig).name) + if child is not None and child.poll() is None: + child.send_signal(sig) + + try: + for sig in SIGNALS: + old_handlers[sig] = signal.signal(sig, signaled) + stop_at = deadline - plan["timeouts"]["post_check_reserve_seconds"] - plan["timeouts"]["cleanup_seconds"] - plan["timeouts"]["node_stop_allowance_seconds"] + command = [sys.executable, str(Path(__file__).resolve()), "--_node-phase", "--plan", str(Path(args.plan).resolve()), + "--port", str(port), "--receipt", str(Path(args.receipt).resolve())] + child_env = {key: value for key, value in os.environ.items() + if key not in ("RUST_LOG", "NAUTILUS_LOG", "TWS_ACCOUNT")} + child = subprocess.Popen(command, stdin=subprocess.PIPE, text=True, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=child_env) + # Anonymous pipe only: no account in argv, environment, file or receipt. + child.stdin.write(json.dumps({"account_id": account_id, "receipt": receipt, "node_stop_at": stop_at, + "quote_admission": admitted, "tob_offset_ticks": admitted["tob_offset_ticks"]})) + child.stdin.close() + try: + child.wait(timeout=max(0, stop_at - time.monotonic())) + except subprocess.TimeoutExpired: + child.send_signal(signal.SIGTERM) + child.wait(timeout=max(0, deadline - plan["timeouts"]["post_check_reserve_seconds"] - time.monotonic())) + except subprocess.TimeoutExpired: + child.kill() + child.wait() + receipt["failures"].append({"acceptance_step": 4, "cases_blocked": ["C2", "C4"], "reason": "node_hard_deadline"}) + except Exception as exc: + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), "reason": frozen().redact(exc)}) + if child is not None and child.poll() is None: + child.send_signal(signal.SIGTERM) + try: + child.wait(timeout=plan["timeouts"]["node_stop_allowance_seconds"]) + except subprocess.TimeoutExpired: + child.kill() + child.wait() + finally: + for sig, handler in old_handlers.items(): + signal.signal(sig, handler) + saved_failures = receipt["failures"] + try: + receipt = json.loads(Path(args.receipt).read_text()) + receipt["failures"].extend(saved_failures) + except (OSError, ValueError): + pass # Preserve provisional state; a corrupt receipt can never pass. + proof, proof_account = official_check(plan, port, with_session=False, deadline=deadline) + if proof["status"] == "passed" and proof_account != account_id: + proof["status"] = "refused_changed_account" + receipt["flat_proof"] = proof + update_observations(receipt, receipt["events"], plan) + status, reason, step, cases = final_status(receipt, interrupted=bool(interrupted), + child_returncode=child.returncode if child is not None else None) + finish(receipt, status, reason=reason, step=step, cases=cases) + write_receipt(args.receipt, receipt, (account_id, proof_account)) + return receipt + + +class PaperArgumentParser(argparse.ArgumentParser): + def error(self, message): + self.exit(3, "refused_arguments: " + frozen().redact(message) + "\n") + + +def parser(): + result = PaperArgumentParser(description=__doc__) + result.add_argument("--plan", type=Path, default=PLAN_PATH) + result.add_argument("--receipt", type=Path, help="sanitized, atomically replaced receipt; required for an order run") + result.add_argument("--plan-only", action="store_true", help="validate plan, pins and window WITHOUT connecting; quote admission is not exercised") + result.add_argument("--port", type=int, help="4002 (default) or 7497 only") + result.add_argument("--node-client-id", type=int, help="91 only") + result.add_argument("--check-client-id", type=int, help="92 only") + result.add_argument("--_node-phase", action="store_true", help=argparse.SUPPRESS) + return result + + +def persist_child_refusal(path, payload, reason): + """Keep a sanitized pre-node reason even though child output is discarded.""" + receipt = payload.get("receipt") if isinstance(payload, dict) else None + if not isinstance(receipt, dict): + try: + receipt = json.loads(Path(path).read_text()) + except (OSError, TypeError, ValueError): + receipt = new_receipt(load_plan(), PLAN_PATH, runtime_versions()) + receipt["node"]["pre_node_refusal"] = reason + finish(receipt, reason, reason=reason) + if path is not None: + account = payload.get("account_id") if isinstance(payload, dict) else None + write_receipt(path, receipt, (account,)) + return receipt + + +def run_child(args): + try: + payload = json.load(sys.stdin) + if not isinstance(payload, dict): + raise ValueError("invalid child payload") + except (ValueError, OSError): + persist_child_refusal(args.receipt, {}, "refused_child_payload") + return 3 + try: + plan = load_plan(args.plan) + reason = "refused_child_plan" if validate_plan(plan, port=args.port) else None + except (ValueError, OSError): + reason = "refused_child_plan" + if reason is None and runtime_refusal(runtime_versions()): + reason = "refused_child_runtime" + if reason is None: + try: + inside = frozen().rth_check(datetime.now(timezone.utc), plan, + horizon_s=max(0, payload["node_stop_at"] - time.monotonic()))[0] + if not inside: + reason = "refused_child_window" + except (KeyError, TypeError, ValueError): + reason = "refused_child_payload" + if reason is None and not validate_child_admission(plan, payload): + reason = "refused_child_quote_admission" + if reason: + persist_child_refusal(args.receipt, payload, reason) + return 3 + receipt = asyncio.run(node_phase(plan, args.port, args.receipt, payload)) + if receipt["node"].get("pre_node_refusal"): + return 3 + return 1 if receipt["failures"] else 0 + + +def main(argv=None): + args = parser().parse_args(argv) + try: + if args._node_phase: + # The pipe supplies the admitted quote and derived offset; validate + # before node creation. This entry never loads an account from env. + return run_child(args) + receipt = run_trial(args) + except Exception as exc: + print(frozen().scrub_serialized(json.dumps({"status": "incomplete", "exit_code": 1, "error": frozen().redact(exc)}))) + return 1 + print(frozen().scrub_serialized(json.dumps({"status": receipt["status"], "exit_code": receipt["exit_code"], "window": receipt.get("window"), + "failures": sanitize(receipt["failures"])}))) + return receipt["exit_code"] + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-passed.json b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-passed.json new file mode 100644 index 000000000..3fc664c35 --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-passed.json @@ -0,0 +1,1090 @@ +{ + "blocked_steps": [ + { + "acceptance_step": 3, + "exercised": false, + "issue_states_checked_at": "2026-10-05", + "issues": [ + { + "number": 5057, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5057" + }, + { + "number": 5060, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5060" + } + ], + "operation": "reconnect_with_open_order" + }, + { + "acceptance_step": 3, + "exercised": false, + "issue_states_checked_at": "2026-10-05", + "issues": [ + { + "number": 5007, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5007" + }, + { + "number": 5057, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5057" + }, + { + "number": 5060, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5060" + } + ], + "operation": "restart_reconciliation" + }, + { + "acceptance_step": 4, + "exercised": false, + "issue_states_checked_at": "2026-10-05", + "issues": [ + { + "number": 5060, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5060" + } + ], + "operation": "kill_switch" + } + ], + "cases": { + "C1": { + "events": [ + { + "at": "2026-10-05T18:55:15.222024+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222024228, + "ts_init": 1791226515222024228, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T18:55:15.222774+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222773383, + "ts_init": 1791226515222773383, + "type": "OrderSubmitted" + }, + { + "at": "2026-10-05T18:55:15.222774+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222773383, + "ts_init": 1791226515222773383, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T18:55:15.380576+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515380575780, + "ts_init": 1791226515380575780, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.394644+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515394644527, + "ts_init": 1791226515394644527, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.506701+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515506700548, + "ts_init": 1791226515506700548, + "type": "OrderPendingCancel" + }, + { + "at": "2026-10-05T18:55:15.507013+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515507013335, + "ts_init": 1791226515507013335, + "type": "OrderPendingCancel" + }, + { + "at": "2026-10-05T18:55:15.516966+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515516965638, + "ts_init": 1791226515516965638, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.611890+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reason": "", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515611889727, + "ts_init": 1791226515611889727, + "type": "OrderCanceled" + } + ], + "id": "C1", + "order": "O2", + "order_type": "LIMIT", + "outcome": "passed", + "price": "387.62", + "reason": null, + "side": "BUY" + }, + "C2": { + "events": [ + { + "at": "2026-10-05T18:55:15.222024+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222024228, + "ts_init": 1791226515222024228, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T18:55:15.222774+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222773383, + "ts_init": 1791226515222773383, + "type": "OrderSubmitted" + }, + { + "at": "2026-10-05T18:55:15.222774+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222773383, + "ts_init": 1791226515222773383, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T18:55:15.380576+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515380575780, + "ts_init": 1791226515380575780, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.394644+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515394644527, + "ts_init": 1791226515394644527, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.506701+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515506700548, + "ts_init": 1791226515506700548, + "type": "OrderPendingCancel" + }, + { + "at": "2026-10-05T18:55:15.507013+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515507013335, + "ts_init": 1791226515507013335, + "type": "OrderPendingCancel" + }, + { + "at": "2026-10-05T18:55:15.516966+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515516965638, + "ts_init": 1791226515516965638, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.611890+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reason": "", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515611889727, + "ts_init": 1791226515611889727, + "type": "OrderCanceled" + } + ], + "id": "C2", + "order": "O2", + "order_type": "LIMIT", + "outcome": "passed", + "price": "387.62", + "reason": null, + "side": "BUY" + }, + "C3": { + "events": [ + { + "at": "2026-10-05T18:55:15.221774+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515221774419, + "ts_init": 1791226515221774419, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T18:55:15.222548+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222547365, + "ts_init": 1791226515222547365, + "type": "OrderSubmitted" + }, + { + "at": "2026-10-05T18:55:15.222548+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222547365, + "ts_init": 1791226515222547365, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T18:55:15.319377+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515319376713, + "ts_init": 1791226515319376713, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15+00:00", + "commission": "1.00", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "775.26", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515000000000, + "ts_init": 1791226515331725867, + "type": "OrderFilled" + } + ], + "id": "C3", + "order": "O1", + "order_type": "MARKET", + "outcome": "passed", + "price": null, + "reason": null, + "side": "BUY" + }, + "C4": { + "events": [ + { + "at": "2026-10-05T18:55:15.506788+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515506788511, + "ts_init": 1791226515506788511, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T18:55:15.507042+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515507041860, + "ts_init": 1791226515507041860, + "type": "OrderSubmitted" + }, + { + "at": "2026-10-05T18:55:15.507042+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515507041860, + "ts_init": 1791226515507041860, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T18:55:15.570492+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515570492015, + "ts_init": 1791226515570492015, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.610740+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515610740329, + "ts_init": 1791226515610740329, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15+00:00", + "commission": "1.02", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "775.24", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515000000000, + "ts_init": 1791226515610820931, + "type": "OrderFilled" + } + ], + "id": "C4", + "order": "O3", + "order_type": "MARKET", + "outcome": "passed", + "price": null, + "reason": null, + "side": "SELL" + } + }, + "ended_at": "2026-10-05T18:56:01.763903+00:00", + "events": [ + { + "at": "2026-10-05T18:55:15.221774+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515221774419, + "ts_init": 1791226515221774419, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T18:55:15.222024+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222024228, + "ts_init": 1791226515222024228, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T18:55:15.222548+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222547365, + "ts_init": 1791226515222547365, + "type": "OrderSubmitted" + }, + { + "at": "2026-10-05T18:55:15.222548+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222547365, + "ts_init": 1791226515222547365, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T18:55:15.222774+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222773383, + "ts_init": 1791226515222773383, + "type": "OrderSubmitted" + }, + { + "at": "2026-10-05T18:55:15.222774+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515222773383, + "ts_init": 1791226515222773383, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T18:55:15.319377+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515319376713, + "ts_init": 1791226515319376713, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15+00:00", + "commission": "1.00", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "775.26", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515000000000, + "ts_init": 1791226515331725867, + "type": "OrderFilled" + }, + { + "at": "2026-10-05T18:55:15.380576+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515380575780, + "ts_init": 1791226515380575780, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.394644+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515394644527, + "ts_init": 1791226515394644527, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.506701+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515506700548, + "ts_init": 1791226515506700548, + "type": "OrderPendingCancel" + }, + { + "at": "2026-10-05T18:55:15.506788+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515506788511, + "ts_init": 1791226515506788511, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T18:55:15.507013+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515507013335, + "ts_init": 1791226515507013335, + "type": "OrderPendingCancel" + }, + { + "at": "2026-10-05T18:55:15.507042+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515507041860, + "ts_init": 1791226515507041860, + "type": "OrderSubmitted" + }, + { + "at": "2026-10-05T18:55:15.507042+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515507041860, + "ts_init": 1791226515507041860, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T18:55:15.516966+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515516965638, + "ts_init": 1791226515516965638, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.570492+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515570492015, + "ts_init": 1791226515570492015, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15.610740+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515610740329, + "ts_init": 1791226515610740329, + "type": "OrderUpdated" + }, + { + "at": "2026-10-05T18:55:15+00:00", + "commission": "1.02", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "775.24", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515000000000, + "ts_init": 1791226515610820931, + "type": "OrderFilled" + }, + { + "at": "2026-10-05T18:55:15.611890+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "LIMIT", + "price": "387.62", + "quantity": "1", + "reason": "", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515611889727, + "ts_init": 1791226515611889727, + "type": "OrderCanceled" + } + ], + "evidence_class": "native_paper", + "exit_code": 0, + "failures": [], + "fills": [ + { + "at": "2026-10-05T18:55:15+00:00", + "commission": "1.00", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "775.26", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226515000000000, + "ts_init": 1791226515331725867, + "type": "OrderFilled" + }, + { + "at": "2026-10-05T18:55:15+00:00", + "commission": "1.02", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "775.24", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O3", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226515000000000, + "ts_init": 1791226515610820931, + "type": "OrderFilled" + } + ], + "flat_proof": { + "client": "ibapi", + "client_id": 92, + "observed": { + "account_count": 1, + "contract": null, + "errors": [], + "info": [ + { + "code": 2104, + "reqId": -1, + "text": "Market data farm connection is OK:usfarm" + }, + { + "code": 2106, + "reqId": -1, + "text": "HMDS data farm connection is OK:ushmds" + }, + { + "code": 2158, + "reqId": -1, + "text": "Sec-def data farm connection is OK:secdefil" + } + ], + "open_order_statuses": [], + "open_orders": 0, + "paper_accounts": true, + "positions": 0, + "server_minus_local_s": -0.644, + "server_time_epoch": 1791226561, + "spy_position": false + }, + "port": 4002, + "requests_completed": { + "accounts": true, + "contract": false, + "orders": true, + "positions": true, + "time": true + }, + "status": "passed" + }, + "harness_sha256": "5a061ff06f41fa1b5721f10fb01037ce790639f715481c2efdd98ea32700ee3b", + "kind": "ibkr_paper_orders_nautilus_2_0_0rc5", + "node": { + "started": true, + "stop_ns": 1791226515413701132, + "stop_reason": "C1_accepted_C3_filled" + }, + "plan_sha256": "24ffca56dfa7bd647bcf423dbc91ae3f605384a81a0ad9523cc024a36ac3ec72", + "pre_check": { + "client": "ibapi", + "client_id": 92, + "observed": { + "account_count": 1, + "contract": { + "currency": "USD", + "exchange": "SMART", + "minTick": 0.01, + "primaryExchange": "ARCA", + "secType": "STK", + "timeZoneId": "US/Eastern" + }, + "errors": [], + "info": [ + { + "code": 2104, + "reqId": -1, + "text": "Market data farm connection is OK:usfarm" + }, + { + "code": 2106, + "reqId": -1, + "text": "HMDS data farm connection is OK:ushmds" + }, + { + "code": 2158, + "reqId": -1, + "text": "Sec-def data farm connection is OK:secdefil" + } + ], + "open_order_statuses": [], + "open_orders": 0, + "paper_accounts": true, + "positions": 0, + "server_minus_local_s": -1.482, + "server_time_epoch": 1791226510, + "spy_position": false + }, + "port": 4002, + "requests_completed": { + "accounts": true, + "contract": true, + "orders": true, + "positions": true, + "time": true + }, + "status": "passed" + }, + "quote_admission": { + "admitted_monotonic": 144645.565294293, + "ask": "775.26", + "ask_notional_usd": "775.26", + "bid": "775.24", + "elapsed_seconds": 0.9157, + "errors": [], + "headroom_usd": "10", + "info": [ + { + "code": 2104, + "reqId": -1, + "text": "Market data farm connection is OK:usfarm" + }, + { + "code": 2106, + "reqId": -1, + "text": "HMDS data farm connection is OK:ushmds" + }, + { + "code": 2158, + "reqId": -1, + "text": "Sec-def data farm connection is OK:secdefil" + } + ], + "last_stage": "clock", + "last_stage_elapsed_seconds": 0.0001, + "quote_age_seconds": 0.0, + "quote_time_epoch": 1791226512, + "server_time_epoch": 1791226511, + "source": "ibapi.reqTickByTickData.BidAsk/reqCurrentTime", + "stage_duration_seconds": { + "accounts": 0.0049, + "clock": 0.0442, + "connected": 0.0156, + "nextValidId": 0.0009, + "quote": 0.6237, + "tick requested": 0.2262 + }, + "stage_elapsed_seconds": { + "accounts": 0.0205, + "clock": 0.9156, + "connected": 0.0156, + "nextValidId": 0.0214, + "quote": 0.8713, + "tick requested": 0.2476 + }, + "status": "passed", + "tob_offset_ticks": 38762 + }, + "reconciled_external": [ + { + "at": "2026-10-05T18:55:14.205701+00:00", + "order": "X1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791226514205701129, + "ts_init": 1791226514205701129, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T14:26:14+00:00", + "order": "X1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "BUY", + "ts_event": 1791210374000000000, + "ts_init": 1791226514205911216, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T14:26:14+00:00", + "commission": "1.00", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "772.93", + "fill_quantity": "1", + "order": "X1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "BUY", + "ts_event": 1791210374000000000, + "ts_init": 1791226514206105212, + "type": "OrderFilled" + }, + { + "at": "2026-10-05T18:55:14.206405+00:00", + "order": "X2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791226514206404603, + "ts_init": 1791226514206404603, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T14:26:15+00:00", + "order": "X2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "SELL", + "ts_event": 1791210375000000000, + "ts_init": 1791226514206424863, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T14:26:15+00:00", + "commission": "1.02", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "772.90", + "fill_quantity": "1", + "order": "X2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "SELL", + "ts_event": 1791210375000000000, + "ts_init": 1791226514206448034, + "type": "OrderFilled" + } + ], + "risk": { + "bypass": false, + "enforcement": "runner_quantity_and_quote_admission", + "engine_route": { + "issue": "https://github.com/nautechsystems/nautilus_trader/issues/4946", + "issue_state": "closed", + "pin_fix_present": false, + "reason": "rc5 stock orders can skip account-scoped max_notional_per_order on the SMART/IB venue path", + "source": "https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/risk/src/engine/mod.rs#L1218", + "upstream_fix": "fixed on develop; included in v2.0.0rc6; absent from the selected rc5 pin" + }, + "max_notional_per_order_usd": "1000", + "note": "engine notional cap configured, not enforced on this route (#4946, fixed on develop, unreleased); bound held by qty=1 and quote admission", + "release_verification": { + "checked_at": "2026-10-05", + "correction": "The requested note's unreleased clause reflects earlier metadata; the fix is now released in rc6, while this trial stays pinned to rc5.", + "fix_commit": "ed6fc8bf47fd37dda97d63b9b2df160719ee2bac", + "published_at": "2026-10-05T03:03:18Z", + "released_in": "v2.0.0rc6", + "source": "https://github.com/nautechsystems/nautilus_trader/releases/tag/v2.0.0rc6" + }, + "submit_budget_enforcement": "engine_throttle" + }, + "roundtrip": { + "buy_quantity": "1", + "closed": true, + "commission_unresolved": false, + "commissions_usd": "2.02", + "gross_usd": "-0.02", + "loss_bound_breach": false, + "net_usd": "-2.04", + "sell_quantity": "1" + }, + "schema_version": 1, + "source_hashes": { + "frozen_checker_sha256": "1b697f237275a725a0dcc39ee886c4f07aa27bfba1c2b442982b3c76b02e1012" + }, + "started_at": "2026-10-05T18:55:11.370630+00:00", + "status": "passed", + "upstream_commit": "1b0a49d2792a9432a3aca3fcb617ce7a630d905e", + "versions": { + "ibapi": "10.45.1", + "nautilus_ibapi_distribution": "10.45.1", + "nautilus_trader": "2.0.0rc5", + "nautilus_trader_distribution": "2.0.0rc5", + "protobuf": "5.29.6", + "python": "3.12.3" + }, + "window": "inside_window" +} diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-run1-not-connected.json b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-run1-not-connected.json new file mode 100644 index 000000000..8e5d3817c --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-run1-not-connected.json @@ -0,0 +1,218 @@ +{ + "blocked_steps": [ + { + "acceptance_step": 3, + "exercised": false, + "issue_states_checked_at": "2026-10-05", + "issues": [ + { + "number": 5057, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5057" + }, + { + "number": 5060, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5060" + } + ], + "operation": "reconnect_with_open_order" + }, + { + "acceptance_step": 3, + "exercised": false, + "issue_states_checked_at": "2026-10-05", + "issues": [ + { + "number": 5007, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5007" + }, + { + "number": 5057, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5057" + }, + { + "number": 5060, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5060" + } + ], + "operation": "restart_reconciliation" + }, + { + "acceptance_step": 4, + "exercised": false, + "issue_states_checked_at": "2026-10-05", + "issues": [ + { + "number": 5060, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5060" + } + ], + "operation": "kill_switch" + } + ], + "cases": { + "C1": { + "events": [], + "id": "C1", + "order": null, + "order_type": "LIMIT", + "outcome": "not_run", + "price": null, + "reason": null, + "side": "BUY" + }, + "C2": { + "events": [], + "id": "C2", + "order": null, + "order_type": "LIMIT", + "outcome": "not_run", + "price": null, + "reason": null, + "side": "BUY" + }, + "C3": { + "events": [], + "id": "C3", + "order": null, + "order_type": "MARKET", + "outcome": "not_run", + "price": null, + "reason": null, + "side": "BUY" + }, + "C4": { + "events": [], + "id": "C4", + "order": null, + "order_type": "MARKET", + "outcome": "not_run", + "price": null, + "reason": null, + "side": "SELL" + } + }, + "ended_at": "2026-10-05T17:40:45.632681+00:00", + "events": [], + "evidence_class": "native_paper", + "exit_code": 2, + "failures": [ + { + "acceptance_step": 2, + "cases_blocked": [ + "C1", + "C2", + "C3", + "C4" + ], + "reason": "quote_admission" + } + ], + "fills": [], + "flat_proof": null, + "harness_sha256": "188e0078fc8f4a36ef54eb856bf2ef2660cad1ae280332dc86e405b7a230a0f5", + "kind": "ibkr_paper_orders_nautilus_2_0_0rc5", + "node": { + "started": false, + "stop_ns": null + }, + "plan_sha256": "24ffca56dfa7bd647bcf423dbc91ae3f605384a81a0ad9523cc024a36ac3ec72", + "pre_check": { + "client": "ibapi", + "client_id": 92, + "observed": { + "account_count": 1, + "contract": { + "currency": "USD", + "exchange": "SMART", + "minTick": 0.01, + "primaryExchange": "ARCA", + "secType": "STK", + "timeZoneId": "US/Eastern" + }, + "errors": [], + "info": [ + { + "code": 2104, + "reqId": -1, + "text": "Market data farm connection is OK:usfarm" + }, + { + "code": 2106, + "reqId": -1, + "text": "HMDS data farm connection is OK:ushmds" + }, + { + "code": 2158, + "reqId": -1, + "text": "Sec-def data farm connection is OK:secdefil" + } + ], + "open_order_statuses": [], + "open_orders": 0, + "paper_accounts": true, + "positions": 0, + "server_minus_local_s": -1.381, + "server_time_epoch": 1791222014, + "spy_position": false + }, + "port": 4002, + "requests_completed": { + "accounts": true, + "contract": true, + "orders": true, + "positions": true, + "time": true + }, + "status": "passed" + }, + "quote_admission": { + "error": "quote_check_deadline", + "status": "not_connected" + }, + "risk": { + "bypass": false, + "enforcement": "runner_quantity_and_quote_admission", + "engine_route": { + "issue": "https://github.com/nautechsystems/nautilus_trader/issues/4946", + "issue_state": "closed", + "pin_fix_present": false, + "reason": "rc5 stock orders can skip account-scoped max_notional_per_order on the SMART/IB venue path", + "source": "https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/risk/src/engine/mod.rs#L1218", + "upstream_fix": "fixed on develop; included in v2.0.0rc6; absent from the selected rc5 pin" + }, + "max_notional_per_order_usd": "1000", + "note": "engine notional cap configured, not enforced on this route (#4946, fixed on develop, unreleased); bound held by qty=1 and quote admission", + "release_verification": { + "checked_at": "2026-10-05", + "correction": "The requested note's unreleased clause reflects earlier metadata; the fix is now released in rc6, while this trial stays pinned to rc5.", + "fix_commit": "ed6fc8bf47fd37dda97d63b9b2df160719ee2bac", + "published_at": "2026-10-05T03:03:18Z", + "released_in": "v2.0.0rc6", + "source": "https://github.com/nautechsystems/nautilus_trader/releases/tag/v2.0.0rc6" + }, + "submit_budget_enforcement": "engine_throttle" + }, + "roundtrip": null, + "schema_version": 1, + "source_hashes": { + "frozen_checker_sha256": "1b697f237275a725a0dcc39ee886c4f07aa27bfba1c2b442982b3c76b02e1012" + }, + "started_at": "2026-10-05T17:40:15.189378+00:00", + "status": "not_connected", + "upstream_commit": "1b0a49d2792a9432a3aca3fcb617ce7a630d905e", + "versions": { + "ibapi": "10.45.1", + "nautilus_ibapi_distribution": "10.45.1", + "nautilus_trader": "2.0.0rc5", + "nautilus_trader_distribution": "2.0.0rc5", + "protobuf": "5.29.6", + "python": "3.12.3" + }, + "window": "inside_window" +} diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-run2-failed.json b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-run2-failed.json new file mode 100644 index 000000000..0c85463f1 --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-run2-failed.json @@ -0,0 +1,531 @@ +{ + "blocked_steps": [ + { + "acceptance_step": 3, + "exercised": false, + "issue_states_checked_at": "2026-10-05", + "issues": [ + { + "number": 5057, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5057" + }, + { + "number": 5060, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5060" + } + ], + "operation": "reconnect_with_open_order" + }, + { + "acceptance_step": 3, + "exercised": false, + "issue_states_checked_at": "2026-10-05", + "issues": [ + { + "number": 5007, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5007" + }, + { + "number": 5057, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5057" + }, + { + "number": 5060, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5060" + } + ], + "operation": "restart_reconciliation" + }, + { + "acceptance_step": 4, + "exercised": false, + "issue_states_checked_at": "2026-10-05", + "issues": [ + { + "number": 5060, + "state": "open", + "url": "https://github.com/nautechsystems/nautilus_trader/issues/5060" + } + ], + "operation": "kill_switch" + } + ], + "cases": { + "C1": { + "events": [], + "id": "C1", + "order": null, + "order_type": "LIMIT", + "outcome": "not_run", + "price": null, + "reason": null, + "side": "BUY" + }, + "C2": { + "events": [], + "id": "C2", + "order": null, + "order_type": "LIMIT", + "outcome": "not_run", + "price": null, + "reason": null, + "side": "BUY" + }, + "C3": { + "events": [ + { + "at": "2026-10-05T18:22:55.675839+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791224575675839175, + "ts_init": 1791224575675839175, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T14:26:14+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "BUY", + "ts_event": 1791210374000000000, + "ts_init": 1791224575676468916, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T14:26:14+00:00", + "commission": "1.00", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "772.93", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "BUY", + "ts_event": 1791210374000000000, + "ts_init": 1791224575676976124, + "type": "OrderFilled" + } + ], + "id": "C3", + "order": "O1", + "order_type": "MARKET", + "outcome": "failed", + "price": null, + "reason": "C3_unconfirmed_or_rejected", + "side": "BUY" + }, + "C4": { + "events": [ + { + "at": "2026-10-05T18:22:55.677837+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791224575677837312, + "ts_init": 1791224575677837312, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T14:26:15+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "SELL", + "ts_event": 1791210375000000000, + "ts_init": 1791224575677857533, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T14:26:15+00:00", + "commission": "1.02", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "772.90", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "SELL", + "ts_event": 1791210375000000000, + "ts_init": 1791224575677879194, + "type": "OrderFilled" + } + ], + "id": "C4", + "order": "O2", + "order_type": "MARKET", + "outcome": "failed", + "price": null, + "reason": "C4_unconfirmed_or_rejected", + "side": "SELL" + } + }, + "ended_at": "2026-10-05T18:23:42.104418+00:00", + "events": [ + { + "at": "2026-10-05T18:22:55.675839+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "BUY", + "ts_event": 1791224575675839175, + "ts_init": 1791224575675839175, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T14:26:14+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "BUY", + "ts_event": 1791210374000000000, + "ts_init": 1791224575676468916, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T14:26:14+00:00", + "commission": "1.00", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "772.93", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "BUY", + "ts_event": 1791210374000000000, + "ts_init": 1791224575676976124, + "type": "OrderFilled" + }, + { + "at": "2026-10-05T18:22:55.677837+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": false, + "side": "SELL", + "ts_event": 1791224575677837312, + "ts_init": 1791224575677837312, + "type": "OrderInitialized" + }, + { + "at": "2026-10-05T14:26:15+00:00", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "SELL", + "ts_event": 1791210375000000000, + "ts_init": 1791224575677857533, + "type": "OrderAccepted" + }, + { + "at": "2026-10-05T14:26:15+00:00", + "commission": "1.02", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "772.90", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "SELL", + "ts_event": 1791210375000000000, + "ts_init": 1791224575677879194, + "type": "OrderFilled" + } + ], + "evidence_class": "native_paper", + "exit_code": 1, + "failures": [ + { + "acceptance_step": 2, + "cases_blocked": [ + "C1", + "C2", + "C3", + "C4" + ], + "reason": "case_or_loss_bound" + } + ], + "fills": [ + { + "at": "2026-10-05T14:26:14+00:00", + "commission": "1.00", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "772.93", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O1", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "BUY", + "ts_event": 1791210374000000000, + "ts_init": 1791224575676976124, + "type": "OrderFilled" + }, + { + "at": "2026-10-05T14:26:15+00:00", + "commission": "1.02", + "commission_currency": "USD", + "currency": "USD", + "fill_price": "772.90", + "fill_quantity": "1", + "instrument_id": "SPY=STK.SMART", + "order": "O2", + "order_type": "MARKET", + "price": null, + "quantity": "1", + "reconciliation": true, + "side": "SELL", + "ts_event": 1791210375000000000, + "ts_init": 1791224575677879194, + "type": "OrderFilled" + } + ], + "flat_proof": { + "client": "ibapi", + "client_id": 92, + "observed": { + "account_count": 1, + "contract": null, + "errors": [], + "info": [ + { + "code": 2104, + "reqId": -1, + "text": "Market data farm connection is OK:usfarm" + }, + { + "code": 2106, + "reqId": -1, + "text": "HMDS data farm connection is OK:ushmds" + }, + { + "code": 2158, + "reqId": -1, + "text": "Sec-def data farm connection is OK:secdefil" + } + ], + "open_order_statuses": [], + "open_orders": 0, + "paper_accounts": true, + "positions": 0, + "server_minus_local_s": -0.983, + "server_time_epoch": 1791224621, + "spy_position": false + }, + "port": 4002, + "requests_completed": { + "accounts": true, + "contract": false, + "orders": true, + "positions": true, + "time": true + }, + "status": "passed" + }, + "harness_sha256": "67429251370f1d397e53e69c5ff0b8b0ef7c2cb0e069687f0b087e6cca2ba013", + "kind": "ibkr_paper_orders_nautilus_2_0_0rc5", + "node": { + "started": true, + "stop_ns": 1791224575758864574, + "stop_reason": "case_failure" + }, + "plan_sha256": "24ffca56dfa7bd647bcf423dbc91ae3f605384a81a0ad9523cc024a36ac3ec72", + "pre_check": { + "client": "ibapi", + "client_id": 92, + "observed": { + "account_count": 1, + "contract": { + "currency": "USD", + "exchange": "SMART", + "minTick": 0.01, + "primaryExchange": "ARCA", + "secType": "STK", + "timeZoneId": "US/Eastern" + }, + "errors": [], + "info": [ + { + "code": 2104, + "reqId": -1, + "text": "Market data farm connection is OK:usfarm" + }, + { + "code": 2106, + "reqId": -1, + "text": "HMDS data farm connection is OK:ushmds" + }, + { + "code": 2158, + "reqId": -1, + "text": "Sec-def data farm connection is OK:secdefil" + } + ], + "open_order_statuses": [], + "open_orders": 0, + "paper_accounts": true, + "positions": 0, + "server_minus_local_s": -0.394, + "server_time_epoch": 1791224573, + "spy_position": false + }, + "port": 4002, + "requests_completed": { + "accounts": true, + "contract": true, + "orders": true, + "positions": true, + "time": true + }, + "status": "passed" + }, + "quote_admission": { + "admitted_monotonic": 142706.909029817, + "ask": "774.88", + "ask_notional_usd": "774.88", + "bid": "774.87", + "elapsed_seconds": 1.3857, + "errors": [], + "headroom_usd": "10", + "info": [ + { + "code": 2104, + "reqId": -1, + "text": "Market data farm connection is OK:usfarm" + }, + { + "code": 2106, + "reqId": -1, + "text": "HMDS data farm connection is OK:ushmds" + }, + { + "code": 2158, + "reqId": -1, + "text": "Sec-def data farm connection is OK:secdefil" + } + ], + "last_stage": "clock", + "last_stage_elapsed_seconds": 0.0002, + "quote_age_seconds": 8.24510061647743e-05, + "quote_time_epoch": 1791224573, + "server_time_epoch": 1791224573, + "source": "ibapi.reqTickByTickData.BidAsk/reqCurrentTime", + "stage_duration_seconds": { + "accounts": 0.0086, + "clock": 0.0488, + "connected": 0.0242, + "nextValidId": 0.001, + "quote": 1.003, + "tick requested": 0.2999 + }, + "stage_elapsed_seconds": { + "accounts": 0.0328, + "clock": 1.3855, + "connected": 0.0242, + "nextValidId": 0.0338, + "quote": 1.3367, + "tick requested": 0.3337 + }, + "status": "passed", + "tob_offset_ticks": 38743 + }, + "risk": { + "bypass": false, + "enforcement": "runner_quantity_and_quote_admission", + "engine_route": { + "issue": "https://github.com/nautechsystems/nautilus_trader/issues/4946", + "issue_state": "closed", + "pin_fix_present": false, + "reason": "rc5 stock orders can skip account-scoped max_notional_per_order on the SMART/IB venue path", + "source": "https://github.com/nautechsystems/nautilus_trader/blob/1b0a49d2792a9432a3aca3fcb617ce7a630d905e/crates/risk/src/engine/mod.rs#L1218", + "upstream_fix": "fixed on develop; included in v2.0.0rc6; absent from the selected rc5 pin" + }, + "max_notional_per_order_usd": "1000", + "note": "engine notional cap configured, not enforced on this route (#4946, fixed on develop, unreleased); bound held by qty=1 and quote admission", + "release_verification": { + "checked_at": "2026-10-05", + "correction": "The requested note's unreleased clause reflects earlier metadata; the fix is now released in rc6, while this trial stays pinned to rc5.", + "fix_commit": "ed6fc8bf47fd37dda97d63b9b2df160719ee2bac", + "published_at": "2026-10-05T03:03:18Z", + "released_in": "v2.0.0rc6", + "source": "https://github.com/nautechsystems/nautilus_trader/releases/tag/v2.0.0rc6" + }, + "submit_budget_enforcement": "engine_throttle" + }, + "roundtrip": { + "buy_quantity": "1", + "closed": true, + "commission_unresolved": false, + "commissions_usd": "2.02", + "gross_usd": "-0.03", + "loss_bound_breach": false, + "net_usd": "-2.05", + "sell_quantity": "1" + }, + "schema_version": 1, + "source_hashes": { + "frozen_checker_sha256": "1b697f237275a725a0dcc39ee886c4f07aa27bfba1c2b442982b3c76b02e1012" + }, + "started_at": "2026-10-05T18:22:52.241961+00:00", + "status": "failed", + "upstream_commit": "1b0a49d2792a9432a3aca3fcb617ce7a630d905e", + "versions": { + "ibapi": "10.45.1", + "nautilus_ibapi_distribution": "10.45.1", + "nautilus_trader": "2.0.0rc5", + "nautilus_trader_distribution": "2.0.0rc5", + "protobuf": "5.29.6", + "python": "3.12.3" + }, + "window": "inside_window" +} diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/plan.json b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/plan.json new file mode 100644 index 000000000..c577758c5 --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/plan.json @@ -0,0 +1,97 @@ +{ + "schema_version": 1, + "kind": "ibkr_paper_orders_plan_rc5", + "frozen_at": "2026-10-05", + "evidence_class": "native_paper", + "engine": { + "version": "2.0.0rc5", + "ibapi_version": "10.45.1", + "adapter": "NautilusTrader native InteractiveBrokers adapter", + "upstream_commit": "1b0a49d2792a9432a3aca3fcb617ce7a630d905e" + }, + "host": "127.0.0.1", + "paper_ports": [4002, 7497], + "default_port": 4002, + "account_prefix": "DU", + "require_single_managed_account": true, + "client_ids": {"node": 91, "check": 92}, + "instrument": { + "symbol": "SPY", + "sec_type": "STK", + "exchange": "SMART", + "primary_exchange": "ARCA", + "currency": "USD", + "nautilus_instrument_id": "SPY=STK.SMART", + "expected_price_increment": "0.01" + }, + "bounds": { + "max_orders": 6, + "max_quantity_per_order": 1, + "max_notional_per_order_usd": 1000, + "max_roundtrip_loss_usd": 5, + "commission_allowance_per_order_usd": 1.0 + }, + "session": { + "timezone": "America/New_York", + "weekdays_only": true, + "open": "09:30", + "close": "16:00", + "close_buffer_minutes": 10, + "whole_run_inside_window": true, + "use_contract_liquid_hours": true + }, + "data": { + "market_data_type": "REALTIME", + "quote_max_age_seconds": 10, + "clock_resolution_tolerance_seconds": 2, + "batch_quotes": false + }, + "timeouts": { + "check_seconds": 30, + "node_start_seconds": 60, + "per_step_seconds": 45, + "cleanup_seconds": 45, + "node_stop_allowance_seconds": 15, + "post_check_reserve_seconds": 60, + "overall_deadline_seconds": 420 + }, + "risk": { + "bypass": false, + "max_order_submit_rate": "6/00:07:00", + "require_enforced_notional_before_connection": false, + "notional_headroom_usd": 10, + "known_pin_blocker": "https://github.com/nautechsystems/nautilus_trader/issues/4946" + }, + "exec_tester": { + "order_qty": "1", + "open_position_on_start_qty": "1", + "open_position_on_first_quote": true, + "open_position_time_in_force": "IOC", + "enable_limit_buys": true, + "enable_limit_sells": false, + "enable_stop_buys": false, + "enable_stop_sells": false, + "enable_brackets": false, + "tob_offset_ticks": "floor(0.5 * admitted_bid / 0.01)", + "limit_time_in_force": "GTD", + "order_expire_time_delta_mins": 7, + "modify_orders_to_maintain_tob_offset": false, + "cancel_replace_orders_to_maintain_tob_offset": false, + "use_post_only": false, + "use_individual_cancels_on_stop": true, + "cancel_orders_on_stop": true, + "close_positions_on_stop": true, + "reduce_only_on_stop": false, + "dry_run": false, + "log_data": false + }, + "cases": [ + {"id": "C1", "name": "accept_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderAccepted"}, + {"id": "C2", "name": "cancel_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderCanceled for C1 after stop"}, + {"id": "C3", "name": "fill_buy", "order_type": "MARKET", "side": "BUY", "expect": "OrderFilled quantity 1"}, + {"id": "C4", "name": "flatten", "order_type": "MARKET", "side": "SELL", "expect": "OrderFilled quantity 1 after stop and independent flat proof"} + ], + "end_disposition": {"positions": 0, "open_orders": 0}, + "receipt": {"kind": "ibkr_paper_orders_nautilus_2_0_0rc5"}, + "exit_codes": {"passed": 0, "failed_or_incomplete": 1, "not_connected": 2, "refused_or_cleanup_required": 3} +} diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/run.py b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/run.py new file mode 100644 index 000000000..bb1dac605 --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/run.py @@ -0,0 +1,1521 @@ +#!/usr/bin/env python3 +"""Bounded rc5 paper trial using upstream's built-in ExecTester. + +The rc5 engine cap is configured; quote admission and quantity one hold this +trial's runner bound on the SMART/IB route affected by #4946. --plan-only and +--help never connect. Synthetic tests do not establish broker acceptance. +""" +from __future__ import annotations + +import argparse +import asyncio +import copy +import hashlib +import importlib.metadata +import importlib.util +import json +import os +import platform +import signal +import subprocess +import sys +import tempfile +import threading +import time +import uuid +from datetime import datetime, timedelta, timezone +from decimal import Decimal, InvalidOperation, ROUND_FLOOR +from functools import lru_cache +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parents[3] +PLAN_PATH = HERE / "plan.json" +FROZEN_RUN = HERE.parent / "ibkr-paper-orders" / "run.py" +KIND = "ibkr_paper_orders_nautilus_2_0_0rc5" +UPSTREAM_COMMIT = "1b0a49d2792a9432a3aca3fcb617ce7a630d905e" +SOURCE = f"https://github.com/nautechsystems/nautilus_trader/blob/{UPSTREAM_COMMIT}" +CASE_IDS = ("C1", "C2", "C3", "C4") +SIGNALS = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP) +TERMINAL_FAILURES = {"OrderDenied", "OrderRejected", "OrderExpired", "OrderCancelRejected"} +# NMS Limit Up-Limit Down Plan, Tier 1 core-session price band. This is a +# single-band admission stress, not a submission-time engine notional cap. +LULD_TIER1_CORE_BAND = Decimal("0.05") +# NMS LULD Plan, Tier 1 last-25-minutes band: do not let the planned +# order/close horizon enter this double-width period with a 5% admission stress. +LULD_TIER1_CLOSING_PERIOD_BAND = Decimal("0.10") +LULD_TIER1_CLOSING_PERIOD_MINUTES = 25 +# The frozen USD 1/order allowance needs USD 0.02 for run 3's USD 1.02 sell fee. +COMMISSION_MARGIN_PER_ORDER_USD = Decimal("0.02") +TESTER_SETTINGS = { + "order_qty": "1", "open_position_on_start_qty": "1", "open_position_on_first_quote": True, + "open_position_time_in_force": "IOC", "enable_limit_buys": True, "enable_limit_sells": False, + "enable_stop_buys": False, "enable_stop_sells": False, "enable_brackets": False, + "tob_offset_ticks": "floor(0.5 * admitted_bid / 0.01)", "limit_time_in_force": "GTD", "order_expire_time_delta_mins": 7, + "modify_orders_to_maintain_tob_offset": False, "cancel_replace_orders_to_maintain_tob_offset": False, + "use_post_only": False, "use_individual_cancels_on_stop": True, + "cancel_orders_on_stop": True, "close_positions_on_stop": True, + "reduce_only_on_stop": False, "dry_run": False, "log_data": False, +} +CASE_DEFINITIONS = [ + {"id": "C1", "name": "accept_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderAccepted"}, + {"id": "C2", "name": "cancel_resting", "order_type": "LIMIT", "side": "BUY", "expect": "OrderCanceled for C1 after stop"}, + {"id": "C3", "name": "fill_buy", "order_type": "MARKET", "side": "BUY", "expect": "OrderFilled quantity 1"}, + {"id": "C4", "name": "flatten", "order_type": "MARKET", "side": "SELL", "expect": "OrderFilled quantity 1 after stop and independent flat proof"}, +] + +# Original sources, read at v2.0.0rc5 (UPSTREAM_COMMIT): +# examples/live/interactive_brokers/exec_tester.py: builder + ExecTester config. +# examples/live/interactive_brokers/_common.py: bounded node timeouts. +# crates/testkit/src/testers/exec/strategy.rs: native entry/cancel/close paths. +# crates/risk/src/engine/mod.rs: check_orders_risk_for_account returns true +# when account_for_venue(SMART) is absent, before checking max_notional. The +# IB account is registered under IB. Config support is not enforcement. +RISK_NOTE = "engine notional cap configured, not enforced on this route (#4946, fixed on develop, unreleased); bound held by qty=1 and quote admission" +RISK_ROUTE = { + "issue": "https://github.com/nautechsystems/nautilus_trader/issues/4946", + "issue_state": "closed", + "pin_fix_present": False, + "upstream_fix": "fixed on develop; included in v2.0.0rc6; absent from the selected rc5 pin", + "source": SOURCE + "/crates/risk/src/engine/mod.rs#L1218", + "reason": "rc5 stock orders can skip account-scoped max_notional_per_order on the SMART/IB venue path", +} +RELEASE_VERIFICATION = { + "checked_at": "2026-10-05", + "fix_commit": "ed6fc8bf47fd37dda97d63b9b2df160719ee2bac", + "released_in": "v2.0.0rc6", + "published_at": "2026-10-05T03:03:18Z", + "source": "https://github.com/nautechsystems/nautilus_trader/releases/tag/v2.0.0rc6", + "correction": "The requested note's unreleased clause reflects earlier metadata; the fix is now released in rc6, while this trial stays pinned to rc5.", +} +BLOCKED_STEPS = [ + {"acceptance_step": 3, "operation": "reconnect_with_open_order", "exercised": False, + "issues": [{"number": 5057, "state": "open"}, {"number": 5060, "state": "open"}]}, + {"acceptance_step": 3, "operation": "restart_reconciliation", "exercised": False, + "issues": [{"number": 5007, "state": "open"}, {"number": 5057, "state": "open"}, + {"number": 5060, "state": "open"}]}, + {"acceptance_step": 4, "operation": "kill_switch", "exercised": False, + "issues": [{"number": 5060, "state": "open"}]}, +] +for _step in BLOCKED_STEPS: + _step["issue_states_checked_at"] = "2026-10-05" + for _issue in _step["issues"]: + _issue["url"] = f"https://github.com/nautechsystems/nautilus_trader/issues/{_issue['number']}" + + +@lru_cache(maxsize=1) +def frozen(): + """Reuse the frozen official-ibapi checker, window parser and redaction. + + Source: ../ibkr-paper-orders/run.py, bound by source_hashes in the receipt. + No frozen strategy is used; the node uses rc5's built-in ExecTester. + """ + spec = importlib.util.spec_from_file_location("ibkr_frozen_readonly", FROZEN_RUN) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +@lru_cache(maxsize=1) +def safety(): + spec = importlib.util.spec_from_file_location("ibkr_orders_rc5_safety", HERE / "safety.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def sha256(path): + return hashlib.sha256(Path(path).read_bytes()).hexdigest() + + +def decimal(value): + result = Decimal(str(value)) + if not result.is_finite(): + raise ValueError("nonfinite numeric input") + return result + + +def load_plan(path=PLAN_PATH): + def unique(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError("duplicate plan key") + result[key] = value + return result + return json.loads(Path(path).read_text(), object_pairs_hook=unique) + + +def validate_plan(plan, *, port=None, node_client_id=None, check_client_id=None): + """All static bounds are checked without importing a client or connecting.""" + errors = [] + + def need(condition, reason): + if not condition: + errors.append(reason) + + try: + need(type(plan["schema_version"]) is int and plan["schema_version"] == 1 + and plan["kind"] == "ibkr_paper_orders_plan_rc5", "plan schema") + need(plan["engine"]["version"] == "2.0.0rc5" and plan["engine"]["ibapi_version"] == "10.45.1", "runtime pins") + need(plan["engine"]["upstream_commit"] == UPSTREAM_COMMIT, "upstream pin") + need(plan["host"] == "127.0.0.1", "loopback host required") + ports = plan["paper_ports"] + need(isinstance(ports, list) and bool(ports) and all(type(p) is int and p in (4002, 7497) for p in ports), "paper ports") + need(plan["default_port"] in ports and type(plan["default_port"]) is int, "default paper port") + need(port is None or type(port) is int and port in ports and port in (4002, 7497), "selected paper port") + need(plan["account_prefix"] == "DU" and plan["require_single_managed_account"] is True, "single DU account") + need(plan["client_ids"] == {"node": 91, "check": 92} + and all(type(v) is int for v in plan["client_ids"].values()), "client ids 91/92") + need(node_client_id is None or type(node_client_id) is int and node_client_id == 91, "node client id 91") + need(check_client_id is None or type(check_client_id) is int and check_client_id == 92, "check client id 92") + inst = plan["instrument"] + need(tuple(inst[k] for k in ("symbol", "sec_type", "exchange", "primary_exchange", "currency")) + == ("SPY", "STK", "SMART", "ARCA", "USD"), "SPY STK SMART/ARCA USD only") + need(inst["nautilus_instrument_id"] == "SPY=STK.SMART" and inst["expected_price_increment"] == "0.01", "RAW stock identity and tick") + b = plan["bounds"] + need(type(b["max_orders"]) is int and 3 <= b["max_orders"] <= 6, "order budget 3..6") + need(type(b["max_quantity_per_order"]) is int and b["max_quantity_per_order"] == 1, "quantity 1") + for name, ceiling in (("max_notional_per_order_usd", 1000), ("max_roundtrip_loss_usd", 5)): + need(not isinstance(b[name], bool) and 0 < decimal(b[name]) <= ceiling, name) + need(decimal(b["commission_allowance_per_order_usd"]) >= 0, "commission allowance") + s = plan["session"] + need((s["timezone"], s["open"], s["close"]) == ("America/New_York", "09:30", "16:00"), "regular trading hours") + need(s["weekdays_only"] is True and s["whole_run_inside_window"] is True + and s["use_contract_liquid_hours"] is True, "whole run with broker liquid hours") + need(type(s["close_buffer_minutes"]) is int and s["close_buffer_minutes"] >= 10, "10 minute close buffer") + expected_times = {"check_seconds": 30, "node_start_seconds": 60, "per_step_seconds": 45, + "cleanup_seconds": 45, "node_stop_allowance_seconds": 15, + "post_check_reserve_seconds": 60, "overall_deadline_seconds": 420} + need(plan["timeouts"] == expected_times and all(type(v) is int for v in plan["timeouts"].values()), + "frozen timeouts and hard deadline") + r = plan["risk"] + need(r["bypass"] is False and r["require_enforced_notional_before_connection"] is False, + "native risk configured with runner quote admission") + need(not isinstance(r["notional_headroom_usd"], bool) + and decimal(r["notional_headroom_usd"]) == 10, "USD 10 quote admission headroom") + need(r["max_order_submit_rate"] == f"{b['max_orders']}/00:07:00", "whole-run native submit throttle") + # No configurable strategy knobs can quietly turn this into another trial. + need(plan["exec_tester"] == TESTER_SETTINGS + and all(type(plan["exec_tester"][k]) is type(v) for k, v in TESTER_SETTINGS.items()), + "frozen ExecTester settings") + need(plan["data"] == {"market_data_type": "REALTIME", "quote_max_age_seconds": 10, + "clock_resolution_tolerance_seconds": 2, "batch_quotes": False}, "real-time quote bounds") + need(plan["cases"] == CASE_DEFINITIONS, "frozen cases C1..C4") + need(plan["end_disposition"] == {"positions": 0, "open_orders": 0}, "flat disposition") + need(plan["receipt"]["kind"] == KIND and plan["evidence_class"] == "native_paper", "receipt identity") + except (KeyError, TypeError, ValueError, InvalidOperation): + errors.append("missing or malformed plan field") + return errors + + +def runtime_versions(): + def version(package): + try: + return importlib.metadata.version(package) + except importlib.metadata.PackageNotFoundError: + return None + try: + import ibapi + ib_version = ibapi.__version__ + except (ImportError, AttributeError): + ib_version = None + try: + import nautilus_trader + engine_version = nautilus_trader.__version__ + except (ImportError, AttributeError): + engine_version = None + return {"nautilus_trader": engine_version, "nautilus_trader_distribution": version("nautilus_trader"), "ibapi": ib_version, + "nautilus_ibapi_distribution": version("nautilus-ibapi"), + "protobuf": version("protobuf"), "python": platform.python_version()} + + +def runtime_refusal(versions): + if versions.get("nautilus_trader") != "2.0.0rc5" or versions.get("ibapi") != "10.45.1": + return "refused_unpinned_runtime" + if "nautilus_trader_distribution" in versions and versions["nautilus_trader_distribution"] != "2.0.0rc5": + return "refused_unpinned_runtime" + return None + + +def resting_offset_ticks(bid, plan): + """Frozen half-bid placement through upstream ExecTester's supported offset.""" + tick = decimal(plan["instrument"]["expected_price_increment"]) + return int((decimal(bid) * Decimal("0.5") / tick).to_integral_value(rounding=ROUND_FLOOR)) + + +def closing_period_admission(plan, start, *, liquid_sessions=None): + """Use the frozen window's session end for closing-band admission. + + node_stop_at reserves proof, cleanup and stop allowance from the overall + deadline; adding the close horizon restores cleanup and stop allowance. + Source: the NMS LULD Plan's Tier 1 last-25-minutes double-width band. + The plan supplies the preliminary end; broker liquidHours narrows it using + ../ibkr-paper-orders/run.py session_window, also used by rth_check. + """ + times = plan["timeouts"] + node_stop_seconds = (times["overall_deadline_seconds"] - times["post_check_reserve_seconds"] + - times["cleanup_seconds"] - times["node_stop_allowance_seconds"]) + close_horizon = times["cleanup_seconds"] + times["node_stop_allowance_seconds"] + horizon = node_stop_seconds + close_horizon + start_et = start.astimezone(frozen().ZoneInfo("America/New_York")) + end_et = start_et + timedelta(seconds=horizon) + result = {"status": "passed", "stage": "luld_tier1_closing_period_band", + "order_horizon_seconds": horizon, "closing_band_fraction": str(LULD_TIER1_CLOSING_PERIOD_BAND), + "planned_order_window_end": end_et.isoformat(), + "session_source": "plan_session" if liquid_sessions is None else "broker_liquid_hours"} + window = frozen().session_window(start, plan, liquid_sessions) + if window is None: + result.update(status="refused_liquid_hours", reason="no_usable_liquid_session") + return result + # session_window returns close minus the plan's buffer; restore that buffer + # to derive the same liquid end, including early closes, without reselecting. + liquid_end = (window[1] + timedelta(minutes=plan["session"]["close_buffer_minutes"])).astimezone(start_et.tzinfo) + closing_start = liquid_end - timedelta(minutes=LULD_TIER1_CLOSING_PERIOD_MINUTES) + result.update(liquid_session_end=liquid_end.isoformat(), closing_period_start=closing_start.isoformat()) + if end_et >= closing_start: + result.update(status="refused_luld_closing_period", reason="planned_order_window_overlaps_luld_closing_period") + return result + + +def admit_quote(plan, quote, server_time_epoch, *, elapsed_seconds=0, delayed=False, errors=()): + """Pure admission using IB's timestamped BidAsk and its independent clock.""" + result = {"source": "ibapi.reqTickByTickData.BidAsk/reqCurrentTime", "status": None, + "bid": None, "ask": None, "quote_age_seconds": None} + if delayed or any(code in (10089, 10167) for code in errors): + result["status"] = "refused_delayed_quote" + return result + if errors: + result["status"] = "refused_quote_error" + return result + try: + bid, ask = decimal(quote["bid"]), decimal(quote["ask"]) + age = decimal(server_time_epoch) - decimal(quote["quote_time_epoch"]) + decimal(elapsed_seconds) + result.update(bid=str(bid), ask=str(ask), quote_time_epoch=int(quote["quote_time_epoch"]), + server_time_epoch=int(server_time_epoch), quote_age_seconds=float(max(age, Decimal(0)))) + if bid <= 0 or ask < bid: + result["status"] = "refused_invalid_quote" + elif age < -decimal(plan["data"]["clock_resolution_tolerance_seconds"]) or age > decimal(plan["data"]["quote_max_age_seconds"]): + result["status"] = "refused_stale_quote" + else: + notional = ask * plan["bounds"]["max_quantity_per_order"] + headroom = decimal(plan["risk"]["notional_headroom_usd"]) + band_notional = notional * (1 + LULD_TIER1_CORE_BAND) + result.update(ask_notional_usd=str(notional), headroom_usd=str(headroom), + luld_core_band_fraction=str(LULD_TIER1_CORE_BAND), band_adjusted_notional_usd=str(band_notional), + tob_offset_ticks=resting_offset_ticks(bid, plan)) + spread_loss = (ask - bid) * plan["bounds"]["max_quantity_per_order"] + commissions = 2 * (decimal(plan["bounds"]["commission_allowance_per_order_usd"]) + COMMISSION_MARGIN_PER_ORDER_USD) + loss_bound = decimal(plan["bounds"]["max_roundtrip_loss_usd"]) + result.update(spread_loss_usd=str(spread_loss), expected_roundtrip_commissions_usd=str(commissions), + commission_margin_per_order_usd=str(COMMISSION_MARGIN_PER_ORDER_USD), + expected_roundtrip_loss_usd=str(spread_loss + commissions), max_roundtrip_loss_usd=str(loss_bound)) + if spread_loss + commissions > loss_bound: + result.update(status="refused_quote_loss_bound", stage="spread_and_commissions") + elif band_notional + headroom > decimal(plan["bounds"]["max_notional_per_order_usd"]) or result["tob_offset_ticks"] <= 0: + result.update(status="refused_quote_notional", stage="notional") + else: + result.update(status="passed", stage="admitted") + except (KeyError, TypeError, ValueError, InvalidOperation): + result["status"] = "refused_invalid_quote" + return result + + +# Official ibapi 10.45.1 client.py reqTickByTickData and wrapper.py callbacks +# were inspected in the prepared runtime. Delayed enums come from ticktype.py. +DELAYED_TICKS = set(range(66, 77)) | {80, 81, 82, 83, 88, 90, 103, 104} +CONNECTIVITY_CODES = {326, 502, 503, 504, 507, 1100, 1300, 2110} + + +class QuoteState: + """Read-only callback state, kept independent of ibapi for synthetic tests.""" + + def __init__(self): + self.ready = threading.Event() + self.quote_ready = threading.Event() + self.clock_ready = threading.Event() + self.accounts_ready = threading.Event() + self.accounts = [] # Memory only. + self.quote = None + self.server_time = None + self.clock_received = None + self.delayed = False + self.errors = [] + self.info = [] + self.started_monotonic = time.monotonic() + self.last_stage = "initializing" + self.stage_elapsed_seconds = {} + self.stage_duration_seconds = {} + self.stage_started = self.started_monotonic + self.quote_phase = False + + def mark(self, stage): + if stage not in self.stage_elapsed_seconds: + now = time.monotonic() + self.stage_elapsed_seconds[stage] = round(now - self.started_monotonic, 4) + self.stage_duration_seconds[stage] = round(now - self.stage_started, 4) + self.stage_started, self.last_stage = now, stage + + def diagnostics(self): + return {"last_stage": self.last_stage, "elapsed_seconds": round(time.monotonic() - self.started_monotonic, 4), + "last_stage_elapsed_seconds": round(time.monotonic() - self.stage_started, 4), + "stage_elapsed_seconds": dict(self.stage_elapsed_seconds), + "stage_duration_seconds": dict(self.stage_duration_seconds), + "errors": list(self.errors), "info": list(self.info)} + + def nextValidId(self, orderId): + self.mark("nextValidId") + self.ready.set() + + def managedAccounts(self, accountsList): + self.accounts = [value.strip() for value in (accountsList or "").split(",") if value.strip()] + self.mark("accounts") + self.accounts_ready.set() + + def tickByTickBidAsk(self, reqId, timestamp, bidPrice, askPrice, bidSize, askSize, tickAttribBidAsk): + if reqId == 9202: + self.quote = {"bid": str(bidPrice), "ask": str(askPrice), "quote_time_epoch": timestamp} + self.mark("quote") + self.quote_ready.set() + + def currentTime(self, timestamp): + self.server_time, self.clock_received = timestamp, time.monotonic() + if self.quote_phase: + self.mark("clock") + self.clock_ready.set() + + def error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson=""): + if errorCode in (10089, 10167): + self.delayed = True + entry = {"reqId": reqId, "code": errorCode, + "text": frozen().scrub_serialized(frozen().redact(errorString)[:160], self.accounts)} + fatal = reqId == 9202 or errorCode in CONNECTIVITY_CODES or errorCode in (10089, 10167) + (self.errors if fatal else self.info).append(entry) + if fatal: + self.quote_ready.set() + self.clock_ready.set() + if errorCode in CONNECTIVITY_CODES: + self.ready.set() + self.accounts_ready.set() + + def marketDataType(self, reqId, marketDataType): + if marketDataType in (3, 4): + self.delayed = True + self.quote_ready.set() + + def delayed_tick(self, tickType): + if tickType in DELAYED_TICKS: + self.delayed = True + self.quote_ready.set() + + def tickPrice(self, reqId, tickType, price, attrib): + self.delayed_tick(tickType) + + def tickSize(self, reqId, tickType, size): + self.delayed_tick(tickType) + + def tickString(self, reqId, tickType, value): + self.delayed_tick(tickType) + + def tickGeneric(self, reqId, tickType, value): + self.delayed_tick(tickType) + + def tickOptionComputation(self, reqId, tickType, *values): + self.delayed_tick(tickType) + + +class AdmissionState(frozen().CheckState, QuoteState): + """Reuse frozen flat-check callbacks and quote callbacks on one session.""" + + def __init__(self): + frozen().CheckState.__init__(self) + QuoteState.__init__(self) + + def managedAccounts(self, accountsList): + frozen().CheckState.managedAccounts(self, accountsList) + QuoteState.managedAccounts(self, accountsList) + + def currentTime(self, timestamp): + frozen().CheckState.currentTime(self, timestamp) + QuoteState.currentTime(self, timestamp) + + def error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson=""): + frozen().CheckState.error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson) + QuoteState.error(self, reqId, errorTime, errorCode, errorString, advancedOrderRejectJson) + + +def build_admission_client(): + from ibapi.client import EClient + from ibapi.wrapper import EWrapper + + class QuoteClient(AdmissionState, EWrapper, EClient): + def __init__(self): + AdmissionState.__init__(self) + EClient.__init__(self, self) + + return QuoteClient() + + +class CheckSession: + """Hold the frozen check's session until quote admission and join its reader. + + Official ibapi 10.45.1 client.py: connect creates EReader and startApi; + run drains the shared message queue; disconnect closes/reset but does not + join those threads. Intercept only the frozen harness's final disconnect, + leaving EClient's own disconnect on connection loss unchanged. + """ + + def __init__(self, client): + self.client, self.runner = client, None + + def __getattr__(self, name): + return getattr(self.client, name) + + def connect(self, host, port, client_id): + self.client.connect(host, port, client_id) + if self.client.isConnected(): + self.client.mark("connected") + + def run(self): + self.runner = threading.current_thread() + self.client.run() + + def disconnect(self): + pass # The outer admission scope owns teardown after both read phases. + + def close(self, deadline): + reader = getattr(self.client, "reader", None) + try: + self.client.disconnect() + finally: + for thread in (self.runner, reader): + if thread is not None and thread is not threading.current_thread(): + thread.join(timeout=max(0, deadline - time.monotonic())) + + +def quote_check(plan, account_id, *, deadline, client): + """Continue the already-connected flat-check client; never reconnect.""" + requested = False + + def wait(event): + while not event.is_set() and time.monotonic() < deadline: + if client.errors or not client.isConnected(): + return False + event.wait(min(0.05, max(0, deadline - time.monotonic()))) + return event.is_set() + + def result(value): + return {**value, **client.diagnostics()} + + def failed_wait(status): + if client.errors or client.delayed: + return result(admit_quote(plan, client.quote, client.server_time, + delayed=client.delayed, errors=[entry["code"] for entry in client.errors])) + return result({"status": status, "error": "quote_check_deadline" if time.monotonic() >= deadline else "connection_lost"}) + + try: + if not client.isConnected(): + return result({"status": "not_connected"}) + if not wait(client.ready) or not wait(client.accounts_ready): + return failed_wait("not_connected") + if client.errors: + return failed_wait("not_connected") + if client.accounts != [account_id] or not account_id.startswith("DU"): + return result({"status": "refused_quote_account_scope"}) + client.quote_phase = True + client.reqMarketDataType(1) + requested = True + client.mark("tick requested") + client.reqTickByTickData(9202, frozen().spy_contract(), "BidAsk", 0, False) + if not wait(client.quote_ready): + return failed_wait("refused_quote_unavailable") + if client.delayed or client.errors: + return failed_wait("refused_quote_error") + client.clock_ready.clear() # Discard the clock from the flat pre-check. + client.server_time, client.clock_received = None, None + client.reqCurrentTime() + if not wait(client.clock_ready): + return failed_wait("refused_quote_clock") + elapsed = max(0, time.monotonic() - client.clock_received) if client.clock_received is not None else 0 + admitted = admit_quote(plan, client.quote, client.server_time, elapsed_seconds=elapsed, + delayed=client.delayed, errors=[entry["code"] for entry in client.errors]) + admitted["admitted_monotonic"] = time.monotonic() + return result(admitted) + finally: + if requested: + client.cancelTickByTickData(9202) + + +def validate_child_admission(plan, payload, *, monotonic_now=None): + """Recompute age, cap and offset from the pipe; no account in diagnostics.""" + try: + admitted = payload["quote_admission"] + now = time.monotonic() if monotonic_now is None else monotonic_now + elapsed = decimal(now) - decimal(admitted["admitted_monotonic"]) + if admitted["status"] != "passed" or elapsed < 0: + return False + # Stored age already includes time since the independent server callback. + rechecked = admit_quote(plan, admitted, decimal(admitted["quote_time_epoch"]) + decimal(admitted["quote_age_seconds"]), + elapsed_seconds=elapsed) + offset = payload["tob_offset_ticks"] + return (rechecked["status"] == "passed" and type(offset) is int + and offset == resting_offset_ticks(admitted["bid"], plan) + and offset == admitted["tob_offset_ticks"] + and isinstance(payload["account_id"], str) and payload["account_id"].startswith("DU")) + except (KeyError, TypeError, ValueError, InvalidOperation): + return False + + +def sanitize(value): + """Prices/quantities remain structured; all untrusted text uses frozen redaction.""" + if isinstance(value, dict): + return {frozen().redact(k): sanitize(v) for k, v in value.items()} + if isinstance(value, (list, tuple)): + return [sanitize(v) for v in value] + if isinstance(value, str): + # Constant source locators are public evidence, not untrusted error text. + public_sources = {RISK_ROUTE["source"], RISK_ROUTE["issue"], RELEASE_VERIFICATION["source"]} + public_sources.update(issue["url"] for step in BLOCKED_STEPS for issue in step["issues"]) + if value in public_sources: + return frozen().scrub_serialized(value) + return frozen().redact(value) + return value + + +def serialized_receipt(receipt, secret_values=()): + text = json.dumps(sanitize(receipt), indent=2, sort_keys=True, allow_nan=False) + # Final id-shaped-text pass also covers dictionary keys and unexpected nested fields. + return frozen().scrub_serialized(text, secret_values) + "\n" + + +def write_receipt(path, receipt, secret_values=(), *, replace=True): + path = Path(path) + protected = {PLAN_PATH.resolve(), Path(__file__).resolve(), FROZEN_RUN.resolve()} + if path.resolve() in protected or frozen().is_gate_receipt(path): + raise ValueError("protected receipt destination") + if not replace and path.exists(): + return False + raw = serialized_receipt(receipt, secret_values) + path.parent.mkdir(parents=True, exist_ok=True) + temporary = None + try: + with tempfile.NamedTemporaryFile(mode="w", dir=path.parent, prefix=".paper-receipt-", delete=False) as stream: + temporary = Path(stream.name) + stream.write(raw) + if replace: + os.replace(temporary, path) + else: + # Publish complete bytes without replacing a receipt created by + # another parent. Python os.link provides atomic create-if-absent. + try: + os.link(temporary, path) + except FileExistsError: + return False + return True + finally: + if temporary is not None: + temporary.unlink(missing_ok=True) + + +def case_outcomes(events, *, stop_ns=None, flat_proof=None): + """Pure mapping of sanitized native cache events; no strings imply acceptance. + + Each event carries an opaque local order alias and its type/side/quantity/price. + C2 must cancel C1, and C4 must fill after stop AND have an independent flat proof. + Duplicate native event ids are removed in snapshot_events, before this mapping. + """ + definitions = (("LIMIT", "BUY"), ("LIMIT", "BUY"), ("MARKET", "BUY"), ("MARKET", "SELL")) + result = {cid: {"id": cid, "order_type": typ, "side": side, "price": None, + "order": None, "events": [], "outcome": "not_run", "reason": None} + for cid, (typ, side) in zip(CASE_IDS, definitions)} + aliases = {} + for event in events: + aliases.setdefault(event["order"], event) + chosen = {} + for cid, shape in (("C1", ("LIMIT", "BUY")), ("C3", ("MARKET", "BUY")), ("C4", ("MARKET", "SELL"))): + candidates = [key for key, e in aliases.items() if (e["order_type"], e["side"]) == shape] + if candidates: + chosen[cid] = candidates[0] + if "C1" in chosen: + chosen["C2"] = chosen["C1"] + for cid, alias in chosen.items(): + case = result[cid] + seq = [e for e in events if e["order"] == alias] + case.update(order=alias, price=seq[0].get("price"), events=seq, outcome="incomplete") + confirmed = [e for e in seq if not e.get("reconciliation", False)] + bad = [e for e in seq if e["type"] in TERMINAL_FAILURES or e.get("reconciliation", False)] + if bad: + case.update(outcome="failed", reason=f"{cid}_unconfirmed_or_rejected") + continue + if decimal(seq[0]["quantity"]) != 1: + case.update(outcome="failed", reason=f"{cid}_quantity_bound") + continue + if cid == "C1": + if any(e["type"] == "OrderFilled" for e in confirmed): + case.update(outcome="failed", reason="C1_resting_order_filled") + elif any(e["type"] == "OrderAccepted" for e in confirmed): + case["outcome"] = "passed" + elif cid == "C2": + accepted = [e for e in confirmed if e["type"] == "OrderAccepted"] + canceled = [e for e in confirmed if e["type"] == "OrderCanceled"] + if stop_ns is not None and accepted and any(e["ts_init"] >= stop_ns and e["ts_init"] >= accepted[0]["ts_init"] for e in canceled): + case["outcome"] = "passed" + else: + fills = [e for e in confirmed if e["type"] == "OrderFilled"] + filled = sum((decimal(e["fill_quantity"]) for e in fills), Decimal(0)) + if filled > 1: + case.update(outcome="failed", reason=f"{cid}_overfill") + elif filled == 1: + if cid == "C3": + case["outcome"] = "passed" + elif stop_ns is not None and all(e["ts_init"] >= stop_ns for e in fills): + case["outcome"] = "filled_unproven" + if flat_proof and flat_proof.get("status") == "passed": + observed = flat_proof.get("observed", {}) + if observed.get("positions") == 0 and observed.get("open_orders") == 0: + case["outcome"] = "passed" + if case["outcome"] == "incomplete": + case["reason"] = f"{cid}_expected_event_missing" + return result + + +def roundtrip(events, loss_bound): + fills = [e for e in events if e["type"] == "OrderFilled"] + if not fills: + return None + buys = sum((decimal(e["fill_quantity"]) for e in fills if e["side"] == "BUY"), Decimal(0)) + sells = sum((decimal(e["fill_quantity"]) for e in fills if e["side"] == "SELL"), Decimal(0)) + gross = sum((decimal(e["fill_price"]) * decimal(e["fill_quantity"]) * (1 if e["side"] == "SELL" else -1) for e in fills), Decimal(0)) + unresolved = any(e.get("commission") is None or decimal(e["commission"]) <= 0 + or e.get("commission_currency") != "USD" or e.get("currency") != "USD" for e in fills) + fees = None if unresolved else sum((decimal(e["commission"]) for e in fills), Decimal(0)) + closed = buys == sells == 1 + net = gross - fees if fees is not None and closed else None + return {"buy_quantity": str(buys), "sell_quantity": str(sells), "closed": closed, + "gross_usd": str(gross) if buys == sells else None, + "commissions_usd": str(fees) if fees is not None else None, + "net_usd": str(net) if net is not None else None, + "commission_unresolved": unresolved, + "loss_bound_breach": net < -decimal(loss_bound) if net is not None else None} + + +def new_receipt(plan, plan_path, versions): + if not isinstance(plan, dict): + plan = {} + bounds = plan.get("bounds") if isinstance(plan.get("bounds"), dict) else {} + try: + cap = str(decimal(bounds["max_notional_per_order_usd"])) + except (KeyError, ValueError, InvalidOperation): + cap = None + try: + plan_hash = sha256(plan_path) + except OSError: + plan_hash = None + return {"schema_version": 1, "kind": KIND, "evidence_class": "native_paper", + "run_nonce": uuid.uuid4().hex, + "started_at": datetime.now(timezone.utc).isoformat(), "versions": versions, + "plan_sha256": plan_hash, "harness_sha256": sha256(__file__), + "source_hashes": {"frozen_checker_sha256": sha256(FROZEN_RUN), "safety_sha256": sha256(HERE / "safety.py")}, + "upstream_commit": UPSTREAM_COMMIT, "pre_check": None, "quote_admission": None, + "closing_period_admission": None, + "cases": case_outcomes([]), "events": [], "reconciled_external": [], "fills": [], "roundtrip": None, + "flat_proof": None, "node": {"started": False, "child_launched": False, "stop_ns": None}, + "account_lease": {"acquired": False}, "journal": None, + "risk": {"bypass": False, "max_notional_per_order_usd": cap, + "enforcement": "runner_quantity_and_quote_admission", "note": RISK_NOTE, + "engine_route": RISK_ROUTE, "submit_budget_enforcement": "engine_throttle", + "release_verification": RELEASE_VERIFICATION}, + "blocked_steps": BLOCKED_STEPS, "failures": [], + "status": "cleanup_required", "exit_code": 3} + + +def finish(receipt, status, *, reason=None, step=2, cases=CASE_IDS): + receipt["status"] = status + receipt["exit_code"] = 3 if status == "not_started" else frozen().exit_code_for(status) + receipt["ended_at"] = datetime.now(timezone.utc).isoformat() + if reason: + receipt["failures"].append({"acceptance_step": step, "cases_blocked": list(cases), "reason": reason}) + return receipt["exit_code"] + + +def native_configs(plan, port, account_id, *, tob_offset_ticks): + """Only upstream configs, with installed rc5 signatures verified offline. + + Source: v2.0.0rc5 examples/live/interactive_brokers/{exec_tester,_common}.py. + Construction does not connect. Actual account_id is supplied by the official + pre-check in memory, never from TWS_ACCOUNT or a command line argument. + """ + from nautilus_trader.adapters import interactive_brokers as ib + from nautilus_trader.config import LiveRiskEngineConfig + from nautilus_trader.model import ClientId, InstrumentId, Quantity, StrategyId, TimeInForce + from nautilus_trader.testkit import ExecTesterConfig + instrument = InstrumentId.from_str(plan["instrument"]["nautilus_instrument_id"]) + provider = ib.InteractiveBrokersInstrumentProviderConfig(symbology_method=ib.SymbologyMethod.RAW, load_ids={instrument}) + common = dict(host=plan["host"], port=port, client_id=plan["client_ids"]["node"], + connection_timeout=10, request_timeout=30, instrument_provider=provider) + tester_values = dict(plan["exec_tester"]) + tester_values["tob_offset_ticks"] = tob_offset_ticks + tester_values["order_qty"] = Quantity.from_str(tester_values["order_qty"]) + tester_values["open_position_on_start_qty"] = Decimal(tester_values["open_position_on_start_qty"]) + for key in ("open_position_time_in_force", "limit_time_in_force"): + tester_values[key] = TimeInForce.from_str(tester_values[key]) + tester = ExecTesterConfig(strategy_id=StrategyId.from_str("EXEC_TESTER-001"), instrument_id=instrument, + client_id=ClientId.from_str("IB"), external_order_instrument_ids=[instrument], + subscribe_quotes=True, subscribe_trades=True, **tester_values) + risk = LiveRiskEngineConfig(bypass=False, max_order_submit_rate=plan["risk"]["max_order_submit_rate"], + max_notional_per_order={str(instrument): str(plan["bounds"]["max_notional_per_order_usd"])}) + return (ib, ib.InteractiveBrokersDataClientConfig(market_data_type=ib.MarketDataType.REALTIME, batch_quotes=False, **common), + ib.InteractiveBrokersExecutionClientConfig(account_id=account_id, **common), risk, tester) + + +def build_node(plan, port, account_id, *, tob_offset_ticks): + from nautilus_trader.common import Environment, LoggerConfig, LogLevel + from nautilus_trader.live import LiveNode + from nautilus_trader.model import TraderId + ib, data, execution, risk, tester = native_configs(plan, port, account_id, tob_offset_ticks=tob_offset_ticks) + node = (LiveNode.builder("IB-EXEC-TESTER-001", TraderId.from_str("TESTER-001"), Environment.LIVE) + .with_reconciliation(True).with_risk_engine_config(risk) + .with_logging(LoggerConfig(stdout_level=LogLevel.OFF, fileout_level=LogLevel.OFF, print_config=False)) + .with_timeout_connection(plan["timeouts"]["node_start_seconds"]) + .with_timeout_reconciliation(5).with_timeout_portfolio(5) + .with_timeout_disconnection_secs(5).with_delay_post_stop_secs(plan["timeouts"]["cleanup_seconds"]) + .add_data_client(None, ib.InteractiveBrokersDataClientFactory(), data) + .add_exec_client(None, ib.InteractiveBrokersExecutionClientFactory(), execution).build()) + node.add_builtin_strategy("ExecTester", tester) + return node + + +class RunOrderScope: + """In-memory client-order-id lineage, including pending local intents. + + Pinned upstream: crates/execution/src/engine/mod.rs materializes and claims + external orders using this strategy id and the current initialization time. + Neither alone proves ownership. IB execution/core_orders.rs emits a local + OrderSubmitted with the original client_order_id on this run's submit path. + The supported factory's O-date-time-trader_tag-strategy_tag-counter identity + identifies local intents before submission; its creation second must belong + to this launch and match initialization. Pinned engine/mod.rs (1176, 1273) + and live/src/execution/manager.rs (4379) set reconciliation=True on external + materialization. model/src/orders/market.rs TryFrom (533) + then calls new_checked, which rebuilds initialization with False (99-108). + This explains the external MARKET init flags in historical cache receipts; + it does not describe the original materialization event. Identity checks + exclude old orders independently. Source: + crates/common/src/generators/client_order_id.rs write_fixed_prefix/generate. + Also retain the matching local submission lineage for already sent orders. + Retain ownership if later events reconcile, so the never-pass rule applies. + Raw identifiers stay here; receipts use stable O/X aliases only. + """ + + def __init__(self, started_ns): + self.started_ns = started_ns + self.owned = {} + self.external = {} + + def generated_here(self, event): + parts = str(event.client_order_id).split("-") + if len(parts) != 6 or parts[0] != "O" or parts[3:5] != ["001", "001"] or not parts[5].isdigit() or int(parts[5]) <= 0: + return False + try: + second = int(datetime.strptime(parts[1] + parts[2], "%Y%m%d%H%M%S").replace(tzinfo=timezone.utc).timestamp()) + except ValueError: + return False + return second >= self.started_ns // 1_000_000_000 and 0 <= event.ts_init // 1_000_000_000 - second <= 1 + + def initialized_here(self, order): + if str(order.strategy_id) != "EXEC_TESTER-001" or str(order.trader_id) != "TESTER-001": + return False + initialized = submitted = generated = False + for event in order.events(): + if (str(event.client_order_id) != str(order.client_order_id) + or str(event.strategy_id) != str(order.strategy_id) + or str(event.trader_id) != str(order.trader_id) + or event.ts_init < self.started_ns + or getattr(event, "reconciliation", False)): + continue + initialized |= type(event).__name__ == "OrderInitialized" + submitted |= type(event).__name__ == "OrderSubmitted" + if type(event).__name__ == "OrderInitialized": + generated |= self.generated_here(event) + return initialized and (generated or submitted) + + def partition(self, orders): + own, other = [], [] + for order in sorted(orders, key=lambda order: (order.ts_init, str(order.client_order_id))): + key = str(order.client_order_id) + if key not in self.owned and self.initialized_here(order): + self.owned[key] = f"O{len(self.owned) + 1}" + if key in self.owned: + own.append(order) + else: + self.external.setdefault(key, f"X{len(self.external) + 1}") + other.append(order) + return own, other + + +def tester_orders(cache, scope): + return scope.partition(cache.orders())[0] + + +def snapshot_events(cache, scope, *, external=False): + """Read cloned order histories, never serialize account/order/venue/trade ids.""" + orders = scope.partition(cache.orders())[1 if external else 0] + orders.sort(key=lambda order: (order.ts_init, str(order.client_order_id))) + events, seen = [], set() + for order in orders: + for event in order.events(): + key = str(event.event_id) + if key in seen: + continue + seen.add(key) + price = getattr(order, "price", None) + alias = (scope.external if external else scope.owned)[str(order.client_order_id)] + record = {"order": alias, "order_type": order.order_type.name, "side": order.side.name, + "instrument_id": str(order.instrument_id), + "quantity": str(order.quantity), "price": str(price) if price is not None else None, + "type": type(event).__name__, "ts_event": int(event.ts_event), "ts_init": int(event.ts_init), + "at": frozen().ns_to_iso(event.ts_event), "reconciliation": bool(getattr(event, "reconciliation", False))} + if record["type"] == "OrderFilled": + commission = event.commission + record.update(fill_price=str(event.last_px), fill_quantity=str(event.last_qty), currency=str(event.currency), + commission=str(commission.as_decimal()) if commission is not None else None, + commission_currency=str(commission.currency) if commission is not None else None) + if external: + record.pop("instrument_id") + if not external and hasattr(event, "reason"): + record["reason"] = frozen().redact(event.reason) + events.append(record) + return sorted(events, key=lambda event: (event["ts_init"], event["ts_event"])) + + +def update_observations(receipt, events, plan): + receipt["events"] = events + receipt["cases"] = case_outcomes(events, stop_ns=receipt["node"]["stop_ns"], flat_proof=receipt["flat_proof"]) + receipt["fills"] = [e for e in events if e["type"] == "OrderFilled"] + receipt["roundtrip"] = roundtrip(events, plan["bounds"]["max_roundtrip_loss_usd"]) + orders = {e["order"]: e for e in events} + violations = [] + if len(orders) > plan["bounds"]["max_orders"]: + violations.append(("observed_order_budget_breach", list(CASE_IDS))) + for e in orders.values(): + cases = ["C1", "C2"] if e["order_type"] == "LIMIT" else ["C3" if e["side"] == "BUY" else "C4"] + quantity = decimal(e["quantity"]) + if quantity != 1: + violations.append(("observed_quantity_bound_breach", cases)) + if e.get("instrument_id") != plan["instrument"]["nautilus_instrument_id"]: + violations.append(("observed_instrument_bound_breach", cases)) + if e.get("price") is not None and decimal(e["price"]) * quantity > decimal(plan["bounds"]["max_notional_per_order_usd"]): + violations.append(("observed_notional_bound_breach", cases)) + for e in events: + if e["type"] == "OrderFilled" and decimal(e["fill_price"]) * decimal(e["fill_quantity"]) > decimal(plan["bounds"]["max_notional_per_order_usd"]): + violations.append(("observed_fill_notional_bound_breach", ["C3" if e["side"] == "BUY" else "C4"])) + for reason, cases in violations: + if not any(f["reason"] == reason and f["cases_blocked"] == cases for f in receipt["failures"]): + receipt["failures"].append({"reason": reason, "cases_blocked": cases, "acceptance_step": 2}) + + +def orders_in_flight(orders): + """Upstream close-on-stop runs once; wait for entry resolution before stop.""" + terminal = {"FILLED", "CANCELED", "EXPIRED", "REJECTED", "DENIED"} + return any(order.status.name not in terminal and + (order.order_type.name == "MARKET" or order.time_in_force.name in ("IOC", "FOK") + or order.status.name == "SUBMITTED" or order.venue_order_id is None) for order in orders) + + +def stop_ready(orders, now, node_stop_at): + return now >= node_stop_at or not orders_in_flight(orders) + + +def journal_order_events(cache, scope, journal, account_id): + """Retain original native identities privately; cache events preserve intent order. + + This is observation after native dispatch, not a pre-dispatch fsync barrier. + rc5's Rust BusTap provides that barrier (common/src/msgbus/mod.rs), but is + not exposed by the selected Python surface. Do not wrap ExecTester submits. + """ + for order in tester_orders(cache, scope): + for event in order.events(): + kind = type(event).__name__ + venue_id = None if kind == "OrderInitialized" else getattr(event, "venue_order_id", None) or order.venue_order_id + record = {"kind": kind, "event_id": str(event.event_id), "client_order_id": str(order.client_order_id), + "venue_order_id": str(venue_id) if venue_id is not None else None, + "order_type": order.order_type.name, "side": order.side.name, "quantity": str(order.quantity), + "price": str(order.price) if getattr(order, "price", None) is not None else None, + "ts_init": int(event.ts_init), "ts_event": int(event.ts_event), + "reconciliation": bool(getattr(event, "reconciliation", False))} + if kind == "OrderFilled": + commission = event.commission + record.update(fill_price=str(event.last_px), fill_quantity=str(event.last_qty), + commission=str(commission.as_decimal()) if commission is not None else None, + commission_currency=str(commission.currency) if commission is not None else None) + if hasattr(event, "reason"): + record["reason"] = frozen().scrub_serialized(frozen().redact(event.reason), (account_id,)) + journal.event(record) + + +async def node_phase(plan, port, receipt_path, payload): + """Hosted upstream run: cache + thread-safe handle captured before run_async. + + Source: crates/live/src/python/node.rs at UPSTREAM_COMMIT. The parent owns + the hard deadline; OS process termination cannot strand this awaitable. + """ + if not validate_child_admission(plan, payload): + return persist_child_refusal(receipt_path, payload, "refused_child_quote_admission") + receipt, account_id = payload["receipt"], payload["account_id"] + journal = None + try: + if payload.get("journal_path"): + journal = safety().Journal(Path(payload["journal_path"]).parent, payload["run_id"], ROOT, create=False) + except (safety().SafetyError, OSError, TypeError, ValueError): + return persist_child_refusal(receipt_path, payload, "refused_child_journal") + node = build_node(plan, port, account_id, tob_offset_ticks=payload["tob_offset_ticks"]) + # Live view: node.rs py_cache uses PyCache::from_rc(kernel.cache()); + # common/src/python/cache.rs PyCache(Rc>) borrows on each + # orders() call. Individual returned orders are clones; re-read each poll. + cache, handle = node.cache, node.handle() + loop = asyncio.get_running_loop() + pending_stop = None + registered_signals = [] + task = None + scope = RunOrderScope(time.time_ns()) + + def observe(): + if journal is not None: + journal_order_events(cache, scope, journal, account_id) + update_observations(receipt, snapshot_events(cache, scope), plan) + receipt["reconciled_external"] = snapshot_events(cache, scope, external=True) + if journal is not None: + receipt["journal"] = journal.summary() + + def stop(reason): + if receipt["node"]["stop_ns"] is None: + receipt["node"].update(stop_ns=time.time_ns(), stop_reason=reason) + try: + if journal is not None: + journal.append("stop_request", reason=reason, stop_ns=receipt["node"]["stop_ns"]) + except Exception as exc: + record_error("journal_stop", exc) + finally: + handle.stop() + + def request_stop(reason): + nonlocal pending_stop + pending_stop = pending_stop or reason + + def record_error(step, exc): + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), + "reason": frozen().redact(f"{step}: {type(exc).__name__}: {exc}")}) + + observed_start = None + try: + for sig in SIGNALS: + loop.add_signal_handler(sig, request_stop, f"signal_{sig.name}") + registered_signals.append(sig) + task = asyncio.create_task(node.run_async()) + try: + while not task.done(): + if handle.is_running: + receipt["node"]["started"] = True + observed_start = observed_start or time.monotonic() + observe() + cases = receipt["cases"] + now = time.monotonic() + if receipt["failures"] or any(c["outcome"] == "failed" for c in cases.values()): + request_stop("case_failure") + elif cases["C1"]["outcome"] == cases["C3"]["outcome"] == "passed": + request_stop("C1_accepted_C3_filled") + elif observed_start is not None and now - observed_start >= plan["timeouts"]["per_step_seconds"]: + request_stop("case_timeout") + if now >= payload["node_stop_at"]: + stop("deadline") + elif pending_stop: + if stop_ready(tester_orders(cache, scope), now, payload["node_stop_at"]): + stop(pending_stop) + write_receipt(receipt_path, receipt, (account_id,)) + await asyncio.sleep(0.1) + except Exception as exc: + record_error("node_poll", exc) + # An accepted MARKET IOC can still fill; preserve close-on-stop's + # one opportunity to see its position even if observation failed. + while not task.done() and time.monotonic() < payload["node_stop_at"]: + try: + if stop_ready(tester_orders(cache, scope), time.monotonic(), payload["node_stop_at"]): + break + except Exception: + pass # Unknown state waits only to the bounded deadline. + await asyncio.sleep(0.1) + stop("poll_exception") + await task # Native cancel/close runs even after an observer exception. + except Exception as exc: + record_error("node_run", exc) + stop("node_exception") + if task is not None and not task.done(): + try: + await task + except Exception as task_exc: + record_error("node_shutdown", task_exc) + finally: + try: + observe() + except Exception as exc: + record_error("final_snapshot", exc) + try: + write_receipt(receipt_path, receipt, (account_id,)) + except Exception as exc: + record_error("final_receipt", exc) + for sig in registered_signals: + try: + loop.remove_signal_handler(sig) + except Exception as exc: + record_error("signal_removal", exc) + try: + node.dispose() + except Exception as exc: + record_error("node_dispose", exc) + if journal is not None: + try: + receipt["journal"] = journal.summary() + except Exception as exc: + record_error("journal_summary", exc) + try: + journal.close() + except Exception as exc: + record_error("journal_close", exc) + # Persist cleanup errors independently, without skipping disposal. + write_receipt(receipt_path, receipt, (account_id,)) + return receipt + + +class CheckTimeout(TimeoutError): + pass + + +def official_check(plan, port, *, with_session, deadline): + """Bound frozen ibapi connect/handshake as well as its request waits.""" + seconds = min(plan["timeouts"]["check_seconds"], max(0, deadline - time.monotonic())) + if seconds <= 0: + return {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}}, None + + def expired(_sig, _frame): + raise CheckTimeout("official_check_deadline") + + previous = signal.signal(signal.SIGALRM, expired) + signal.setitimer(signal.ITIMER_REAL, seconds) + try: + return frozen().run_check(plan, port, with_session=with_session, deadline_s=seconds) + except Exception as exc: + return {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}, + "error": frozen().redact(f"{type(exc).__name__}: {exc}")}, None + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, previous) + + +def official_admission(plan, port, *, deadline, client_factory=None, started_at=None): + """Flat pre-check and quote admission share one official client-92 session.""" + seconds = min(plan["timeouts"]["check_seconds"], max(0, deadline - time.monotonic())) + client = (client_factory or build_admission_client)() + session = CheckSession(client) + check_deadline = time.monotonic() + seconds + pre = {"client": "ibapi", "client_id": 92, "status": "incomplete", "observed": {}} + account_id = None + phase = "pre_check" + pre_timed_out = False + + def failure(exc): + status = "not_connected" + if "tick requested" in client.stage_elapsed_seconds: + status = "refused_quote_clock" if "quote" in client.stage_elapsed_seconds else "refused_quote_unavailable" + if client.delayed: + status = "refused_delayed_quote" + elif client.errors and status != "not_connected": + status = "refused_quote_error" + return {"status": status, "error": frozen().redact(exc), **client.diagnostics()} + + if seconds <= 0: + pre["error"] = "CheckTimeout: pre_check_deadline" + return pre, None, {"status": "not_run", **client.diagnostics()} + + def expired(_sig, _frame): + nonlocal pre_timed_out + pre_timed_out = phase == "pre_check" + raise CheckTimeout(f"{phase}_deadline") + + previous = signal.signal(signal.SIGALRM, expired) + signal.setitimer(signal.ITIMER_REAL, seconds) + try: + pre, account_id = frozen().run_check(plan, port, with_session=True, + client_factory=lambda: session, deadline_s=seconds) + pre["observed"] = copy.deepcopy(pre.get("observed", {})) + # ibapi.connect / frozen run_check can catch TimeoutError as OSError. + # Preserve the alarm's phase even when that path returns normally. + if pre_timed_out: + pre.update(status="incomplete", error="CheckTimeout: pre_check_deadline") + return pre, None, {"status": "not_run", **client.diagnostics()} + phase = "quote_check" + if pre["status"] != "passed" or not account_id: + return pre, account_id, {"status": "not_run", **client.diagnostics()} + if started_at is not None: + sessions = frozen().parse_liquid_hours(pre.get("liquid_hours", ""), + pre.get("time_zone_id", "America/New_York"), + started_at.astimezone(frozen().ZoneInfo("America/New_York")).date()) + closing = (closing_period_admission(plan, started_at, liquid_sessions=sessions) if sessions is not None + else {"status": "refused_liquid_hours", "reason": "no_usable_liquid_session"}) + if closing["status"] != "passed": + # Contract liquidHours is now known; refuse before requesting a + # quote, while the same read-only session still owns teardown. + return pre, account_id, {"status": "not_run", "closing_period_admission": closing, + **client.diagnostics()} + admitted = quote_check(plan, account_id, deadline=check_deadline, client=client) + return pre, account_id, admitted + except Exception as exc: + if phase == "pre_check": + pre.update(status="incomplete", observed=copy.deepcopy(client.r), requests_completed=client.completed(), + error=frozen().redact(f"pre_check: {type(exc).__name__}: {exc}")) + return pre, None, {"status": "not_run", **client.diagnostics()} + return pre, account_id, failure(exc) + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, previous) + session.close(check_deadline) + + +def final_status(receipt, *, interrupted=False, child_returncode=0): + """Flat-proof failure dominates; observed/case/loss breaches remain failed.""" + if receipt["flat_proof"]["status"] != "passed": + return "cleanup_required", "independent_flat_proof", 4, ("C4",) + child_refusal = receipt["node"].get("pre_node_refusal") + if not receipt["node"]["started"] and child_refusal: + return child_refusal, child_refusal, 2, CASE_IDS + rt = receipt["roundtrip"] + if (any(f["reason"].startswith("observed_") for f in receipt["failures"]) + or any(c["outcome"] == "failed" for c in receipt["cases"].values()) + or rt and rt["loss_bound_breach"]): + return "failed", "case_or_loss_bound", 2, CASE_IDS + if interrupted or receipt["failures"] or child_returncode != 0: + return "incomplete", "signal_or_node_failure", 3, CASE_IDS + if all(c["outcome"] == "passed" for c in receipt["cases"].values()) and rt and rt["closed"] and not rt["commission_unresolved"]: + return "passed", None, 2, CASE_IDS + return "incomplete", "cases_or_commissions_unresolved", 3, CASE_IDS + + +def parent_failure_status(receipt): + """Parent I/O failure must preserve launch evidence and any known failure.""" + node = receipt["node"] + if not (node.get("child_launched", False) or node.get("started", False)): + return "not_started" + if not receipt.get("flat_proof") or receipt["flat_proof"].get("status") != "passed": + return "cleanup_required" + return "failed" if receipt["status"] == "failed" else "incomplete" + + +def run_trial(args, *, now=None, versions=None): + """Static refusal precedes connection; liquid-hours admission precedes node.""" + started_monotonic = time.monotonic() + try: + plan = load_plan(args.plan) + except (ValueError, OSError): + plan = {} + versions = runtime_versions() if versions is None else versions + receipt = new_receipt(plan, args.plan, versions) + errors = validate_plan(plan, port=args.port, node_client_id=args.node_client_id, check_client_id=args.check_client_id) + status = "refused_plan" if errors else runtime_refusal(versions) + now = now or datetime.now(timezone.utc) + if status is None: + inside, window = frozen().rth_check(now, plan) + receipt["window"] = window + if not inside: + status = "refused_outside_window" + if status is None: + closing_admission = closing_period_admission(plan, now) + receipt["closing_period_admission"] = closing_admission + if closing_admission["status"] != "passed": + status = closing_admission["status"] + if status: + finish(receipt, status, reason="; ".join(errors) if errors else status) + if args.receipt: + write_receipt(args.receipt, receipt, replace=False) + return receipt + if args.plan_only: + finish(receipt, "passed") + receipt["evidence_class"] = "structural_validation" + if args.receipt: + write_receipt(args.receipt, receipt, replace=False) + return receipt + if args.receipt is None: + finish(receipt, "refused_receipt_required", reason="receipt required before node") + return receipt + # Command-center r2 decision: runner quantity + quote admission hold the + # bound; the engine cap remains configured with its route limitation stated. + deadline = started_monotonic + plan["timeouts"]["overall_deadline_seconds"] + expected_account = os.environ.get("TWS_ACCOUNT") + private_accounts = [expected_account] + lease = journal = None + try: + # Account-derived lock keys stay on the host; never put them in evidence. + lease = safety().AccountLease(expected_account) + receipt["account_lease"]["acquired"] = True + run_id = args.run_id or uuid.uuid4().hex + output = args.private_output_dir or Path(args.receipt).resolve().parent / ("private-" + run_id) + journal = safety().Journal(output, run_id, ROOT, create=True, + hashes={"plan_sha256": receipt["plan_sha256"], "harness_sha256": receipt["harness_sha256"]}) + receipt["journal"] = journal.summary() + receipt = run_admitted_trial(args, plan, receipt, deadline, expected_account, lease, journal, now, private_accounts) + except (safety().SafetyError, OSError) as exc: + launched = receipt["node"].get("child_launched", False) + reason = str(exc) if isinstance(exc, safety().SafetyError) else ( + "post_launch_io_error" if launched else "lease_or_journal_storage_error") + finish(receipt, parent_failure_status(receipt), reason=reason, step=4 if launched else 2) + finally: + try: + if journal is not None: + try: + receipt["journal"] = journal.summary() + finally: + journal.close() + except Exception as exc: + finish(receipt, parent_failure_status(receipt), + reason="private_journal: " + frozen().redact(exc)) + finally: + try: + # Refusals are console-only if any prior receipt already exists. + # A child launch owns its provisional receipt and can refresh it. + write_receipt(args.receipt, receipt, private_accounts, + replace=receipt["node"].get("child_launched", False)) + except OSError: + launched = receipt["node"].get("child_launched", False) + finish(receipt, parent_failure_status(receipt), + reason="final_receipt_write_error", step=4 if launched else 2) + finally: + if lease is not None: + lease.close() + return receipt + + +def run_admitted_trial(args, plan, receipt, deadline, expected_account, lease, journal, now, private_accounts): + port = args.port if args.port is not None else plan["default_port"] + pre, account_id, admitted = official_admission(plan, port, deadline=deadline, started_at=now) + private_accounts.append(account_id) + liquid = pre.pop("liquid_hours", "") + pre.pop("trading_hours", None) + zone = pre.pop("time_zone_id", "America/New_York") + receipt["pre_check"] = pre + receipt["quote_admission"] = admitted + if admitted.get("closing_period_admission"): + receipt["closing_period_admission"] = admitted["closing_period_admission"] + if pre["status"] != "passed" or not account_id: + finish(receipt, pre["status"] if pre["status"] != "passed" else "refused_account_scope", reason="pre_check") + return receipt + if account_id != expected_account: + finish(receipt, "refused_changed_account", reason="pre_check_expected_account") + return receipt + sessions = frozen().parse_liquid_hours(liquid, zone, now.astimezone(frozen().ZoneInfo("America/New_York")).date()) + if sessions is None or not frozen().rth_check(datetime.now(timezone.utc), plan, sessions, horizon_s=deadline - time.monotonic())[0]: + finish(receipt, "refused_liquid_hours", reason="whole_run_broker_window") + return receipt + closing = closing_period_admission(plan, now, liquid_sessions=sessions) + receipt["closing_period_admission"] = closing + if closing["status"] != "passed": + finish(receipt, closing["status"], reason=closing["reason"]) + return receipt + if admitted["status"] != "passed": + finish(receipt, admitted["status"], reason="quote_admission") + return receipt + # Atomic provisional cleanup_required receipt BEFORE creating the child/node. + write_receipt(args.receipt, receipt, (account_id,)) + child = None + run_nonce = receipt["run_nonce"] + interrupted = [] + old_handlers = {} + + def signaled(sig, _frame): + interrupted.append(signal.Signals(sig).name) + if child is not None and child.poll() is None: + child.send_signal(sig) + + try: + for sig in SIGNALS: + old_handlers[sig] = signal.signal(sig, signaled) + stop_at = deadline - plan["timeouts"]["post_check_reserve_seconds"] - plan["timeouts"]["cleanup_seconds"] - plan["timeouts"]["node_stop_allowance_seconds"] + command = [sys.executable, str(Path(__file__).resolve()), "--_node-phase", "--plan", str(Path(args.plan).resolve()), + "--port", str(port), "--receipt", str(Path(args.receipt).resolve())] + child_env = {key: value for key, value in os.environ.items() + if key not in ("RUST_LOG", "NAUTILUS_LOG", "TWS_ACCOUNT")} + # Conservative launch marker survives Popen/pipe/storage failures even + # when the child never manages to persist node.started. + receipt["node"]["child_launched"] = True + child = subprocess.Popen(command, stdin=subprocess.PIPE, text=True, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=child_env, pass_fds=(lease.fd,)) + # Anonymous pipe only: no account in argv, environment, file or receipt. + child.stdin.write(json.dumps({"account_id": account_id, "receipt": receipt, "node_stop_at": stop_at, + "run_nonce": run_nonce, + "quote_admission": admitted, "tob_offset_ticks": admitted["tob_offset_ticks"], + "journal_path": str(journal.path), "run_id": journal.run_id})) + child.stdin.close() + try: + child.wait(timeout=max(0, stop_at - time.monotonic())) + except subprocess.TimeoutExpired: + child.send_signal(signal.SIGTERM) + child.wait(timeout=max(0, deadline - plan["timeouts"]["post_check_reserve_seconds"] - time.monotonic())) + except subprocess.TimeoutExpired: + child.kill() + child.wait() + receipt["failures"].append({"acceptance_step": 4, "cases_blocked": ["C2", "C4"], "reason": "node_hard_deadline"}) + except Exception as exc: + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), "reason": frozen().redact(exc)}) + if child is not None and child.poll() is None: + child.send_signal(signal.SIGTERM) + try: + child.wait(timeout=plan["timeouts"]["node_stop_allowance_seconds"]) + except subprocess.TimeoutExpired: + child.kill() + child.wait() + finally: + for sig, handler in old_handlers.items(): + signal.signal(sig, handler) + saved_failures = receipt["failures"] + launched = receipt["node"].get("child_launched", False) + try: + latest = json.loads(Path(args.receipt).read_text()) + if latest.get("run_nonce") != run_nonce: + # Other accounts can hold separate leases but share this path. + # Preserve our provisional evidence and still perform flat proof. + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), + "reason": "child_receipt_nonce_mismatch"}) + else: + latest["failures"].extend(saved_failures) + latest["node"]["child_launched"] = launched + # Keep the caller's object current even if flat proof or a later + # receipt write raises. Never revert to pre-node evidence. + receipt.clear() + receipt.update(latest) + except Exception as exc: + # JSON lists and malformed nested fields must not skip independent + # flat proof. No child evidence has replaced the provisional object. + receipt["failures"].append({"acceptance_step": 3, "cases_blocked": list(CASE_IDS), + "reason": "child_receipt_unreadable", + "error": frozen().redact(f"{type(exc).__name__}: {exc}")}) + proof, proof_account = official_check(plan, port, with_session=False, deadline=deadline) + private_accounts.append(proof_account) + if proof["status"] == "passed" and proof_account != account_id: + proof["status"] = "refused_changed_account" + receipt["flat_proof"] = proof + try: + journal.append("flat_proof", status=proof["status"], positions=proof.get("observed", {}).get("positions"), + open_orders=proof.get("observed", {}).get("open_orders")) + except Exception as exc: + receipt["failures"].append({"acceptance_step": 4, "cases_blocked": ["C4"], "reason": "journal_flat_proof: " + frozen().redact(exc)}) + update_observations(receipt, receipt["events"], plan) + status, reason, step, cases = final_status(receipt, interrupted=bool(interrupted), + child_returncode=child.returncode if child is not None else None) + finish(receipt, status, reason=reason, step=step, cases=cases) + try: + write_receipt(args.receipt, receipt, (account_id, proof_account)) + except OSError: + finish(receipt, parent_failure_status(receipt), + reason="final_receipt_write_error", step=4) + return receipt + + +class PaperArgumentParser(argparse.ArgumentParser): + def error(self, message): + self.exit(3, "refused_arguments: " + frozen().redact(message) + "\n") + + +def parser(): + result = PaperArgumentParser(description=__doc__) + result.add_argument("--plan", type=Path, default=PLAN_PATH) + result.add_argument("--receipt", type=Path, help="sanitized, atomically replaced receipt; required for an order run") + result.add_argument("--plan-only", action="store_true", help="validate plan, pins and window WITHOUT connecting; quote admission is not exercised") + result.add_argument("--private-output-dir", type=Path, help="0700 directory outside the repository for the private journal; defaults beside receipt") + result.add_argument("--run-id", help="stable private journal identity; reuse refuses before connection; default is a new UUID") + result.add_argument("--port", type=int, help="4002 (default) or 7497 only") + result.add_argument("--node-client-id", type=int, help="91 only") + result.add_argument("--check-client-id", type=int, help="92 only") + result.add_argument("--_node-phase", action="store_true", help=argparse.SUPPRESS) + return result + + +def persist_child_refusal(path, payload, reason): + """Keep a sanitized pre-node reason even though child output is discarded.""" + receipt = payload.get("receipt") if isinstance(payload, dict) else None + if not isinstance(receipt, dict): + try: + receipt = json.loads(Path(path).read_text()) + except (OSError, TypeError, ValueError): + receipt = new_receipt(load_plan(), PLAN_PATH, runtime_versions()) + receipt["node"]["pre_node_refusal"] = reason + finish(receipt, reason, reason=reason) + if path is not None: + account = payload.get("account_id") if isinstance(payload, dict) else None + write_receipt(path, receipt, (account,)) + return receipt + + +def run_child(args): + try: + payload = json.load(sys.stdin) + if not isinstance(payload, dict): + raise ValueError("invalid child payload") + except (ValueError, OSError): + persist_child_refusal(args.receipt, {}, "refused_child_payload") + return 3 + try: + plan = load_plan(args.plan) + reason = "refused_child_plan" if validate_plan(plan, port=args.port) else None + except (ValueError, OSError): + reason = "refused_child_plan" + if reason is None and runtime_refusal(runtime_versions()): + reason = "refused_child_runtime" + if reason is None: + nonce = payload.get("run_nonce") + child_receipt = payload.get("receipt") + if (not isinstance(nonce, str) or len(nonce) != 32 or any(c not in "0123456789abcdef" for c in nonce) + or not isinstance(child_receipt, dict) or child_receipt.get("run_nonce") != nonce): + reason = "refused_child_receipt_nonce" + if reason is None: + try: + inside = frozen().rth_check(datetime.now(timezone.utc), plan, + horizon_s=max(0, payload["node_stop_at"] - time.monotonic()))[0] + if not inside: + reason = "refused_child_window" + except (KeyError, TypeError, ValueError): + reason = "refused_child_payload" + if reason is None and not validate_child_admission(plan, payload): + reason = "refused_child_quote_admission" + if reason is None: + try: + journal_path = Path(payload["journal_path"]).resolve() + if journal_path.name != payload["run_id"] + ".jsonl" or ROOT in journal_path.parents: + reason = "refused_child_journal" + elif not journal_path.is_file(): + reason = "refused_child_journal" + except (KeyError, TypeError, ValueError): + reason = "refused_child_journal" + if reason: + persist_child_refusal(args.receipt, payload, reason) + return 3 + receipt = asyncio.run(node_phase(plan, args.port, args.receipt, payload)) + if receipt["node"].get("pre_node_refusal"): + return 3 + return 1 if receipt["failures"] else 0 + + +def main(argv=None): + args = parser().parse_args(argv) + try: + if args._node_phase: + # The pipe supplies the admitted quote and derived offset; validate + # before node creation. This entry never loads an account from env. + return run_child(args) + receipt = run_trial(args) + except Exception as exc: + print(frozen().scrub_serialized(json.dumps({"status": "incomplete", "exit_code": 1, "error": frozen().redact(exc)}))) + return 1 + print(frozen().scrub_serialized(json.dumps({"status": receipt["status"], "exit_code": receipt["exit_code"], "window": receipt.get("window"), + "failures": sanitize(receipt["failures"])}))) + return receipt["exit_code"] + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/safety.py b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/safety.py new file mode 100644 index 000000000..a54ac73f3 --- /dev/null +++ b/blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/safety.py @@ -0,0 +1,127 @@ +"""Host-local lease and private append-only lifecycle journal; no broker APIs. + +Lease reference: native-agent-stack dca821cca85dce3647fa7b488d5a23fbe5b85d4a, +blueprints/us-equities/adaptive-paper/safety.py account_lock_fingerprint. +Durability uses Python's documented os.open O_APPEND/O_EXCL, flush and os.fsync. +""" +from __future__ import annotations + +import fcntl +import hashlib +import json +import os +from pathlib import Path +import re +import time + + +class SafetyError(RuntimeError): + """Constant reason codes, with no private values or paths.""" + + +def account_lock_name(account_id): + if not isinstance(account_id, str) or not re.fullmatch(r"DU\d{5,}", account_id) or len(account_id) > 128: + raise SafetyError("paper_account_required_for_lease") + return hashlib.sha256(account_id.encode()).hexdigest() + ".lock" + + +class AccountLease: + def __init__(self, account_id): + self.lock_name = account_lock_name(account_id) + state = Path(os.environ.get("XDG_STATE_HOME") or Path.home() / ".local/state") + if not state.is_absolute(): + raise SafetyError("absolute_state_home_required") + root = state / "native-agent-stack/ibkr-paper/locks" + root.mkdir(parents=True, exist_ok=True, mode=0o700) + root.chmod(0o700) + self.fd = os.open(root / self.lock_name, os.O_CREAT | os.O_RDWR | os.O_NOFOLLOW, 0o600) + try: + os.fchmod(self.fd, 0o600) + fcntl.flock(self.fd, fcntl.LOCK_EX | fcntl.LOCK_NB) + except BlockingIOError: + os.close(self.fd) + self.fd = None + raise SafetyError("account_writer_already_running") from None + except BaseException: + os.close(self.fd) + self.fd = None + raise + + def close(self): + if self.fd is not None: + os.close(self.fd) + self.fd = None + + +def private_output(directory, repository): + path = Path(directory).resolve() + if path == repository or repository in path.parents: + raise SafetyError("private_output_must_be_outside_repository") + path.mkdir(parents=True, exist_ok=True, mode=0o700) + path.chmod(0o700) + return path + + +class Journal: + """One process appends at a time; parent resumes after the child has exited.""" + + def __init__(self, directory, run_id, repository, *, create, hashes=None): + if not isinstance(run_id, str) or not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_-]{0,79}", run_id): + raise SafetyError("invalid_run_id") + root = private_output(directory, Path(repository).resolve()) + self.path = root / (run_id + ".jsonl") + flags = os.O_APPEND | os.O_RDWR | os.O_NOFOLLOW + flags |= os.O_CREAT | os.O_EXCL if create else 0 + try: + fd = os.open(self.path, flags, 0o600) + except FileExistsError: + raise SafetyError("journal_run_already_exists") from None + self.stream = os.fdopen(fd, "a", encoding="utf-8") + self.run_id = run_id + self.seen = set() + try: + os.fchmod(fd, 0o600) + if create: + self.append("run_start", run_id=run_id, **(hashes or {})) + # Persist the file's directory entry as well as its contents. + directory_fd = os.open(root, os.O_RDONLY | os.O_DIRECTORY) + try: + os.fsync(directory_fd) + finally: + os.close(directory_fd) + else: + records = self.records() + if not records or records[0].get("kind") != "run_start" or records[0].get("run_id") != run_id: + raise SafetyError("journal_run_mismatch") + self.seen = {record["event_id"] for record in records if "event_id" in record} + except BaseException: + self.close() + raise + + def append(self, kind, **fields): + record = {"kind": kind, "recorded_ns": time.time_ns(), **fields} + self.stream.write(json.dumps(record, sort_keys=True, allow_nan=False) + "\n") + self.stream.flush() + os.fsync(self.stream.fileno()) + + def event(self, record): + if record["event_id"] not in self.seen: + self.append(record["kind"], **{key: value for key, value in record.items() if key != "kind"}) + self.seen.add(record["event_id"]) + + def records(self): + try: + records = [json.loads(line) for line in self.path.read_text().splitlines()] + except (ValueError, UnicodeError): + raise SafetyError("journal_corrupt") from None + if any(not isinstance(record, dict) or not isinstance(record.get("kind"), str) for record in records): + raise SafetyError("journal_corrupt") + return records + + def summary(self): + records = self.records() + return {"sha256": hashlib.sha256(self.path.read_bytes()).hexdigest(), + "record_count": len(records), "record_kinds": list(dict.fromkeys(record["kind"] for record in records))} + + def close(self): + self.stream.close() diff --git a/catalogs/foundation/new-wsl-architecture-20261001.json b/catalogs/foundation/new-wsl-architecture-20261001.json index 3b9750faf..5577f8ef4 100644 --- a/catalogs/foundation/new-wsl-architecture-20261001.json +++ b/catalogs/foundation/new-wsl-architecture-20261001.json @@ -1846,7 +1846,7 @@ "component_id": "alpaca-py", "repository": "https://github.com/alpacahq/alpaca-py", "pin": "0.44.0", - "pin_source": "catalogs/us-equities/runtime-target.json:158", + "pin_source": "catalogs/us-equities/runtime-target.json:159", "install": {"command": "uv pip install --python \"$GATE_TOOL_DIR/.venv/bin/python3\" with the recipe's pinned requirements (the research-runtime profile has no Linux pins)", "kind": "repo_recipe", "source_path": "blueprints/us-equities/order-contract/README.md"}, "acceptance": {"command": "the September 20 run of installed alpaca-py 0.44.0: five authenticated GET requests, all returned HTTP 200", "evidence_class": "upstream_example_or_native_operation", "source_path": "blueprints/us-equities/authenticated-data/README.md"}, "upstream_currency": {"latest_release": "v0.44.0 (2026-08-11)", "checked_at": "2026-10-01T07:42Z", "pin_is_latest": "yes", "url": "https://github.com/alpacahq/alpaca-py/releases"} @@ -1923,7 +1923,7 @@ "component_id": "alpaca-py", "repository": "https://github.com/alpacahq/alpaca-py", "pin": "0.44.0", - "pin_source": "catalogs/us-equities/runtime-target.json:158", + "pin_source": "catalogs/us-equities/runtime-target.json:159", "install": {"command": "uv pip install --python \"$GATE_TOOL_DIR/.venv/bin/python3\" with the recipe's pinned requirements (the research-runtime profile has no Linux pins)", "kind": "repo_recipe", "source_path": "blueprints/us-equities/order-contract/README.md"}, "acceptance": {"command": "the September 20 run of installed alpaca-py 0.44.0: five authenticated GET requests, all returned HTTP 200", "evidence_class": "upstream_example_or_native_operation", "source_path": "blueprints/us-equities/authenticated-data/README.md"}, "upstream_currency": {"latest_release": "v0.44.0 (2026-08-11)", "checked_at": "2026-10-01T07:42Z", "pin_is_latest": "yes", "url": "https://github.com/alpacahq/alpaca-py/releases"} @@ -2018,7 +2018,7 @@ "component_id": "alpaca-py", "repository": "https://github.com/alpacahq/alpaca-py", "pin": "0.44.0", - "pin_source": "catalogs/us-equities/runtime-target.json:158", + "pin_source": "catalogs/us-equities/runtime-target.json:159", "install": {"command": "uv pip install --python \"$GATE_TOOL_DIR/.venv/bin/python3\" with the recipe's pinned requirements (the research-runtime profile has no Linux pins)", "kind": "repo_recipe", "source_path": "blueprints/us-equities/order-contract/README.md"}, "acceptance": {"command": "the September 20 run of installed alpaca-py 0.44.0: five authenticated GET requests, all returned HTTP 200", "evidence_class": "upstream_example_or_native_operation", "source_path": "blueprints/us-equities/authenticated-data/README.md"}, "upstream_currency": {"latest_release": "v0.44.0 (2026-08-11)", "checked_at": "2026-10-01T07:42Z", "pin_is_latest": "yes", "url": "https://github.com/alpacahq/alpaca-py/releases"} @@ -2294,7 +2294,7 @@ "role": "separate Alpaca path", "repository": "https://github.com/alpacahq/alpaca-py", "pin": "0.44.0", - "pin_source": "catalogs/us-equities/runtime-target.json:158", + "pin_source": "catalogs/us-equities/runtime-target.json:159", "install": {"command": "uv pip install --python \"$GATE_TOOL_DIR/.venv/bin/python3\" with the recipe's pinned requirements (the trading-nautilus profile has no Linux pins)", "kind": "repo_recipe", "source_path": "blueprints/us-equities/order-contract/README.md"}, "acceptance": {"command": "the 2026-09-29 pre-market Alpaca paper series through the adaptive-paper engine (alpaca-py 0.44.0 LimitOrderRequest orders, engine b528bb55): 3 of 3 attempts passed, 9 entries and 9 exits filled, every fill matched to the broker, flat at the series end", "evidence_class": "local_integration_check", "source_path": "blueprints/us-equities/adaptive-paper/trials/pre-20260929/receipt.json"}, "upstream_currency": {"latest_release": "v0.44.0 (2026-08-11)", "checked_at": "2026-10-01T07:42Z", "pin_is_latest": "yes", "url": "https://github.com/alpacahq/alpaca-py/releases"} @@ -2359,7 +2359,7 @@ "role": "Alpaca broker boundary", "repository": "https://github.com/alpacahq/alpaca-py", "pin": "0.44.0", - "pin_source": "catalogs/us-equities/runtime-target.json:158", + "pin_source": "catalogs/us-equities/runtime-target.json:159", "install": {"command": "uv pip install --python \"$GATE_TOOL_DIR/.venv/bin/python3\" with the recipe's pinned requirements (the trading-nautilus profile has no Linux pins)", "kind": "repo_recipe", "source_path": "blueprints/us-equities/order-contract/README.md"}, "acceptance": {"command": "bounded Alpaca paper roundtrips through the adaptive-paper engine in the 2026-09-29 pre-market series (engine b528bb55, alpaca-py 0.44.0): 3 of 3 attempts passed, every fill matched to the broker, flat at the series end", "evidence_class": "local_integration_check", "source_path": "blueprints/us-equities/adaptive-paper/trials/pre-20260929/receipt.json"}, "upstream_currency": {"latest_release": "v0.44.0 (2026-08-11)", "checked_at": "2026-10-01T07:42Z", "pin_is_latest": "yes", "url": "https://github.com/alpacahq/alpaca-py/releases"} @@ -2454,7 +2454,7 @@ "role": "Alpaca boundary", "repository": "https://github.com/alpacahq/alpaca-py", "pin": "0.44.0", - "pin_source": "catalogs/us-equities/runtime-target.json:158", + "pin_source": "catalogs/us-equities/runtime-target.json:159", "install": {"command": "uv pip install --python \"$GATE_TOOL_DIR/.venv/bin/python3\" with the recipe's pinned requirements (the trading-nautilus profile has no Linux pins)", "kind": "repo_recipe", "source_path": "blueprints/us-equities/order-contract/README.md"}, "acceptance": {"command": "the 2026-09-29 pre-market Alpaca paper series through the adaptive-paper engine (alpaca-py 0.44.0 LimitOrderRequest orders, engine b528bb55): 3 of 3 attempts passed, 9 entries and 9 exits filled, every fill matched to the broker, flat at the series end", "evidence_class": "local_integration_check", "source_path": "blueprints/us-equities/adaptive-paper/trials/pre-20260929/receipt.json"}, "upstream_currency": {"latest_release": "v0.44.0 (2026-08-11)", "checked_at": "2026-10-01T07:42Z", "pin_is_latest": "yes", "url": "https://github.com/alpacahq/alpaca-py/releases"} diff --git a/catalogs/us-equities/runtime-target.json b/catalogs/us-equities/runtime-target.json index 90c07487d..c454d4795 100644 --- a/catalogs/us-equities/runtime-target.json +++ b/catalogs/us-equities/runtime-target.json @@ -97,11 +97,12 @@ { "ref": "https://github.com/nautechsystems/nautilus_trader/issues/4946", "state": "closed", - "state_checked_at": "2026-10-01", + "state_checked_at": "2026-10-05", "path": "pre-trade risk: account-scoped checks", "reported_against": "2.0.0rc4; the fail-open branch is at crates/risk/src/engine/mod.rs:1218-1234 at tag v2.0.0rc5", "effect": "For a broker-routed SMART instrument the account registered under venue IB is not found, so every account-scoped pre-trade check (max_notional_per_order, free balance, cumulative notional) is skipped with only a debug log.", - "upstream_fix": "Closed 2026-09-25T11:03:11Z by develop commit ed6fc8bf47fd37dda97d63b9b2df160719ee2bac ('Apply pre-trade risk to the destination execution account'), which is in no release (the newest release is the prerelease v2.0.0rc5; GitHub's releases/latest is v1.231.0; read 2026-10-01); PR #5041 also lists it as covered." + "upstream_fix": "Closed 2026-09-25T11:03:11Z by develop commit ed6fc8bf47fd37dda97d63b9b2df160719ee2bac ('Apply pre-trade risk to the destination execution account'). It was in no release at the October 1 check; GitHub now lists v2.0.0rc6, published 2026-10-05T03:03:18Z, and the fix commit is its ancestor (compare reports ahead, behind_by 0). The selected rc5 pin remains affected.", + "rerun_trigger": "On an authorized engine-pin change to a release containing ed6fc8bf, re-run a discriminating over-limit native test with zero broker submissions, then the bounded IBKR paper cases and independent flat proof. The rc5 runner states that its configured engine cap is not enforced on SMART/IB, and holds the bound with four rules: qty=1; timestamped quote admission with a 5% LULD upward stress plus USD 10 headroom (ask x 1.05 + 10 <= 1000); a spread-plus-commission round-trip bound (spread + 2.04 <= 5 USD); and no run whose order and close window reaches the closing period (the liquid session end minus 25 minutes, so the latest start is 15:29 ET on a normal day). A separate host-local account lease keeps one writer per account." } ], "rc5_reclassified_reports": [ diff --git a/docs/new-wsl-handbook.json b/docs/new-wsl-handbook.json index 8303ec164..ec08626e5 100644 --- a/docs/new-wsl-handbook.json +++ b/docs/new-wsl-handbook.json @@ -7803,7 +7803,7 @@ }, { "path": "catalogs/foundation/new-wsl-architecture-20261001.json", - "sha256": "8420f58ef2f613bab8da7a4486e907a9118b178d9d8e5eb8c70e4d84950c6c73" + "sha256": "ebad29544bcd86c22b0fc4d76c2882156c3735b8d003b8b1259e4bf96ba22d8e" }, { "path": "catalogs/landscape/research-state.json", diff --git a/docs/new-wsl-handbook.md b/docs/new-wsl-handbook.md index 61e0aabb7..7bcbe49c2 100644 --- a/docs/new-wsl-handbook.md +++ b/docs/new-wsl-handbook.md @@ -2130,7 +2130,7 @@ Reference edition: [catalogs/foundation/new-wsl-architecture-20261001.json](../c | [adoption/manifest.json](../adoption/manifest.json) | `159ad1f56a9a654951eb16a4a66645378d225d8b924eede282815f49342add55` (without source.release_tag, source.release_commit, updated_at) | | [adoption/new-wsl-profile.json](../adoption/new-wsl-profile.json) | `7504e55c2e66730d5e0b0ead2be5abef39f3989afc69e1c3640e6709bdb3db04` | | [adoption/platforms/linux-wsl2-new-distro.md](../adoption/platforms/linux-wsl2-new-distro.md) | `4e505c4e7ca64d769bb0f9be4e506526dfaed79231e4a30a26bb811bcb967abe` | -| [catalogs/foundation/new-wsl-architecture-20261001.json](../catalogs/foundation/new-wsl-architecture-20261001.json) | `8420f58ef2f613bab8da7a4486e907a9118b178d9d8e5eb8c70e4d84950c6c73` | +| [catalogs/foundation/new-wsl-architecture-20261001.json](../catalogs/foundation/new-wsl-architecture-20261001.json) | `ebad29544bcd86c22b0fc4d76c2882156c3735b8d003b8b1259e4bf96ba22d8e` | | [catalogs/landscape/research-state.json](../catalogs/landscape/research-state.json) | `fe142c1b8b8ec8b92d69ec5971d68028225899c101ffc43cea618883b119332a` | | [catalogs/landscape/us-equities.json](../catalogs/landscape/us-equities.json) | `cf441d393b1a1ee49c57d62592e2c7477de2003ca40aa9e1933a6c9b28d4a618` | | [evidence/artifacts/new-wsl-clean-install-selection-20261001/ownership.json](../evidence/artifacts/new-wsl-clean-install-selection-20261001/ownership.json) | `ad86d1d4c2cddeeebc2bc327fddc897b161cec092706db24c779603321246bc7` | diff --git a/evidence/artifacts/new-wsl-handbook-20261001/receipt.json b/evidence/artifacts/new-wsl-handbook-20261001/receipt.json index 6fae320ec..dfd397504 100644 --- a/evidence/artifacts/new-wsl-handbook-20261001/receipt.json +++ b/evidence/artifacts/new-wsl-handbook-20261001/receipt.json @@ -6,8 +6,8 @@ "generator_sha256": "6e6ee9cfb4d697b3bea2d7684e042e638be20634792f43e517b205463b1967f9", "profile_sha256": "7504e55c2e66730d5e0b0ead2be5abef39f3989afc69e1c3640e6709bdb3db04", "outputs": { - "docs/new-wsl-handbook.md": "b1dd8bdb72f32d3ab449aa90014b9621f779a5a7b3db590e4253120ba6c0bdfe", - "docs/new-wsl-handbook.json": "32da2ef7bbb47cbd8e75a1674b19c9dd1ee8e3cc855a05efee322765186e079e" + "docs/new-wsl-handbook.md": "aa6515ab958fd48038e853e91ebf29e196da7d4d6736aa90240c12a8cf6dabd5", + "docs/new-wsl-handbook.json": "e30b11544f28a0a8682f930afef47a2af9c441591d25a8dbadbfb255921d8212" }, "inventory": { "layers": 37, @@ -249,6 +249,16 @@ "docs/new-wsl-handbook.json": "c49d7d2457692e6d3895cd0896ed601d81d55a90b2f3dd9ff27cec6116da7a16" }, "check": "python3 scripts/build_new_wsl_handbook.py --check" + }, + { + "date": "2026-10-05", + "command": "python3 scripts/build_new_wsl_handbook.py --write", + "cause": "PR #754 refreshed onto main ecfa11276. #754's #4946 entry in catalogs/us-equities/runtime-target.json shifted the alpaca selected_path line from 158 to 159, so the foundation catalog's six alpaca-py pin_source citations were remapped through the difflib line map (MERGE_HEAD to merged, 6 changed, 0 unmapped). The catalog digest changed the handbook inputs; regenerated with --write, and --check passed.", + "previous_outputs": { + "docs/new-wsl-handbook.md": "b1dd8bdb72f32d3ab449aa90014b9621f779a5a7b3db590e4253120ba6c0bdfe", + "docs/new-wsl-handbook.json": "32da2ef7bbb47cbd8e75a1674b19c9dd1ee8e3cc855a05efee322765186e079e" + }, + "check": "python3 scripts/build_new_wsl_handbook.py --check" } ] } diff --git a/manifests/evidence.json b/manifests/evidence.json index 9dfe095f7..d90487687 100644 --- a/manifests/evidence.json +++ b/manifests/evidence.json @@ -14041,6 +14041,56 @@ "sha256": "44c902e51e44718a39500f2dc4e0061f5fa8a178aa15861bff83b36fbd60fe0f", "bytes": 47180 }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/README.md", + "sha256": "41356b001e424f10d6e4be8668a8991a83c8ffb72b12cd00bd6ff9b5c55c4036", + "bytes": 36439 + }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.188e0078fc8f", + "sha256": "188e0078fc8f4a36ef54eb856bf2ef2660cad1ae280332dc86e405b7a230a0f5", + "bytes": 51113 + }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.5a061ff06f41", + "sha256": "5a061ff06f41fa1b5721f10fb01037ce790639f715481c2efdd98ea32700ee3b", + "bytes": 62212 + }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/harness/run.py.67429251370f", + "sha256": "67429251370f1d397e53e69c5ff0b8b0ef7c2cb0e069687f0b087e6cca2ba013", + "bytes": 58848 + }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-passed.json", + "sha256": "757252a952b5b3481f9c51c89e76b2947e40bfdeba5187d0b17b5c2c8bfb43dd", + "bytes": 31250 + }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-run1-not-connected.json", + "sha256": "3275357f7673f50d41bda2686ed6712d2202b7b02180f9b8e5cb9120a9f977df", + "bytes": 6092 + }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/evidence/receipt-20261005-run2-failed.json", + "sha256": "2778c4d476c1f9a711590acd3b95155958b5af1b12d2d6d85b49b6ca9db430a9", + "bytes": 14712 + }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/plan.json", + "sha256": "24ffca56dfa7bd647bcf423dbc91ae3f605384a81a0ad9523cc024a36ac3ec72", + "bytes": 3390 + }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/run.py", + "sha256": "af6c817e73938bb446a9026e1713c3c35db7a519b7e5ff9162523a916f35a091", + "bytes": 79117 + }, + { + "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5/safety.py", + "sha256": "4165946e022deeb7725e63041c0f5c8a529a6490d46bf78d4b1bd131a1a90a1d", + "bytes": 5047 + }, { "path": "blueprints/us-equities/engine-nautilus/ibkr-paper-orders/README.md", "sha256": "e8c551a8082ebb5909ed00507d39a635b995e61d44546860710108040becb2f4", @@ -16003,7 +16053,7 @@ }, { "path": "catalogs/foundation/new-wsl-architecture-20261001.json", - "sha256": "8420f58ef2f613bab8da7a4486e907a9118b178d9d8e5eb8c70e4d84950c6c73", + "sha256": "ebad29544bcd86c22b0fc4d76c2882156c3735b8d003b8b1259e4bf96ba22d8e", "bytes": 322942 }, { @@ -16448,8 +16498,8 @@ }, { "path": "catalogs/us-equities/runtime-target.json", - "sha256": "110b4c884323b0ead55981925bdb0645fedcff1ed02af375a5f12be0e3a2c617", - "bytes": 25545 + "sha256": "8e7dfb142491128ab77e4ce462c0d088c587c1500c394d58b16bbbc9b179baf6", + "bytes": 26361 }, { "path": "catalogs/us-equities/security-identity-review.json", @@ -17358,12 +17408,12 @@ }, { "path": "docs/new-wsl-handbook.json", - "sha256": "32da2ef7bbb47cbd8e75a1674b19c9dd1ee8e3cc855a05efee322765186e079e", + "sha256": "e30b11544f28a0a8682f930afef47a2af9c441591d25a8dbadbfb255921d8212", "bytes": 685881 }, { "path": "docs/new-wsl-handbook.md", - "sha256": "b1dd8bdb72f32d3ab449aa90014b9621f779a5a7b3db590e4253120ba6c0bdfe", + "sha256": "aa6515ab958fd48038e853e91ebf29e196da7d4d6736aa90240c12a8cf6dabd5", "bytes": 362513 }, { @@ -39498,8 +39548,8 @@ }, { "path": "evidence/artifacts/new-wsl-handbook-20261001/receipt.json", - "sha256": "36385bc3af6e6cfdaac4dc6fbdfad33eb8f149a80ec726dcc8a130c8689b8749", - "bytes": 12145 + "sha256": "1845593bc80d441b231b59ab2cbf567cc94ac8f3893797bf818c64e4c0d50a35", + "bytes": 12991 }, { "path": "evidence/artifacts/new-wsl-handbook-20261001/slot-state-and-core-recipes.json", @@ -54601,6 +54651,11 @@ "sha256": "787be818d3b4e5693597979da34fa09683697275de146520ccc438c3b897afc6", "bytes": 67010 }, + { + "path": "tests/test_ibkr_paper_orders_rc5.py", + "sha256": "4a058d2fbb31c6d5007035ce49941a7ee67b0cd538c408a0c9a6804fd4736475", + "bytes": 90541 + }, { "path": "tests/test_ingest_snapshot.py", "sha256": "a0b7e43afafcffcc942483398f99d3ad476fcef9df09d85c30fef5bbde6946b4", diff --git a/tests/test_ibkr_paper_orders_rc5.py b/tests/test_ibkr_paper_orders_rc5.py new file mode 100644 index 000000000..ab5494500 --- /dev/null +++ b/tests/test_ibkr_paper_orders_rc5.py @@ -0,0 +1,1516 @@ +"""Offline local integration checks and synthetic fixtures; never a broker run. + +Native configuration/model checks require exactly rc5 and otherwise skip. +Every orchestration test supplies fake checker/process results. No test starts +a LiveNode, official-ibapi reader or gateway connection. +""" +import copy +import asyncio +import contextlib +import importlib.util +import io +import json +import re +import os +import stat +import tempfile +import time +import unittest +from datetime import datetime, timezone +from pathlib import Path +from types import SimpleNamespace +from unittest import mock +from zoneinfo import ZoneInfo + +ROOT = Path(__file__).resolve().parents[1] +DIRECTORY = ROOT / "blueprints/us-equities/engine-nautilus/ibkr-paper-orders-rc5" +SPEC = importlib.util.spec_from_file_location("ibkr_orders_rc5", DIRECTORY / "run.py") +RUN = importlib.util.module_from_spec(SPEC) +SPEC.loader.exec_module(RUN) +PLAN = RUN.load_plan() +PINNED = {"nautilus_trader": "2.0.0rc5", "ibapi": "10.45.1"} +NY = ZoneInfo("America/New_York") +# Generated synthetic identifier, never an actual account. +FAKE_ACCOUNT = "DU" + "7654321" +FAKE_HOME = "/" + "home/" + "synthetic" +NOW = datetime(2026, 10, 5, 10, tzinfo=NY).astimezone(timezone.utc) +STOP_NS = 2_000_000_000 +FLAT = {"client": "ibapi", "client_id": 92, "status": "passed", + "observed": {"account_count": 1, "paper_accounts": True, "positions": 0, "open_orders": 0}} + + +def admitted_quote(): + result = RUN.admit_quote(PLAN, {"bid": "770.01", "ask": "770.02", "quote_time_epoch": 1000}, 1001) + result["admitted_monotonic"] = time.monotonic() + return result + + +def node_payload(receipt): + admission = admitted_quote() + return {"account_id": FAKE_ACCOUNT, "receipt": receipt, "node_stop_at": time.monotonic() + 30, + "run_nonce": receipt.get("run_nonce"), "quote_admission": admission, "tob_offset_ticks": admission["tob_offset_ticks"]} + + +def event(kind, order="O1", typ="LIMIT", side="BUY", when=1_000_000_000, **changes): + value = {"type": kind, "order": order, "order_type": typ, "side": side, + "quantity": "1", "price": "765.00" if typ == "LIMIT" else None, + "instrument_id": "SPY=STK.SMART", "ts_event": when, "ts_init": when, + "at": RUN.frozen().ns_to_iso(when), "reconciliation": False} + if kind == "OrderFilled": + value.update(fill_price="770.00" if side == "BUY" else "769.94", fill_quantity="1", + commission="1.00", commission_currency="USD", currency="USD") + value.update(changes) + return value + + +def sequence(): + # ExecTester opens the market position before resting the limit: case labels + # describe obligations rather than implying the frozen strategy's chronology. + return [event("OrderInitialized", "O2", "MARKET"), + event("OrderSubmitted", "O2", "MARKET"), + event("OrderFilled", "O2", "MARKET"), + event("OrderInitialized"), event("OrderSubmitted"), event("OrderAccepted"), + event("OrderPendingCancel", when=3_000_000_000), event("OrderCanceled", when=3_000_000_000), + event("OrderInitialized", "O3", "MARKET", "SELL", 3_000_000_000), + event("OrderSubmitted", "O3", "MARKET", "SELL", 3_000_000_000), + event("OrderFilled", "O3", "MARKET", "SELL", 3_000_000_000)] + + +def cache_order(client_id, events, *, strategy="EXEC_TESTER-001", status="FILLED"): + """Synthetic native histories, including external orders claimed by ExecTester.""" + native = [] + for index, record in enumerate(events): + item = type(record["type"], (), {})() + item.__dict__.update(record, event_id=f"{client_id}-{index}", client_order_id=client_id, + strategy_id=strategy, trader_id="TESTER-001") + if record["type"] == "OrderFilled": + item.last_px, item.last_qty, item.currency = record["fill_price"], record["fill_quantity"], "USD" + item.commission = SimpleNamespace(as_decimal=lambda: RUN.decimal("1.00"), currency="USD") + native.append(item) + first = events[0] + return SimpleNamespace(client_order_id=client_id, strategy_id=strategy, trader_id="TESTER-001", + ts_init=first["ts_init"], events=lambda: native, + order_type=SimpleNamespace(name=first["order_type"]), side=SimpleNamespace(name=first["side"]), + instrument_id=first["instrument_id"], quantity=first["quantity"], price=first["price"], + status=SimpleNamespace(name=status), venue_order_id=None, time_in_force=SimpleNamespace(name="IOC")) + + +class RunOrderOwnership(unittest.TestCase): + def test_started_second_is_required_even_when_skew_and_init_time_fit(self): + scope = RUN.RunOrderScope(2_100_000_000) + order = cache_order("O-19700101-000001-001-001-1", [ + event("OrderInitialized", typ="MARKET", when=2_500_000_000)], status="INITIALIZED") + native = order.events()[0] + self.assertGreaterEqual(native.ts_init, scope.started_ns) + # Removing the started-second guard leaves every other condition true. + self.assertEqual(native.ts_init // 1_000_000_000 - 1, 1) + self.assertFalse(scope.generated_here(native)) + self.assertEqual(scope.partition([order])[0], []) + + def test_generated_identity_excludes_old_or_mismatched_intents(self): + scope = RUN.RunOrderScope(2_100_000_000) + cases = [ + ("O-19700101-000001-001-001-1", 3_000_000_000), + ("O-19700101-000002-001-001-1", 4_000_000_000), + ("O-19700101-000002-002-001-1", 3_000_000_000), + ("O-19700101-000002-001-002-1", 3_000_000_000), + ("O-19700101-000002-001-001-0", 3_000_000_000), + ] + for client_id, ts_init in cases: + with self.subTest(client_id=client_id, ts_init=ts_init): + order = cache_order(client_id, [event("OrderInitialized", typ="MARKET", when=ts_init)], status="INITIALIZED") + self.assertFalse(scope.generated_here(order.events()[0])) + self.assertEqual(scope.partition([order])[0], []) + current = cache_order("O-19700101-000002-001-001-1", [event("OrderInitialized", typ="MARKET", when=3_000_000_000)]) + self.assertTrue(scope.generated_here(current.events()[0])) + + def test_run3_old_generator_id_without_submission_stays_external(self): + old = cache_order("O-19700101-000001-001-001-2", [ + event("OrderInitialized", typ="MARKET", when=3_000_000_000), + event("OrderAccepted", typ="MARKET", when=1_000_000_000, reconciliation=True), + event("OrderFilled", typ="MARKET", when=1_000_000_000, reconciliation=True)], status="ACCEPTED") + cache = SimpleNamespace(orders=lambda: [old]) + scope = RUN.RunOrderScope(2_100_000_000) + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + RUN.update_observations(receipt, RUN.snapshot_events(cache, scope), PLAN) + receipt["reconciled_external"] = RUN.snapshot_events(cache, scope, external=True) + self.assertEqual(len(receipt["reconciled_external"]), 3) + self.assertEqual(receipt["events"], []) + self.assertEqual(receipt["fills"], []) + self.assertEqual(receipt["failures"], []) + self.assertTrue(RUN.stop_ready(RUN.tester_orders(cache, scope), 1, 2)) + + def test_pending_native_intent_and_pre_submit_denial_are_owned(self): + client_id = "O-19700101-000001-001-001-9" + initialized = event("OrderInitialized", typ="MARKET") + pending = cache_order(client_id, [initialized], status="INITIALIZED") + cache = SimpleNamespace(orders=lambda: [pending]) + scope = RUN.RunOrderScope(900_000_000) + self.assertEqual(RUN.tester_orders(cache, scope), [pending]) + self.assertTrue(RUN.orders_in_flight(RUN.tester_orders(cache, scope))) + self.assertEqual(RUN.snapshot_events(cache, scope, external=True), []) + denied = cache_order(client_id, [initialized, event("OrderDenied", typ="MARKET", when=1_100_000_000)], status="DENIED") + cache.orders = lambda: [denied] + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + RUN.update_observations(receipt, RUN.snapshot_events(cache, scope), PLAN) + self.assertEqual(receipt["cases"]["C3"]["outcome"], "failed") + self.assertEqual(receipt["cases"]["C3"]["events"][-1]["type"], "OrderDenied") + self.assertFalse(RUN.orders_in_flight(RUN.tester_orders(cache, scope))) + many = [cache_order(f"O-19700101-000001-001-001-{i}", [initialized, event("OrderDenied", typ="MARKET")], status="DENIED") for i in range(1, 8)] + cache.orders = lambda: many + RUN.update_observations(receipt, RUN.snapshot_events(cache, scope), PLAN) + self.assertTrue(any(f["reason"] == "observed_order_budget_breach" for f in receipt["failures"])) + + def test_run2_external_fills_then_current_submissions(self): + # Run 2 imported fills had a fresh, non-reconciled initialization and + # even the claimed EXEC_TESTER strategy. Neither establishes ownership. + history = [] + for side, price in (("BUY", "772.93"), ("SELL", "772.90")): + history.append(cache_order("historical-" + side, [ + event("OrderInitialized", typ="MARKET", side=side, when=1_000_000_000), + event("OrderAccepted", typ="MARKET", side=side, when=500_000_000, reconciliation=True), + event("OrderFilled", typ="MARKET", side=side, when=500_000_000, reconciliation=True, fill_price=price)], + status="ACCEPTED")) # Nonterminal external MARKET must not defer stop either. + cache = SimpleNamespace(orders=lambda **kwargs: list(history)) + scope = RUN.RunOrderScope(900_000_000) + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + + def observe(): + RUN.update_observations(receipt, RUN.snapshot_events(cache, scope), PLAN) + receipt["reconciled_external"] = RUN.snapshot_events(cache, scope, external=True) + + observe() + self.assertEqual([c["outcome"] for c in receipt["cases"].values()], ["not_run"] * 4) + self.assertEqual(receipt["fills"], []) + self.assertIsNone(receipt["roundtrip"]) + self.assertEqual(receipt["failures"], []) + self.assertTrue(RUN.stop_ready(RUN.tester_orders(cache, scope), 1, 2)) + external = copy.deepcopy(receipt["reconciled_external"]) + self.assertEqual({e["fill_price"] for e in external if e["type"] == "OrderFilled"}, {"772.93", "772.90"}) + for alias in ("O1", "O2", "O3"): + history.append(cache_order("current-" + alias, [e for e in sequence() if e["order"] == alias])) + receipt["node"]["stop_ns"], receipt["flat_proof"] = STOP_NS, copy.deepcopy(FLAT) + observe() + self.assertEqual([c["outcome"] for c in receipt["cases"].values()], ["passed"] * 4) + self.assertEqual(receipt["roundtrip"]["net_usd"], "-2.06") + self.assertEqual(len(receipt["fills"]), 2) + self.assertEqual(receipt["failures"], []) + self.assertEqual(receipt["reconciled_external"], external) + raw = RUN.serialized_receipt(receipt) + self.assertNotIn("historical-", raw) + self.assertNotIn("current-", raw) + + def test_lineage_needs_local_init_and_submit_for_same_strategy_and_id(self): + local = sequence()[:3] + cases = [cache_order("own", local), cache_order("wrong-strategy", local, strategy="OTHER-001"), + cache_order("old", [dict(e, ts_init=100) for e in local]), + cache_order("reconciled-init", [dict(local[0], reconciliation=True), *local[1:]]), + cache_order("reconciled-submit", [local[0], dict(local[1], reconciliation=True), local[2]])] + mismatch = cache_order("mismatch", local) + mismatch.events()[1].client_order_id = "different" + cases.append(mismatch) + cache = SimpleNamespace(orders=lambda **kwargs: cases) + scope = RUN.RunOrderScope(900_000_000) + self.assertEqual([str(o.client_order_id) for o in RUN.tester_orders(cache, scope)], ["own"]) + # Reconciliation on an already owned order still fails its cases. + cases[0].events()[-1].reconciliation = True + outcomes = RUN.case_outcomes(RUN.snapshot_events(cache, scope)) + self.assertEqual(outcomes["C3"]["outcome"], "failed") + + +class PlanValidation(unittest.TestCase): + def test_frozen_plan(self): + self.assertEqual(RUN.validate_plan(PLAN), []) + self.assertEqual(PLAN["bounds"]["max_orders"], 6) + self.assertEqual(PLAN["timeouts"]["overall_deadline_seconds"], 420) + + def test_destination_keeps_rc5_route_issue_and_rerun_trigger(self): + destination = json.loads((ROOT / "catalogs/us-equities/runtime-target.json").read_text()) + ibkr = next(boundary for boundary in destination["broker_boundaries"] if boundary["id"] == "ibkr") + issues = {entry["ref"].rsplit("/", 1)[-1]: entry for entry in ibkr["rc5_blockers"]} + self.assertTrue({"4946", "5007", "5057", "5060"} <= set(issues)) + self.assertEqual(issues["4946"]["state"], "closed") + self.assertIn("selected rc5 pin remains affected", issues["4946"]["upstream_fix"]) + self.assertIn("over-limit", issues["4946"]["rerun_trigger"]) + + def test_bound_mutations(self): + mutations = [("max_orders", 7), ("max_orders", True), ("max_quantity_per_order", 2), + ("max_quantity_per_order", True), ("max_notional_per_order_usd", 1001), + ("max_notional_per_order_usd", 0), ("max_notional_per_order_usd", float("nan")), + ("max_roundtrip_loss_usd", 5.01), ("max_roundtrip_loss_usd", float("inf"))] + for name, value in mutations: + with self.subTest(name=name, value=value): + plan = copy.deepcopy(PLAN) + plan["bounds"][name] = value + self.assertTrue(RUN.validate_plan(plan)) + + def test_symbol_host_account_and_strategy_scope(self): + for section, key, value in [("instrument", "symbol", "AAPL"), ("instrument", "currency", "EUR"), + ("exec_tester", "order_qty", "2"), ("exec_tester", "enable_limit_sells", True), + ("exec_tester", "enable_brackets", True), ("risk", "bypass", True), + ("risk", "require_enforced_notional_before_connection", True), + ("session", "weekdays_only", False), ("session", "close_buffer_minutes", 9)]: + with self.subTest(section=section, key=key): + plan = copy.deepcopy(PLAN) + plan[section][key] = value + self.assertTrue(RUN.validate_plan(plan)) + for key, value in [("host", "192.0.2.1"), ("account_prefix", "U"), ("require_single_managed_account", False)]: + plan = copy.deepcopy(PLAN) + plan[key] = value + self.assertTrue(RUN.validate_plan(plan)) + + def test_ports_and_client_ids(self): + for port in (4001, 7496, 0, True, 4002.0): + self.assertTrue(RUN.validate_plan(PLAN, port=port)) + for port in (4002, 7497): + self.assertEqual(RUN.validate_plan(PLAN, port=port), []) + for node, check in ((92, 92), (91, 91), (101, 92), (91.0, 92), (91, 92.0)): + self.assertTrue(RUN.validate_plan(PLAN, node_client_id=node, check_client_id=check)) + plan = copy.deepcopy(PLAN) + plan["paper_ports"] = [4001] + self.assertTrue(RUN.validate_plan(plan)) + + def test_malformed(self): + for value in (None, [], {}, {"schema_version": 1}, "not a plan"): + self.assertTrue(RUN.validate_plan(value)) + + def test_duplicate_plan_keys(self): + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "duplicate.json" + path.write_text('{"host": "127.0.0.1", "host": "elsewhere"}') + with self.assertRaises(ValueError): + RUN.load_plan(path) + + def test_native_argument_types_are_checked_before_connection(self): + for section, key, value in (("exec_tester", "enable_limit_buys", 1), + ("exec_tester", "order_expire_time_delta_mins", 7.0), + ("timeouts", "node_start_seconds", 60.0)): + plan = copy.deepcopy(PLAN) + plan[section][key] = value + self.assertTrue(RUN.validate_plan(plan)) + plan = copy.deepcopy(PLAN) + plan["schema_version"] = True + self.assertTrue(RUN.validate_plan(plan)) + + def test_malformed_cli_is_refused_with_exit_three(self): + stderr = io.StringIO() + with contextlib.redirect_stderr(stderr), self.assertRaises(SystemExit) as refused: + RUN.parser().parse_args(["--port", FAKE_ACCOUNT]) + self.assertEqual(refused.exception.code, 3) + self.assertNotIn(FAKE_ACCOUNT, stderr.getvalue()) + + +class WindowsAndPins(unittest.TestCase): + def test_whole_run_and_buffer(self): + for hour, minute, allowed in ((9, 29, False), (9, 30, True), (15, 43, True), + (15, 44, False), (15, 50, False), (16, 0, False)): + now = NOW.astimezone(NY).replace(hour=hour, minute=minute) + self.assertEqual(RUN.frozen().rth_check(now, PLAN)[0], allowed) + + def test_weekend_and_early_close(self): + self.assertFalse(RUN.frozen().rth_check(NOW.replace(day=10), PLAN)[0]) + local = NOW.astimezone(NY) + sessions = [(local.replace(hour=9, minute=30), local.replace(hour=13, minute=0))] + self.assertFalse(RUN.frozen().rth_check(local.replace(hour=12, minute=44), PLAN, sessions)[0]) + self.assertFalse(RUN.frozen().rth_check(NOW, PLAN, [])[0]) + self.assertIsNone(RUN.frozen().parse_liquid_hours("malformed", "US/Eastern", local.date())) + + def test_runtime_pins(self): + self.assertIsNone(RUN.runtime_refusal(PINNED)) + for versions in ({}, {"nautilus_trader": "1.231.0", "ibapi": "10.45.1"}, + {"nautilus_trader": "2.0.0rc5", "ibapi": "10.44.0"}, + {"nautilus_trader": "2.0.0rc5", "ibapi": "10.45.1", "nautilus_trader_distribution": "1.231.0"}, + {"nautilus_trader": "2.0.0rc5.dev1", "ibapi": "10.45.1"}): + self.assertEqual(RUN.runtime_refusal(versions), "refused_unpinned_runtime") + + +class QuoteAdmission(unittest.TestCase): + def test_spread_and_expected_commissions_fit_the_loss_bound(self): + quote = {"bid": "770", "ask": "990", "quote_time_epoch": 1000} + result = RUN.admit_quote(PLAN, quote, 1001) + self.assertEqual(result["status"], "refused_quote_loss_bound") + self.assertEqual(result["stage"], "spread_and_commissions") + self.assertEqual(RUN.decimal(result["spread_loss_usd"]), 220) + self.assertEqual(RUN.decimal(result["expected_roundtrip_commissions_usd"]), RUN.decimal("2.04")) + self.assertEqual(RUN.decimal(result["expected_roundtrip_loss_usd"]), RUN.decimal("222.04")) + self.assertEqual(RUN.decimal(result["max_roundtrip_loss_usd"]), 5) + for ask, status in (("772.96", "passed"), ("772.97", "refused_quote_loss_bound")): + self.assertEqual(RUN.admit_quote(PLAN, dict(quote, ask=ask), 1001)["status"], status) + + def test_tier1_core_band_notional_boundary(self): + for ask, expected in (("942.85", "passed"), ("942.86", "refused_quote_notional"), ("990", "refused_quote_notional")): + with self.subTest(ask=ask): + quote = {"bid": str(RUN.decimal(ask) - RUN.decimal("0.01")), "ask": ask, "quote_time_epoch": 1000} + result = RUN.admit_quote(PLAN, quote, 1001) + self.assertEqual(result["status"], expected) + self.assertEqual(RUN.decimal(result["luld_core_band_fraction"]), RUN.decimal("0.05")) + stressed = RUN.decimal(ask) * RUN.decimal("1.05") + RUN.decimal("10") + self.assertEqual(RUN.decimal(result["band_adjusted_notional_usd"]) + RUN.decimal(result["headroom_usd"]), stressed) + self.assertEqual(stressed <= 1000, expected == "passed") + + def test_commission_allowance_covers_observed_sell_commission(self): + result = RUN.admit_quote(PLAN, {"bid": "775.24", "ask": "775.26", "quote_time_epoch": 1000}, 1001) + self.assertEqual(result["status"], "passed") + self.assertEqual(RUN.decimal(result["commission_margin_per_order_usd"]), RUN.decimal("0.02")) + self.assertGreaterEqual(RUN.decimal(result["expected_roundtrip_commissions_usd"]), RUN.decimal("1.00") + RUN.decimal("1.02")) + + def test_precheck_observed_is_frozen_before_quote_callbacks(self): + client = RUN.AdmissionState() + client.disconnect = mock.Mock() + + def precheck(*args, **kwargs): + client.managedAccounts(FAKE_ACCOUNT) + client.currentTime(1000) + client.r["server_minus_local_s"] = 0.1 + return {"status": "passed", "observed": client.r}, FAKE_ACCOUNT + + def quote(*args, **kwargs): + client.currentTime(1001) + client.error(9202, 1001, 10190, "quote error") + return {"status": "refused_quote_error"} + + with mock.patch.object(RUN.frozen(), "run_check", side_effect=precheck), mock.patch.object(RUN, "quote_check", side_effect=quote): + pre, _, _ = RUN.official_admission(PLAN, 4002, deadline=time.monotonic() + 1, client_factory=lambda: client) + self.assertEqual(pre["observed"]["server_time_epoch"], 1000) + self.assertEqual(pre["observed"]["server_minus_local_s"], 0.1) + self.assertEqual(pre["observed"]["errors"], []) + self.assertIsNot(pre["observed"], client.r) + self.assertIsNot(pre["observed"]["errors"], client.r["errors"]) + self.assertEqual(client.r["server_time_epoch"], 1001) + client.disconnect.assert_called_once() + + def test_precheck_exception_and_alarm_have_the_correct_phase(self): + for mode in ("exception", "alarm", "caught-alarm"): + with self.subTest(mode=mode): + client = RUN.AdmissionState() + client.disconnect = mock.Mock() + + def precheck(*args, **kwargs): + client.mark("connected") + client.managedAccounts(FAKE_ACCOUNT) + if mode == "exception": + raise RuntimeError("positions wait failed " + FAKE_ACCOUNT) + handler = RUN.signal.getsignal(RUN.signal.SIGALRM) + if mode == "caught-alarm": + try: + handler(None, None) + except OSError: + return {"status": "not_connected", "observed": client.r}, None + handler(None, None) + + with mock.patch.object(RUN.frozen(), "run_check", side_effect=precheck), mock.patch.object(RUN, "quote_check") as quote: + pre, account, result = RUN.official_admission(PLAN, 4002, deadline=time.monotonic() + 1, client_factory=lambda: client) + self.assertEqual(pre["status"], "incomplete") + self.assertEqual(RUN.frozen().exit_code_for(pre["status"]), 1) + self.assertIn("pre_check", pre["error"]) + self.assertNotIn("quote_check_deadline", json.dumps((pre, result))) + self.assertNotIn(FAKE_ACCOUNT, json.dumps((pre, result))) + self.assertEqual(pre["observed"]["account_count"], 1) + self.assertIsNone(account) + self.assertEqual(result["status"], "not_run") + quote.assert_not_called() + client.disconnect.assert_called_once() + + def test_unrelated_farm_notices_are_info_and_relevant_errors_are_fatal(self): + state = RUN.QuoteState() + state.managedAccounts(FAKE_ACCOUNT) + for code in (2103, 2105, 2157, 2104, 2106, 2158, 1101, 1102): + state.error(-1, 1000, code, "farm notice " + FAKE_ACCOUNT) + state.error(999, 1000, 354, "unrelated entitlement") + self.assertEqual(state.errors, []) + self.assertEqual(len(state.info), 9) + self.assertNotIn(FAKE_ACCOUNT, json.dumps(state.diagnostics())) + for req_id, code in ((9202, 354), (-1, 326), (-1, 1100), (-1, 10089), (-1, 10167)): + other = RUN.QuoteState() + other.error(req_id, 1000, code, "failure") + self.assertEqual(other.errors[0]["reqId"], req_id) + self.assertEqual(other.errors[0]["code"], code) + + def test_quote_wait_deadlines_identify_each_missing_callback(self): + for missing, stage, status in (("nextValidId", "accounts", "not_connected"), + ("quote", "tick requested", "refused_quote_unavailable"), + ("clock", "quote", "refused_quote_clock")): + class Client(RUN.QuoteState): + def isConnected(self): + return True + + def reqMarketDataType(self, kind): + pass + + def reqTickByTickData(self, *args): + if missing != "quote": + self.tickByTickBidAsk(9202, 1000, 770, 771, 1, 1, None) + + def reqCurrentTime(self): + if missing != "clock": + self.currentTime(1001) + + def cancelTickByTickData(self, req_id): + pass + + client = Client() + client.mark("connected") + if missing != "nextValidId": + client.nextValidId(1) + client.managedAccounts(FAKE_ACCOUNT) + with mock.patch.object(RUN.frozen(), "spy_contract", return_value=object()): + result = RUN.quote_check(PLAN, FAKE_ACCOUNT, deadline=time.monotonic() + 0.01, client=client) + self.assertEqual((result["last_stage"], result["status"]), (stage, status)) + self.assertIn(stage, result["stage_duration_seconds"]) + self.assertGreaterEqual(result["elapsed_seconds"], 0.01) + self.assertEqual(result["error"], "quote_check_deadline") + + def test_sigalrm_path_preserves_stages_codes_and_teardown(self): + client = RUN.AdmissionState() + client.disconnect = mock.Mock() + + def precheck(*args, **kwargs): + client.mark("connected") + client.nextValidId(1) + client.managedAccounts(FAKE_ACCOUNT) + return copy.deepcopy(FLAT), FAKE_ACCOUNT + + def interrupted(*args, **kwargs): + client.mark("tick requested") + client.error(-1, 1000, 2105, "notice " + FAKE_ACCOUNT) + client.error(9202, 1000, 10190, "limit " + FAKE_ACCOUNT) + raise RUN.CheckTimeout("quote_check_deadline") + + with mock.patch.object(RUN.frozen(), "run_check", side_effect=precheck), mock.patch.object(RUN, "quote_check", side_effect=interrupted): + pre, account, result = RUN.official_admission(PLAN, 4002, deadline=time.monotonic() + 1, client_factory=lambda: client) + self.assertEqual(pre["status"], "passed") + self.assertEqual(account, FAKE_ACCOUNT) + self.assertEqual(result["last_stage"], "tick requested") + self.assertEqual(result["error"], "quote_check_deadline") + self.assertEqual(result["errors"][0]["code"], 10190) + self.assertEqual(result["info"][0]["code"], 2105) + self.assertNotIn(FAKE_ACCOUNT, json.dumps(result)) + client.disconnect.assert_called_once() + + def test_connectivity_failure_wakes_the_quote_wait(self): + client = RUN.QuoteState() + client.isConnected = lambda: True + client.mark("connected") + client.managedAccounts(FAKE_ACCOUNT) + client.error(-1, 1000, 326, "client id in use") + result = RUN.quote_check(PLAN, FAKE_ACCOUNT, deadline=time.monotonic() + 1, client=client) + self.assertEqual(result["errors"][0]["code"], 326) + self.assertLess(result["elapsed_seconds"], 0.1) + + def test_fresh_quote_and_headroom(self): + admitted = admitted_quote() + self.assertEqual(admitted["status"], "passed") + self.assertEqual((admitted["bid"], admitted["ask"], admitted["quote_age_seconds"]), ("770.01", "770.02", 1)) + for ask, expected in (("1001", "refused_quote_notional"), ("991", "refused_quote_notional"), ("990", "refused_quote_notional"), ("942.85", "passed")): + result = RUN.admit_quote(PLAN, {"bid": str(RUN.decimal(ask) - RUN.decimal("0.01")), "ask": ask, "quote_time_epoch": 1000}, 1001) + self.assertEqual(result["status"], expected) + + def test_stale_future_invalid_and_delayed_quotes(self): + quote = {"bid": "770", "ask": "771", "quote_time_epoch": 1000} + for server_time, expected in ((1010, "passed"), (1011, "refused_stale_quote"), (997, "refused_stale_quote")): + self.assertEqual(RUN.admit_quote(PLAN, quote, server_time)["status"], expected) + self.assertEqual(RUN.admit_quote(PLAN, quote, 1001, elapsed_seconds=10)["status"], "refused_stale_quote") + for code in (10089, 10167): + self.assertEqual(RUN.admit_quote(PLAN, quote, 1001, errors=[code])["status"], "refused_delayed_quote") + for bad in ({"bid": "0", "ask": "1", "quote_time_epoch": 1000}, + {"bid": "771", "ask": "770", "quote_time_epoch": 1000}, + {"bid": "NaN", "ask": "770", "quote_time_epoch": 1000}, None): + self.assertEqual(RUN.admit_quote(PLAN, bad, 1001)["status"], "refused_invalid_quote") + + def test_delayed_callbacks_are_latched(self): + for tick in RUN.DELAYED_TICKS: + state = RUN.QuoteState() + state.tickPrice(9202, tick, 770, None) + state.tickByTickBidAsk(9202, 1000, 770, 771, 1, 1, None) + self.assertTrue(state.delayed) + for code in (10089, 10167): + state = RUN.QuoteState() + state.error(9202, 1000, code, FAKE_ACCOUNT) + self.assertTrue(state.delayed) + self.assertIn(code, [entry["code"] for entry in state.errors]) + state = RUN.QuoteState() + state.marketDataType(9202, 3) + self.assertTrue(state.delayed) + + def test_half_bid_offset_and_child_validation(self): + self.assertEqual(RUN.resting_offset_ticks("770.01", PLAN), 38500) + self.assertEqual(RUN.resting_offset_ticks("770.03", PLAN), 38501) + payload = node_payload({}) + self.assertTrue(RUN.validate_child_admission(PLAN, payload)) + for mutate in (lambda p: p.update(tob_offset_ticks=500), + lambda p: p.update(tob_offset_ticks=38500.0), + lambda p: p["quote_admission"].update(ask="1001"), + lambda p: p["quote_admission"].update(status="refused_delayed_quote"), + lambda p: p["quote_admission"].update(admitted_monotonic=time.monotonic() - 11)): + changed = copy.deepcopy(payload) + mutate(changed) + self.assertFalse(RUN.validate_child_admission(PLAN, changed)) + self.assertFalse(RUN.validate_child_admission(PLAN, {})) + + def test_invalid_child_admission_never_builds_node(self): + with tempfile.TemporaryDirectory() as directory, mock.patch.object(RUN, "build_node") as build: + path = Path(directory) / "receipt.json" + receipt = asyncio.run(RUN.node_phase(PLAN, 4002, path, {})) + self.assertEqual(receipt["status"], "refused_child_quote_admission") + self.assertEqual(json.loads(path.read_text())["exit_code"], 3) + build.assert_not_called() + + def test_official_requests_with_synthetic_client(self): + calls = [] + + class Client(RUN.AdmissionState): + def __init__(self): + super().__init__() + self.closed = RUN.threading.Event() + self.connected = False + + def connect(self, host, port, client_id): + calls.append(("connect", host, port, client_id)) + self.connected = True + + def run(self): + self.nextValidId(1) + self.managedAccounts(FAKE_ACCOUNT) + self.closed.wait(1) + + def isConnected(self): + return self.connected + + def reqPositions(self): + self.positionEnd() + + def cancelPositions(self): + pass + + def reqAllOpenOrders(self): + self.openOrderEnd() + + def reqContractDetails(self, req_id, contract): + self._liquid_hours = "20261005:0930-20261005:1600" + self._time_zone_id = "US/Eastern" + self.done["contract"].set() + + def reqMarketDataType(self, kind): + calls.append(("data_type", kind)) + + def reqTickByTickData(self, req_id, contract, tick_type, count, ignore_size): + calls.append(("quote", req_id, tick_type, count, ignore_size)) + self.tickByTickBidAsk(req_id, 1000, 770, 771, 1, 1, None) + + def reqCurrentTime(self): + calls.append(("clock",)) + self.currentTime(1001) + + def cancelTickByTickData(self, req_id): + calls.append(("cancel", req_id)) + + def disconnect(self): + calls.append(("disconnect",)) + self.connected = False + self.closed.set() + + with mock.patch.object(RUN.frozen(), "spy_contract", return_value=object()): + pre, account, result = RUN.official_admission(PLAN, 4002, deadline=time.monotonic() + 1, client_factory=Client) + self.assertEqual(pre["status"], "passed") + self.assertEqual(account, FAKE_ACCOUNT) + self.assertEqual(result["status"], "passed") + self.assertIn(("quote", 9202, "BidAsk", 0, False), calls) + self.assertIn(("connect", "127.0.0.1", 4002, 92), calls) + self.assertEqual(sum(call[0] == "connect" for call in calls), 1) + self.assertEqual(sum(call[0] == "clock" for call in calls), 2) + self.assertLess(calls.index(("clock",)), calls.index(("quote", 9202, "BidAsk", 0, False))) + self.assertEqual(calls[-3], ("clock",)) + self.assertEqual(result["last_stage"], "clock") + self.assertEqual(set(result["stage_elapsed_seconds"]), {"connected", "nextValidId", "accounts", "tick requested", "quote", "clock"}) + self.assertEqual(calls[-1], ("disconnect",)) + + +class StopLifecycle(unittest.IsolatedAsyncioTestCase): + @staticmethod + def order(status, venue_id=None, order_type="LIMIT", tif="GTD"): + return SimpleNamespace(status=SimpleNamespace(name=status), venue_order_id=venue_id, + order_type=SimpleNamespace(name=order_type), time_in_force=SimpleNamespace(name=tif)) + + def test_deferred_stop_and_deadline(self): + for status, venue_id, expected in (("SUBMITTED", "venue", False), ("INITIALIZED", None, False), + ("ACCEPTED", None, False), ("ACCEPTED", "venue", True), + ("FILLED", None, True), ("DENIED", None, True)): + orders = [self.order(status, venue_id)] + self.assertEqual(RUN.stop_ready(orders, 1, 2), expected) + self.assertTrue(RUN.stop_ready(orders, 2, 2)) + for order_type, tif in (("MARKET", "GTC"), ("MARKET", "IOC"), ("LIMIT", "IOC"), ("LIMIT", "FOK")): + self.assertFalse(RUN.stop_ready([self.order("ACCEPTED", "venue", order_type, tif)], 1, 2)) + self.assertTrue(RUN.stop_ready([self.order("FILLED", "venue", order_type, tif)], 1, 2)) + + async def test_external_startup_fills_do_not_trigger_case_failure(self): + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + history = [cache_order("external-" + side, [ + event("OrderInitialized", typ="MARKET", side=side, when=time.time_ns()), + event("OrderFilled", typ="MARKET", side=side, when=100, reconciliation=True)]) + for side in ("BUY", "SELL")] + done = asyncio.Event() + before_submit = [] + + def current_order(alias, records): + when = time.time_ns() + return cache_order("current-" + alias, [dict(e, ts_init=when, ts_event=when) for e in records]) + + def stop(): + done.set() + limit = next(o for o in history if str(o.client_order_id) == "current-O1") + cancels = current_order("O1", sequence()[6:8]).events() + for item in cancels: + item.event_id += "-cancel" + limit.events().extend(cancels) + history.append(current_order("O3", sequence()[8:])) + + async def run_async(): + await asyncio.sleep(0.15) + before_submit.append(copy.deepcopy(receipt["cases"])) + history.extend([current_order("O2", sequence()[:3]), current_order("O1", sequence()[3:6])]) + await done.wait() + + node = SimpleNamespace(cache=SimpleNamespace(orders=lambda: list(history)), + handle=lambda: SimpleNamespace(is_running=True, stop=stop), run_async=run_async, + dispose=mock.Mock()) + loop = asyncio.get_running_loop() + with mock.patch.object(RUN, "build_node", return_value=node), mock.patch.object(RUN, "write_receipt"), mock.patch.object(loop, "add_signal_handler"), mock.patch.object(loop, "remove_signal_handler"): + result = await asyncio.wait_for(RUN.node_phase(PLAN, 4002, Path("unused-receipt"), node_payload(receipt)), 2) + self.assertEqual([c["outcome"] for c in before_submit[0].values()], ["not_run"] * 4) + self.assertEqual(result["node"]["stop_reason"], "C1_accepted_C3_filled") + self.assertEqual(result["failures"], []) + self.assertEqual([result["cases"][c]["outcome"] for c in RUN.CASE_IDS], ["passed", "passed", "passed", "filled_unproven"]) + self.assertEqual(len(result["fills"]), 2) + self.assertEqual(len(result["reconciled_external"]), 4) + + async def exercise(self, *, observer_error=False): + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + done = asyncio.Event() + probes = [] + stop_calls = [] + disposed = [] + + def stop(): + stop_calls.append(len(probes)) + done.set() + + async def run_async(): + await done.wait() + await asyncio.sleep(0.15) # Simulate native post-stop callback grace. + + node = SimpleNamespace(cache=object(), handle=lambda: SimpleNamespace(is_running=True, stop=stop), + run_async=run_async, dispose=lambda: disposed.append(True)) + + def orders(_cache, _scope): + probes.append(True) + status = "SUBMITTED" if len(probes) == 1 else "ACCEPTED" if len(probes) == 2 else "FILLED" + return [self.order(status, "venue", "MARKET", "IOC")] + + snapshots = mock.Mock(side_effect=RuntimeError("snapshot failed")) if observer_error else mock.Mock(return_value=[event("OrderRejected")]) + writes = mock.Mock(side_effect=[RuntimeError("write failed"), None]) if observer_error else mock.Mock() + loop = asyncio.get_running_loop() + with mock.patch.object(RUN, "build_node", return_value=node), mock.patch.object(RUN, "snapshot_events", snapshots), mock.patch.object(RUN, "tester_orders", side_effect=orders), mock.patch.object(RUN, "write_receipt", writes), mock.patch.object(loop, "add_signal_handler") as add_signal, mock.patch.object(loop, "remove_signal_handler") as remove_signal: + result = await RUN.node_phase(PLAN, 4002, Path("unused-receipt"), node_payload(receipt)) + self.assertEqual(len(stop_calls), 1) + self.assertEqual(disposed, [True]) + self.assertEqual(add_signal.call_count, 3) + self.assertEqual(remove_signal.call_count, 3) + self.assertTrue(done.is_set()) + return result, stop_calls + + async def test_case_failure_waits_for_entry_then_stops_once(self): + _, stop_calls = await self.exercise() + self.assertGreaterEqual(stop_calls[0], 3) + + async def test_poll_and_final_snapshot_errors_still_stop_await_and_dispose(self): + receipt, _ = await self.exercise(observer_error=True) + reasons = [failure["reason"] for failure in receipt["failures"]] + self.assertTrue(any("node_poll" in reason for reason in reasons)) + self.assertTrue(any("final_snapshot" in reason for reason in reasons)) + self.assertTrue(any("final_receipt" in reason for reason in reasons)) + + +class EventsAndRoundtrip(unittest.TestCase): + def test_complete_sequence(self): + outcomes = RUN.case_outcomes(sequence(), stop_ns=STOP_NS, flat_proof=FLAT) + self.assertEqual([outcomes[c]["outcome"] for c in RUN.CASE_IDS], ["passed"] * 4) + self.assertEqual(outcomes["C1"]["order"], outcomes["C2"]["order"]) + self.assertEqual(outcomes["C3"]["order_type"], "MARKET") + self.assertEqual(outcomes["C1"]["price"], "765.00") + rt = RUN.roundtrip(sequence(), 5) + self.assertEqual((rt["gross_usd"], rt["commissions_usd"], rt["net_usd"]), ("-0.06", "2.00", "-2.06")) + self.assertFalse(rt["loss_bound_breach"]) + + def test_independent_flat_proof_required(self): + for proof in (None, {"status": "passed"}, {"status": "incomplete"}, + {"status": "passed", "observed": {"positions": 1, "open_orders": 0}}): + self.assertEqual(RUN.case_outcomes(sequence(), stop_ns=STOP_NS, flat_proof=proof)["C4"]["outcome"], "filled_unproven") + + def test_wrong_cancel_or_before_stop_cannot_pass(self): + events = sequence() + events[7]["order"] = "another_order" + self.assertNotEqual(RUN.case_outcomes(events, stop_ns=STOP_NS, flat_proof=FLAT)["C2"]["outcome"], "passed") + self.assertNotEqual(RUN.case_outcomes(sequence(), stop_ns=4_000_000_000, flat_proof=FLAT)["C2"]["outcome"], "passed") + self.assertNotEqual(RUN.case_outcomes(sequence(), stop_ns=4_000_000_000, flat_proof=FLAT)["C4"]["outcome"], "passed") + + def test_reconciliation_cannot_supply_venue_acceptance(self): + events = sequence() + events[5]["reconciliation"] = True + outcomes = RUN.case_outcomes(events, stop_ns=STOP_NS, flat_proof=FLAT) + self.assertEqual(outcomes["C1"]["outcome"], "failed") + self.assertEqual(outcomes["C2"]["outcome"], "failed") + + def test_missing_fill_and_wrong_quantity(self): + events = [e for e in sequence() if not (e["type"] == "OrderFilled" and e["side"] == "BUY")] + self.assertEqual(RUN.case_outcomes(events, stop_ns=STOP_NS, flat_proof=FLAT)["C3"]["outcome"], "incomplete") + events = sequence() + for e in events: + if e["order"] == "O2": + e["quantity"] = "2" + self.assertEqual(RUN.case_outcomes(events, stop_ns=STOP_NS, flat_proof=FLAT)["C3"]["outcome"], "failed") + + def test_accidental_resting_fill_fails(self): + events = sequence() + [event("OrderFilled")] + self.assertEqual(RUN.case_outcomes(events, stop_ns=STOP_NS, flat_proof=FLAT)["C1"]["outcome"], "failed") + self.assertFalse(RUN.roundtrip(events, 5)["closed"]) + + def test_missing_or_zero_commission_and_loss_breach(self): + for fee in (None, "0.00"): + events = sequence() + events[-1]["commission"] = fee + rt = RUN.roundtrip(events, 5) + self.assertTrue(rt["commission_unresolved"]) + self.assertIsNone(rt["net_usd"]) + events = sequence() + events[-1]["fill_price"] = "760.00" + self.assertTrue(RUN.roundtrip(events, 5)["loss_bound_breach"]) + + def test_empty_sequence(self): + self.assertIsNone(RUN.roundtrip([], 5)) + self.assertEqual([c["outcome"] for c in RUN.case_outcomes([]).values()], ["not_run"] * 4) + + def test_observed_bounds_never_pass_silently(self): + for field, value in (("quantity", "2"), ("price", "1000.01"), ("instrument_id", "AAPL=STK.SMART")): + events = sequence() + for e in events: + if e["order"] == "O1": + e[field] = value + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + RUN.update_observations(receipt, events, PLAN) + self.assertTrue(receipt["failures"]) + + def test_event_seconds_need_not_match_observation_nanoseconds(self): + events = sequence() + events[-1]["ts_event"] = 1_000_000_000 + self.assertEqual(RUN.case_outcomes(events, stop_ns=STOP_NS, flat_proof=FLAT)["C4"]["outcome"], "passed") + + +class SanitizationAndReceipt(unittest.TestCase): + def test_account_ids_endpoints_paths_and_amounts(self): + raw = {"error": f"IB-{FAKE_ACCOUNT}: margin [123456.78 USD], Available Funds: 98765.43 at 127.0.0.1:4002 {FAKE_HOME}/private/file.txt", + "nested": {"I1234567": ["U1234567", "DF1234567", "F1234567", FAKE_ACCOUNT]}, + "fill_price": "770.00", "commission": "1.00"} + encoded = RUN.serialized_receipt(raw, (FAKE_ACCOUNT,)) + self.assertIsNone(re.search(r"\b(?:D?[UF]|I)\d{5,}\b", encoded)) + for secret in (FAKE_ACCOUNT, "127.0.0.1", FAKE_HOME, "123456.78", "98765.43"): + self.assertNotIn(secret, encoded) + self.assertEqual(json.loads(encoded)["fill_price"], "770.00") + + def test_exact_known_secret_removed_without_word_boundaries(self): + self.assertNotIn(FAKE_ACCOUNT, RUN.serialized_receipt({"error": "prefix" + FAKE_ACCOUNT + "suffix"}, (FAKE_ACCOUNT,))) + + def test_source_urls_survive_but_untrusted_urls_are_scrubbed(self): + raw = {"source": RUN.RISK_ROUTE["source"], "error": f"https://github.com/nautechsystems/nautilus_trader/{FAKE_ACCOUNT}/127.0.0.1/file"} + encoded = RUN.serialized_receipt(raw, (FAKE_ACCOUNT,)) + self.assertEqual(json.loads(encoded)["source"], RUN.RISK_ROUTE["source"]) + self.assertNotIn("127.0.0.1", encoded) + + def test_shape_and_atomic_replacement(self): + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + for key in ("kind", "evidence_class", "versions", "plan_sha256", "harness_sha256", "source_hashes", + "pre_check", "cases", "fills", "roundtrip", "flat_proof", "status", "exit_code", "blocked_steps"): + self.assertIn(key, receipt) + self.assertEqual(receipt["kind"], RUN.KIND) + self.assertEqual(receipt["status"], "cleanup_required") + self.assertEqual(json.loads(RUN.serialized_receipt(receipt))["risk"]["note"], RUN.RISK_NOTE) + self.assertEqual(set(receipt["cases"]), {"C1", "C2", "C3", "C4"}) + for key in ("plan_sha256", "harness_sha256"): + self.assertRegex(receipt[key], r"^[a-f0-9]{64}$") + issues = {issue["number"]: issue["state"] for step in receipt["blocked_steps"] for issue in step["issues"]} + self.assertEqual(issues, {5007: "open", 5057: "open", 5060: "open"}) + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "receipt.json" + RUN.write_receipt(path, receipt) + RUN.finish(receipt, "refused_unpinned_runtime", reason="pin") + RUN.write_receipt(path, receipt) + self.assertEqual(json.loads(path.read_text())["exit_code"], 3) + self.assertEqual(list(path.parent.glob(".paper-receipt-*")), []) + + def test_protected_destinations(self): + for destination in (RUN.PLAN_PATH, DIRECTORY / "run.py", RUN.FROZEN_RUN, RUN.frozen().GATE_RECEIPT): + with self.assertRaises(ValueError): + RUN.write_receipt(destination, {}) + + def test_exit_codes(self): + for status, expected in (("passed", 0), ("failed", 1), ("incomplete", 1), ("not_connected", 2), + ("refused_plan", 3), ("cleanup_required", 3), ("not_started", 3)): + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + self.assertEqual(RUN.finish(receipt, status), expected) + + +class LeaseAndJournal(unittest.TestCase): + def args(self, root, run_id="offline-run"): + return RUN.parser().parse_args(["--receipt", str(root / "receipt.json"), + "--private-output-dir", str(root / "private"), "--run-id", run_id]) + + def test_held_second_file_description_refuses_before_connect(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + with mock.patch.dict(RUN.os.environ, {"TWS_ACCOUNT": FAKE_ACCOUNT, "XDG_STATE_HOME": directory}): + first = RUN.safety().AccountLease(FAKE_ACCOUNT) + try: + with mock.patch.object(RUN, "official_admission") as admission, mock.patch.object(RUN.subprocess, "Popen") as process: + receipt = RUN.run_trial(self.args(root), now=NOW, versions=PINNED) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("not_started", 3)) + self.assertEqual(receipt["failures"][-1]["reason"], "account_writer_already_running") + self.assertFalse(receipt["account_lease"]["acquired"]) + self.assertEqual(receipt["account_lease"], {"acquired": False}) + raw = json.dumps(receipt) + self.assertNotIn(first.lock_name, raw) + self.assertNotIn(first.lock_name.removesuffix(".lock"), raw) + admission.assert_not_called() + process.assert_not_called() + self.assertFalse((root / "private").exists()) + self.assertNotIn(FAKE_ACCOUNT, (root / "receipt.json").read_text()) + locks = root / "native-agent-stack/ibkr-paper/locks" + self.assertEqual(stat.S_IMODE(locks.stat().st_mode), 0o700) + self.assertEqual(stat.S_IMODE((locks / first.lock_name).stat().st_mode), 0o600) + finally: + first.close() + second = RUN.safety().AccountLease(FAKE_ACCOUNT) + second.close() + + def test_missing_account_refuses_without_connect(self): + with tempfile.TemporaryDirectory() as directory, mock.patch.dict(RUN.os.environ, {"TWS_ACCOUNT": "", "XDG_STATE_HOME": directory}), mock.patch.object(RUN, "official_admission") as admission: + receipt = RUN.run_trial(self.args(Path(directory)), now=NOW, versions=PINNED) + self.assertEqual(receipt["status"], "not_started") + self.assertEqual(receipt["failures"][-1]["reason"], "paper_account_required_for_lease") + admission.assert_not_called() + + def test_pre_admission_refusals_preserve_existing_receipt_bytes(self): + for condition in ("held_lease", "existing_journal", "missing_account", "changed_account", "failed_precheck", "quote_refusal"): + with self.subTest(condition=condition), tempfile.TemporaryDirectory() as directory, contextlib.ExitStack() as stack: + root = Path(directory) + args = self.args(root) + original = b'{"status":"passed","marker":"previous-run"}\n' + args.receipt.write_bytes(original) + account = "" if condition == "missing_account" else FAKE_ACCOUNT + stack.enter_context(mock.patch.dict(RUN.os.environ, {"TWS_ACCOUNT": account, "XDG_STATE_HOME": directory})) + if condition == "held_lease": + lease = RUN.safety().AccountLease(FAKE_ACCOUNT) + stack.callback(lease.close) + if condition == "existing_journal": + journal = RUN.safety().Journal(root / "private", "offline-run", ROOT, create=True) + journal.close() + pre = copy.deepcopy(FLAT) + pre.update(liquid_hours="20261005:0930-20261005:1600", time_zone_id="US/Eastern") + pre_account = "DU" + "9999999" if condition == "changed_account" else FAKE_ACCOUNT + if condition == "failed_precheck": + pre["status"] = "refused_existing_state" + quote = {"status": "refused_quote_loss_bound"} if condition == "quote_refusal" else admitted_quote() + admission = stack.enter_context(mock.patch.object(RUN, "official_admission", return_value=(pre, pre_account, quote))) + process = stack.enter_context(mock.patch.object(RUN.subprocess, "Popen")) + stack.enter_context(mock.patch.object(RUN.frozen(), "rth_check", return_value=(True, "synthetic"))) + receipt = RUN.run_trial(args, now=NOW, versions=PINNED) + self.assertNotEqual(receipt["status"], "passed") + self.assertEqual(args.receipt.read_bytes(), original) + process.assert_not_called() + if condition in ("held_lease", "existing_journal", "missing_account"): + admission.assert_not_called() + + def test_journal_denial_reason_scrubs_account_and_keeps_lock_key_private(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + journal = RUN.safety().Journal(root / "private", "reason-run", ROOT, create=True) + self.addCleanup(journal.close) + order = cache_order("O-19700101-000001-001-001-9", [ + event("OrderInitialized", typ="MARKET"), + event("OrderDenied", typ="MARKET", reason="account " + FAKE_ACCOUNT + "; echoed prefix" + FAKE_ACCOUNT + "suffix")], status="DENIED") + scope = RUN.RunOrderScope(900_000_000) + RUN.journal_order_events(SimpleNamespace(orders=lambda: [order]), scope, journal, FAKE_ACCOUNT) + records = journal.records() + self.assertEqual(records[-1]["kind"], "OrderDenied") + self.assertIn("reason", records[-1]) + self.assertNotIn(FAKE_ACCOUNT, records[-1]["reason"]) + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + receipt["account_lease"]["acquired"] = True + receipt["journal"] = journal.summary() + lock_name = RUN.safety().account_lock_name(FAKE_ACCOUNT) + for raw in (json.dumps(receipt), RUN.serialized_receipt(receipt), journal.path.read_text()): + for private in (FAKE_ACCOUNT, lock_name, lock_name.removesuffix(".lock")): + self.assertNotIn(private, raw) + self.assertEqual(receipt["account_lease"], {"acquired": True}) + + def test_existing_journal_refuses_before_admission_and_preserves_bytes(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + journal = RUN.safety().Journal(root / "private", "offline-run", ROOT, create=True, + hashes={"plan_sha256": "a" * 64, "harness_sha256": "b" * 64}) + path = journal.path + journal.close() + original = path.read_bytes() + with mock.patch.dict(RUN.os.environ, {"TWS_ACCOUNT": FAKE_ACCOUNT, "XDG_STATE_HOME": directory}), mock.patch.object(RUN, "official_admission") as admission: + receipt = RUN.run_trial(self.args(root), now=NOW, versions=PINNED) + self.assertEqual(receipt["status"], "not_started") + self.assertEqual(receipt["failures"][-1]["reason"], "journal_run_already_exists") + self.assertEqual(path.read_bytes(), original) + admission.assert_not_called() + + def test_fsynced_event_order_mapping_deduplication_and_hash_only_receipt(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + journal = RUN.safety().Journal(root / "private", "offline-run", ROOT, create=True, + hashes={"plan_sha256": receipt["plan_sha256"], "harness_sha256": receipt["harness_sha256"]}) + rows = [event("OrderInitialized", typ="MARKET"), event("OrderSubmitted", typ="MARKET"), + event("OrderAccepted", typ="MARKET"), event("OrderFilled", typ="MARKET")] + order = cache_order("O-19700101-000001-001-001-9", rows) + order.venue_order_id = "synthetic-venue" + cache = SimpleNamespace(orders=lambda: [order]) + scope = RUN.RunOrderScope(900_000_000) + synced_counts = [] + original_fsync = RUN.safety().os.fsync + + def fsync(fd): + original_fsync(fd) + synced_counts.append(len(journal.path.read_text().splitlines())) + + try: + with mock.patch.object(RUN.safety().os, "fsync", side_effect=fsync): + RUN.journal_order_events(cache, scope, journal, FAKE_ACCOUNT) + RUN.journal_order_events(cache, scope, journal, FAKE_ACCOUNT) + journal.append("stop_request", reason="C1_accepted_C3_filled") + journal.append("flat_proof", status="passed", positions=0, open_orders=0) + records = journal.records() + expected = ["run_start", "OrderInitialized", "OrderSubmitted", "OrderAccepted", "OrderFilled", "stop_request", "flat_proof"] + self.assertEqual([record["kind"] for record in records], expected) + self.assertEqual(synced_counts, [2, 3, 4, 5, 6, 7]) + self.assertIsNone(records[1]["venue_order_id"]) + self.assertEqual(records[3]["venue_order_id"], "synthetic-venue") + self.assertEqual(records[1]["client_order_id"], str(order.client_order_id)) + flags = RUN.safety().fcntl.fcntl(journal.stream.fileno(), RUN.safety().fcntl.F_GETFL) + self.assertTrue(flags & os.O_APPEND) + self.assertEqual(stat.S_IMODE(journal.path.stat().st_mode), 0o600) + self.assertEqual(stat.S_IMODE(journal.path.parent.stat().st_mode), 0o700) + receipt["journal"] = journal.summary() + self.assertEqual(set(receipt["journal"]), {"sha256", "record_count", "record_kinds"}) + self.assertEqual(receipt["journal"]["sha256"], RUN.sha256(journal.path)) + self.assertEqual(receipt["journal"]["record_count"], 7) + raw = RUN.serialized_receipt(receipt, (FAKE_ACCOUNT,)) + for private in (FAKE_ACCOUNT, str(order.client_order_id), "synthetic-venue", str(journal.path), "offline-run"): + self.assertNotIn(private, raw) + self.assertNotIn(FAKE_ACCOUNT, journal.path.read_text()) + finally: + journal.close() + + def test_private_output_rejects_repository_paths(self): + with self.assertRaises(RUN.safety().SafetyError): + RUN.safety().Journal(DIRECTORY, "offline-run", ROOT, create=True) + + def test_refusal_publication_preserves_a_concurrent_receipt(self): + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "receipt.json" + original = b'{"status":"passed","marker":"concurrent-run"}\n' + + def concurrent_writer(source, destination): + Path(destination).write_bytes(original) + raise FileExistsError("synthetic concurrent publication") + + with mock.patch.object(RUN.os, "link", side_effect=concurrent_writer): + result = RUN.write_receipt(path, RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED), replace=False) + self.assertFalse(result) + self.assertEqual(path.read_bytes(), original) + self.assertEqual(list(path.parent.glob(".paper-receipt-*")), []) + + +class AdmissionRefusals(unittest.TestCase): + def args(self, path, *extra): + return RUN.parser().parse_args(["--receipt", str(path), *extra]) + + def test_luld_closing_overlap_refuses_before_any_broker_request(self): + for minute in (29, 35, 43): + with self.subTest(minute=minute), tempfile.TemporaryDirectory() as directory: + now = NOW.astimezone(NY).replace(hour=15, minute=minute, second=0) + # These times satisfy the old whole-run RTH rule. + self.assertTrue(RUN.frozen().rth_check(now, PLAN)[0]) + with mock.patch.object(RUN, "official_admission") as admission, mock.patch.object(RUN, "official_check") as proof, mock.patch.object(RUN, "build_node") as node, mock.patch.object(RUN.subprocess, "Popen") as process, mock.patch.object(RUN.safety(), "AccountLease") as lease: + receipt = RUN.run_trial(self.args(Path(directory) / "receipt.json"), now=now, versions=PINNED) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("refused_luld_closing_period", 3)) + self.assertIsNone(receipt["quote_admission"]) + closing = receipt["closing_period_admission"] + self.assertEqual(closing["stage"], "luld_tier1_closing_period_band") + self.assertEqual(closing["reason"], "planned_order_window_overlaps_luld_closing_period") + self.assertEqual(closing["order_horizon_seconds"], 360) + self.assertEqual(closing["closing_band_fraction"], "0.10") + self.assertIsNone(receipt["pre_check"]) + for operation in (admission, proof, node, process, lease): + operation.assert_not_called() + + def test_luld_order_window_boundary_in_new_york_and_utc(self): + start = NOW.astimezone(NY).replace(hour=15, minute=28, second=59) + admitted = RUN.closing_period_admission(PLAN, start.astimezone(timezone.utc)) + self.assertEqual(admitted["status"], "passed") + self.assertTrue(admitted["planned_order_window_end"].startswith("2026-10-05T15:34:59")) + refused = RUN.closing_period_admission(PLAN, start.replace(minute=29, second=0)) + self.assertEqual(refused["status"], "refused_luld_closing_period") + self.assertTrue(refused["planned_order_window_end"].startswith("2026-10-05T15:35:00")) + + def test_luld_cutoff_uses_the_same_liquid_session_as_the_window(self): + for end_hour in (16, 13): + with self.subTest(end_hour=end_hour): + start = NOW.astimezone(NY).replace(hour=end_hour - 1, minute=28, second=59) + sessions = RUN.frozen().parse_liquid_hours( + f"20261005:0930-20261005:{end_hour:02}00", "US/Eastern", start.date()) + admitted = RUN.closing_period_admission(PLAN, start.astimezone(timezone.utc), liquid_sessions=sessions) + self.assertEqual(admitted["status"], "passed") + self.assertEqual(admitted["session_source"], "broker_liquid_hours") + self.assertEqual(admitted["closing_period_start"], start.replace(minute=35, second=0).isoformat()) + for minute in (29, 35, 43): + later = start.replace(minute=minute, second=0) + self.assertTrue(RUN.frozen().rth_check(later, PLAN, sessions)[0]) + refused = RUN.closing_period_admission(PLAN, later, liquid_sessions=sessions) + self.assertEqual(refused["status"], "refused_luld_closing_period") + + def test_early_close_refuses_before_quote_or_node(self): + for minute in (29, 35, 43): + with self.subTest(minute=minute), tempfile.TemporaryDirectory() as directory: + now = NOW.astimezone(NY).replace(hour=12, minute=minute, second=0) + pre = copy.deepcopy(FLAT) + pre.update(liquid_hours="20261005:0930-20261005:1300", time_zone_id="US/Eastern") + client = RUN.AdmissionState() + client.disconnect = mock.Mock() + + class FixedDateTime(datetime): + @classmethod + def now(cls, tz=None): + return now.astimezone(tz) if tz is not None else now.replace(tzinfo=None) + + with mock.patch.object(RUN, "datetime", FixedDateTime), mock.patch.object(RUN.frozen(), "run_check", return_value=(pre, FAKE_ACCOUNT)), mock.patch.object(RUN, "build_admission_client", return_value=client), mock.patch.object(RUN, "quote_check") as quote, mock.patch.object(RUN, "official_check") as proof, mock.patch.object(RUN, "build_node") as node, mock.patch.object(RUN.subprocess, "Popen") as process, mock.patch.dict(RUN.os.environ, {"TWS_ACCOUNT": FAKE_ACCOUNT, "XDG_STATE_HOME": directory}): + receipt = RUN.run_trial(self.args(Path(directory) / "receipt.json"), now=now, versions=PINNED) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("refused_luld_closing_period", 3)) + self.assertEqual(receipt["pre_check"]["status"], "passed") + self.assertEqual(receipt["quote_admission"]["status"], "not_run") + closing = receipt["closing_period_admission"] + self.assertTrue(closing["closing_period_start"].startswith("2026-10-05T12:35:00")) + self.assertTrue(closing["liquid_session_end"].startswith("2026-10-05T13:00:00")) + self.assertEqual(closing["session_source"], "broker_liquid_hours") + for operation in (quote, proof, node, process): + operation.assert_not_called() + client.disconnect.assert_called_once() + + def test_pins_window_ports_and_ids_refuse_without_connecting(self): + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "receipt.json" + tests = [((), {"nautilus_trader": "1.231.0", "ibapi": "10.45.1"}, NOW, "refused_unpinned_runtime"), + ((), PINNED, NOW.replace(hour=1), "refused_outside_window"), + (("--port", "4001"), PINNED, NOW, "refused_plan"), + (("--node-client-id", "101"), PINNED, NOW, "refused_plan"), + (("--check-client-id", "91"), PINNED, NOW, "refused_plan")] + for extra, versions, now, expected in tests: + with self.subTest(expected=expected, extra=extra), mock.patch.object(RUN, "official_admission") as check, mock.patch.object(RUN, "build_node") as node: + receipt = RUN.run_trial(self.args(path, *extra), now=now, versions=versions) + self.assertEqual(receipt["status"], expected) + self.assertEqual(receipt["exit_code"], 3) + check.assert_not_called() + node.assert_not_called() + + def test_runner_admission_replaces_unenforced_cap_refusal(self): + with tempfile.TemporaryDirectory() as directory, mock.patch.object(RUN, "official_admission") as check, mock.patch.object(RUN, "build_node") as node, mock.patch.object(RUN.subprocess, "Popen") as process: + receipt = RUN.run_trial(self.args(Path(directory) / "receipt.json", "--plan-only"), now=NOW, versions=PINNED) + self.assertEqual(receipt["status"], "passed") + self.assertEqual(receipt["risk"]["engine_route"]["issue_state"], "closed") + self.assertFalse(receipt["risk"]["engine_route"]["pin_fix_present"]) + self.assertEqual(receipt["risk"]["note"], RUN.RISK_NOTE) + check.assert_not_called() + node.assert_not_called() + process.assert_not_called() + + def test_plan_only_has_no_socket_path(self): + with tempfile.TemporaryDirectory() as directory, mock.patch.object(RUN, "official_check") as check, mock.patch.object(RUN, "official_admission") as quote: + receipt = RUN.run_trial(self.args(Path(directory) / "receipt.json", "--plan-only"), now=NOW, versions=PINNED) + self.assertEqual(receipt["exit_code"], 0) + check.assert_not_called() + quote.assert_not_called() + + def test_invalid_json_refused(self): + with tempfile.TemporaryDirectory() as directory: + plan = Path(directory) / "bad.json" + plan.write_text("{broken") + args = self.args(Path(directory) / "receipt.json", "--plan", str(plan)) + receipt = RUN.run_trial(args, now=NOW, versions=PINNED) + self.assertEqual(receipt["status"], "refused_plan") + + +class ChildRefusals(unittest.TestCase): + def test_corrupt_or_mismatched_journal_refuses_before_node_build(self): + for content in ("{broken\n", "7\n", '{"kind": "run_start", "run_id": "different"}\n'): + with self.subTest(content=content), tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "receipt.json" + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + payload = node_payload(receipt) + journal_path = Path(directory) / "offline-run.jsonl" + journal_path.write_text(content) + payload.update(run_id="offline-run", journal_path=str(journal_path)) + with mock.patch.object(RUN, "build_node") as node: + result = asyncio.run(RUN.node_phase(PLAN, 4002, path, payload)) + node.assert_not_called() + self.assertEqual(result["status"], "refused_child_journal") + self.assertEqual(json.loads(path.read_text())["node"]["pre_node_refusal"], "refused_child_journal") + self.assertNotIn(FAKE_ACCOUNT, path.read_text()) + + def test_pre_node_refusals_are_written_and_preserved_by_parent_mapping(self): + for condition, expected in (("plan", "refused_child_plan"), ("runtime", "refused_child_runtime"), + ("window", "refused_child_window"), ("quote", "refused_child_quote_admission"), + ("payload", "refused_child_payload"), ("nonce", "refused_child_receipt_nonce")): + with self.subTest(condition=condition), tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "receipt.json" + receipt = RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED) + RUN.write_receipt(path, receipt) + payload = node_payload(receipt) + plan = copy.deepcopy(PLAN) + if condition == "plan": + plan["host"] = "not_loopback" + if condition == "quote": + payload["quote_admission"]["admitted_monotonic"] -= 11 + if condition == "payload": + del payload["node_stop_at"] + if condition == "nonce": + payload["run_nonce"] = "0" * 32 + versions = {} if condition == "runtime" else PINNED + args = RUN.parser().parse_args(["--_node-phase", "--port", "4002", "--receipt", str(path)]) + with mock.patch.object(RUN.sys, "stdin", io.StringIO(json.dumps(payload))), mock.patch.object(RUN, "load_plan", return_value=plan), mock.patch.object(RUN, "runtime_versions", return_value=versions), mock.patch.object(RUN.frozen(), "rth_check", return_value=(condition != "window", "synthetic")), mock.patch.object(RUN, "build_node") as node: + self.assertEqual(RUN.run_child(args), 3) + node.assert_not_called() + saved = json.loads(path.read_text()) + self.assertEqual(saved["status"], expected) + self.assertEqual(saved["node"]["pre_node_refusal"], expected) + self.assertEqual(saved["failures"][-1]["cases_blocked"], list(RUN.CASE_IDS)) + self.assertNotIn(FAKE_ACCOUNT, path.read_text()) + saved["flat_proof"] = copy.deepcopy(FLAT) + self.assertEqual(RUN.final_status(saved, child_returncode=3)[0], expected) + + def test_bad_stdin_payload_preserves_provisional_receipt(self): + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "receipt.json" + RUN.write_receipt(path, RUN.new_receipt(PLAN, DIRECTORY / "plan.json", PINNED)) + args = RUN.parser().parse_args(["--_node-phase", "--receipt", str(path)]) + with mock.patch.object(RUN.sys, "stdin", io.StringIO("{broken")), mock.patch.object(RUN, "build_node") as node: + self.assertEqual(RUN.run_child(args), 3) + node.assert_not_called() + self.assertEqual(json.loads(path.read_text())["status"], "refused_child_payload") + + +class SyntheticOrchestration(unittest.TestCase): + """Official checks, quote admission and node execution are synthetic.""" + def drive(self, proof=None, proof_account=FAKE_ACCOUNT, pre_status="passed", events=None, quote=None, proof_hook=None, pre_account=FAKE_ACCOUNT, fail_final_write=False, foreign_receipt=False, malformed_receipt=False): + directory = tempfile.TemporaryDirectory() + self.addCleanup(directory.cleanup) + path = Path(directory.name) / "receipt.json" + args = RUN.parser().parse_args(["--receipt", str(path)]) + pre = copy.deepcopy(FLAT) + pre.update(status=pre_status, liquid_hours="20261005:0930-20261005:1600", time_zone_id="US/Eastern") + proof = copy.deepcopy(FLAT if proof is None else proof) + test = self + own_nonce = [] + + class FixedDateTime(datetime): + @classmethod + def now(cls, tz=None): + return NOW.astimezone(tz) if tz is not None else NOW.replace(tzinfo=None) + + class Pipe(io.StringIO): + def close(pipe): + payload = json.loads(pipe.getvalue()) + test.assertEqual(payload["account_id"], FAKE_ACCOUNT) + receipt = payload["receipt"] + if not foreign_receipt: + test.assertEqual(payload["run_nonce"], receipt["run_nonce"]) + test.assertRegex(payload["run_nonce"], r"^[0-9a-f]{32}$") + own_nonce.append(receipt.get("run_nonce")) + receipt["node"].update(started=True, stop_ns=STOP_NS) + test.assertTrue(RUN.validate_child_admission(PLAN, payload)) + RUN.update_observations(receipt, sequence() if events is None else events, PLAN) + if foreign_receipt: + # Another account's independently initialized parent uses + # the same receipt path and writes otherwise passing evidence. + receipt["run_nonce"] = "0" * 32 + RUN.write_receipt(path, receipt, (FAKE_ACCOUNT,)) + if malformed_receipt: + path.write_text(json.dumps(["synthetic", "non-dict receipt"])) + super().close() + + class Process: + returncode = None + + def __init__(self, command, **kwargs): + # The provisional state exists before process construction. + provisional = json.loads(path.read_text()) + test.assertEqual(provisional["status"], "cleanup_required") + test.assertNotIn(FAKE_ACCOUNT, path.read_text()) + test.assertNotIn(FAKE_ACCOUNT, str(command)) + test.assertEqual(kwargs["stdout"], RUN.subprocess.DEVNULL) + test.assertEqual(kwargs["stderr"], RUN.subprocess.DEVNULL) + test.assertEqual(len(kwargs["pass_fds"]), 1) + lease_fd = kwargs["pass_fds"][0] + lock = Path(directory.name) / "native-agent-stack/ibkr-paper/locks" / RUN.safety().account_lock_name(FAKE_ACCOUNT) + test.assertEqual(os.fstat(lease_fd).st_ino, lock.stat().st_ino) + with test.assertRaises(RUN.safety().SafetyError): + RUN.safety().AccountLease(FAKE_ACCOUNT) + for key in ("RUST_LOG", "NAUTILUS_LOG", "TWS_ACCOUNT"): + test.assertNotIn(key, kwargs["env"]) + self.stdin = Pipe() + + def wait(self, timeout=None): + self.returncode = 0 + + def poll(self): + return self.returncode + + original_write = RUN.write_receipt + injected = [] + + def write(path, value, *args, **kwargs): + if fail_final_write and value["flat_proof"] is not None and not injected: + injected.append(True) + raise OSError("synthetic final receipt storage failure") + return original_write(path, value, *args, **kwargs) + + with mock.patch.object(RUN, "datetime", FixedDateTime), mock.patch.object(RUN, "official_admission", return_value=(pre, pre_account, admitted_quote() if quote is None else quote)), mock.patch.object(RUN, "official_check", return_value=(proof, proof_account), side_effect=proof_hook) as checker, mock.patch.object(RUN.subprocess, "Popen", Process), mock.patch.object(RUN, "write_receipt", side_effect=write), mock.patch.dict(RUN.os.environ, {"RUST_LOG": "trace", "NAUTILUS_LOG": "trace", "TWS_ACCOUNT": FAKE_ACCOUNT, "XDG_STATE_HOME": directory.name}): + receipt = RUN.run_trial(args, now=NOW, versions=PINNED) + if fail_final_write: + self.assertEqual(injected, [True]) + self.assertNotIn(FAKE_ACCOUNT, path.read_text()) + if foreign_receipt: + self.assertEqual(receipt.get("run_nonce"), own_nonce[0]) + self.assertEqual(receipt["account_lease"], {"acquired": True}) + lock_name = RUN.safety().account_lock_name(FAKE_ACCOUNT) + for raw in (json.dumps(receipt), path.read_text()): + self.assertNotIn(lock_name, raw) + self.assertNotIn(lock_name.removesuffix(".lock"), raw) + return receipt, checker + + def test_provisional_receipt_and_final_flat_proof(self): + receipt, checker = self.drive() + self.assertEqual(checker.call_count, 1) + self.assertFalse(checker.call_args_list[0].kwargs["with_session"]) + self.assertEqual(receipt["status"], "passed") + + def test_foreign_run_receipt_is_not_adopted_and_flat_proof_still_runs(self): + receipt, checker = self.drive(foreign_receipt=True) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("incomplete", 1)) + self.assertFalse(receipt["node"]["started"]) + self.assertTrue(receipt["node"]["child_launched"]) + self.assertEqual(receipt["events"], []) + self.assertEqual(receipt["fills"], []) + self.assertIsNone(receipt["roundtrip"]) + self.assertEqual([case["outcome"] for case in receipt["cases"].values()], ["not_run"] * 4) + self.assertTrue(any(f["reason"] == "child_receipt_nonce_mismatch" for f in receipt["failures"])) + self.assertEqual(receipt["flat_proof"]["status"], "passed") + self.assertEqual(checker.call_count, 1) + + def test_list_receipt_keeps_provisional_evidence_and_runs_flat_proof(self): + receipt, checker = self.drive(malformed_receipt=True) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("incomplete", 1)) + self.assertFalse(receipt["node"]["started"]) + self.assertTrue(receipt["node"]["child_launched"]) + self.assertEqual(receipt["events"], []) + self.assertEqual(receipt["fills"], []) + self.assertIsNone(receipt["roundtrip"]) + self.assertEqual([case["outcome"] for case in receipt["cases"].values()], ["not_run"] * 4) + self.assertTrue(any(f["reason"] == "child_receipt_unreadable" for f in receipt["failures"])) + self.assertEqual(receipt["flat_proof"]["status"], "passed") + self.assertEqual(checker.call_count, 1) + + def test_final_write_failure_retains_child_evidence_and_never_not_started(self): + receipt, checker = self.drive(fail_final_write=True) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("incomplete", 1)) + self.assertTrue(receipt["node"]["child_launched"]) + self.assertTrue(receipt["node"]["started"]) + self.assertEqual(len(receipt["fills"]), 2) + self.assertTrue(receipt["roundtrip"]["closed"]) + self.assertEqual(receipt["flat_proof"]["status"], "passed") + self.assertEqual(checker.call_count, 1) + self.assertTrue(any(f["reason"] == "final_receipt_write_error" for f in receipt["failures"])) + + def test_failure_after_child_launch_preserves_latest_evidence(self): + def proof(*args, **kwargs): + raise OSError("synthetic proof storage error") + + receipt, checker = self.drive(proof_hook=proof) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("cleanup_required", 3)) + self.assertTrue(receipt["node"]["child_launched"]) + self.assertTrue(receipt["node"]["started"]) + self.assertEqual(len(receipt["fills"]), 2) + self.assertEqual(checker.call_count, 1) + + def test_final_write_error_does_not_downgrade_an_observed_bound_breach(self): + events = sequence() + events[2]["fill_price"] = "1001" + receipt, checker = self.drive(events=events, fail_final_write=True) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("failed", 1)) + self.assertTrue(receipt["node"]["child_launched"]) + self.assertEqual(checker.call_count, 1) + self.assertTrue(any(f["reason"].startswith("observed_") for f in receipt["failures"])) + self.assertTrue(any(f["reason"] == "final_receipt_write_error" for f in receipt["failures"])) + + def test_unproven_flat_is_cleanup_required(self): + receipt, _ = self.drive({"status": "refused_existing_state", "observed": {"positions": 1, "open_orders": 0}}, proof_account=None) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("cleanup_required", 3)) + self.assertEqual(receipt["failures"][-1]["cases_blocked"], ["C4"]) + self.assertEqual(receipt["flat_proof"]["status"], "refused_existing_state") + + def test_disconnected_proof_preserves_its_cause(self): + receipt, _ = self.drive({"status": "not_connected", "observed": {}}, proof_account=None) + self.assertEqual(receipt["status"], "cleanup_required") + self.assertEqual(receipt["flat_proof"]["status"], "not_connected") + + def test_observed_bounds_map_to_failed(self): + for changes in ({"quantity": "2"}, {"fill_price": "1001"}, {"instrument_id": "AAPL=STK.SMART"}): + events = sequence() + events[2].update(changes) + receipt, _ = self.drive(events=events) + self.assertEqual((receipt["status"], receipt["exit_code"]), ("failed", 1)) + events = sequence() + events.extend(event("OrderInitialized", order=f"EXTRA{number}") for number in range(4)) + receipt, _ = self.drive(events=events) + self.assertEqual(receipt["status"], "failed") + + def test_loss_breach_is_failed_and_unresolved_fees_are_incomplete(self): + events = sequence() + events[-1]["fill_price"] = "760" + self.assertEqual(self.drive(events=events)[0]["status"], "failed") + events[-1]["fill_price"] = "770" + events[-1]["commission"] = None + self.assertEqual(self.drive(events=events)[0]["status"], "incomplete") + + def test_quote_refusal_blocks_node_and_cases(self): + for status in ("refused_quote_notional", "refused_stale_quote", "refused_delayed_quote", "refused_quote_loss_bound"): + receipt, checker = self.drive(quote={"status": status}) + self.assertEqual((receipt["status"], receipt["exit_code"]), (status, 3)) + self.assertEqual(checker.call_count, 0) + self.assertFalse(receipt["node"]["started"]) + self.assertEqual(receipt["failures"][-1]["cases_blocked"], list(RUN.CASE_IDS)) + + def test_writer_lease_is_held_through_proof_and_then_released(self): + states = [] + + def proof(*args, **kwargs): + states.append(RUN.os.environ["XDG_STATE_HOME"]) + with self.assertRaises(RUN.safety().SafetyError): + RUN.safety().AccountLease(FAKE_ACCOUNT) + return copy.deepcopy(FLAT), FAKE_ACCOUNT + + receipt, checker = self.drive(proof_hook=proof) + self.assertTrue(receipt["account_lease"]["acquired"]) + self.assertEqual(checker.call_count, 1) + with mock.patch.dict(RUN.os.environ, {"XDG_STATE_HOME": states[0]}): + subsequent = RUN.safety().AccountLease(FAKE_ACCOUNT) + subsequent.close() + + def test_precheck_must_match_the_locally_leased_account(self): + receipt, checker = self.drive(pre_account="DU" + "9999999") + self.assertEqual(receipt["status"], "refused_changed_account") + self.assertFalse(receipt["node"]["started"]) + self.assertEqual(checker.call_count, 0) + + def test_changed_account_is_cleanup_required(self): + receipt, _ = self.drive(proof_account="DU" + "9999999") + self.assertEqual(receipt["status"], "cleanup_required") + + def test_precheck_refusal_does_not_construct_node(self): + receipt, checker = self.drive(pre_status="refused_existing_state") + self.assertEqual(checker.call_count, 0) + self.assertEqual(receipt["exit_code"], 3) + + +@unittest.skipUnless(RUN.runtime_versions()["nautilus_trader"] == "2.0.0rc5", "requires installed NautilusTrader 2.0.0rc5; no broker") +class InstalledRc5Surface(unittest.TestCase): + def test_reconstructed_market_init_explains_external_receipt_flag(self): + # engine/mod.rs and live execution manager create True. The pinned + # MarketOrder TryFrom calls new_checked, which reconstructs False. + # Python from_dict exercises the same constructor, without any engine run. + from nautilus_trader.core import UUID4 + from nautilus_trader.model import ClientOrderId, InstrumentId, MarketOrder, OrderInitialized, OrderSide, Quantity, StrategyId, TimeInForce, TraderId + order = MarketOrder(trader_id=TraderId.from_str("TESTER-001"), strategy_id=StrategyId.from_str("EXEC_TESTER-001"), + instrument_id=InstrumentId.from_str("SPY=STK.SMART"), client_order_id=ClientOrderId.from_str("O-19700101-000001-001-001-1"), + order_side=OrderSide.BUY, quantity=Quantity.from_str("1"), init_id=UUID4(), ts_init=3_000_000_000, + time_in_force=TimeInForce.IOC, reduce_only=False, quote_quantity=False) + materialization = order.init_event.to_dict() + materialization["reconciliation"] = True + initialized = OrderInitialized.from_dict(materialization) + self.assertTrue(initialized.reconciliation) + state = order.to_dict() + state["reconciliation"] = initialized.reconciliation + reconstructed = MarketOrder.from_dict(state) + self.assertFalse(reconstructed.events()[0].reconciliation) + scope = RUN.RunOrderScope(2_100_000_000) + cache = SimpleNamespace(orders=lambda: [reconstructed]) + self.assertEqual(RUN.tester_orders(cache, scope), []) + self.assertEqual(len(RUN.snapshot_events(cache, scope, external=True)), 1) + + def test_real_builder_and_shared_cache_handle_without_running(self): + # Upstream builder.rs build_in_place constructs clients; connect is in + # the run/start path. This test never calls run, run_async or start. + node = RUN.build_node(PLAN, 4002, FAKE_ACCOUNT, tob_offset_ticks=38500) + try: + cache = node.cache + self.assertEqual(cache.orders(), []) + self.assertFalse(node.handle().is_running) + finally: + node.dispose() + + def test_real_configs_without_node_or_connection(self): + _, data, execution, risk, tester = RUN.native_configs(PLAN, 4002, FAKE_ACCOUNT, tob_offset_ticks=38500) + self.assertEqual((data.client_id, execution.client_id), (91, 91)) + self.assertEqual(execution.account_id, FAKE_ACCOUNT) + self.assertFalse(risk.bypass) + self.assertEqual(risk.max_notional_per_order, {"SPY=STK.SMART": "1000"}) + self.assertEqual(risk.max_order_submit_rate, "6/00:07:00") + self.assertEqual(str(tester.order_qty), "1") + self.assertEqual(str(tester.open_position_on_start_qty), "1") + self.assertTrue(tester.cancel_orders_on_stop and tester.close_positions_on_stop) + self.assertFalse(tester.enable_limit_sells or tester.enable_stop_buys or tester.enable_stop_sells or tester.enable_brackets) + self.assertFalse(tester.modify_orders_to_maintain_tob_offset or tester.cancel_replace_orders_to_maintain_tob_offset) + self.assertFalse(tester.use_post_only) + self.assertTrue(tester.use_individual_cancels_on_stop) + self.assertEqual(tester.tob_offset_ticks, 38500) + from nautilus_trader.common import LoggerConfig, LogLevel + logger = LoggerConfig(stdout_level=LogLevel.OFF, fileout_level=LogLevel.OFF, print_config=False) + self.assertEqual(logger.stdout_level, LogLevel.OFF) + + def test_real_market_order_cache_projection(self): + from nautilus_trader.core import UUID4 + from nautilus_trader.model import AccountId, ClientOrderId, InstrumentId, MarketOrder, OrderSide, OrderSubmitted, Quantity, StrategyId, TimeInForce, TraderId + order = MarketOrder(trader_id=TraderId.from_str("TESTER-001"), strategy_id=StrategyId.from_str("EXEC_TESTER-001"), + instrument_id=InstrumentId.from_str("SPY=STK.SMART"), client_order_id=ClientOrderId.from_str("O-19700101-000001-001-001-1"), + order_side=OrderSide.BUY, quantity=Quantity.from_str("1"), init_id=UUID4(), + ts_init=1_000_000_000, time_in_force=TimeInForce.IOC, reduce_only=False, quote_quantity=False) + cache = SimpleNamespace(orders=lambda **kwargs: [order]) + scope = RUN.RunOrderScope(900_000_000) + self.assertEqual(len(RUN.snapshot_events(cache, scope)), 1) + self.assertEqual(RUN.tester_orders(cache, scope), [order]) + order.apply(OrderSubmitted(trader_id=order.trader_id, strategy_id=order.strategy_id, + instrument_id=order.instrument_id, client_order_id=order.client_order_id, + account_id=AccountId.from_str("IB-" + FAKE_ACCOUNT), event_id=UUID4(), + ts_event=1_100_000_000, ts_init=1_100_000_000)) + projected = RUN.snapshot_events(cache, scope) + self.assertEqual(len(projected), 2) + self.assertEqual(projected[0]["type"], "OrderInitialized") + self.assertIsNone(projected[0]["price"]) + self.assertEqual(projected[0]["order_type"], "MARKET") + self.assertTrue(RUN.orders_in_flight([order])) + self.assertNotIn(str(order.client_order_id), json.dumps(projected)) + + +if __name__ == "__main__": + unittest.main()