Overnight volume watch: data-only BOATS significant-volume monitor over the full overnight-tradable universe by seathatflowsinourveins · Pull Request #480 · native-agent-stack/native-agent-stack

seathatflowsinourveins · 2026-09-29T02:11:08Z

Summary

This is the user's 2026-09-28 request (about 21:05 ET): "continue e2e in after market and monitor significant volume". The after-hours session had closed at 20:00, so the watch runs on Alpaca's overnight session, 20:00-04:00 ET on the BOATS feed. It hands over to the 04:00 pre-market session.

Data only. No order, cancel or position call, and no trading-API write. The only POST goes to the loopback ntfy.

blueprints/us-equities/overnight-volume/thresholds-v1.json holds the monitoring thresholds, not a trading rule. It was committed before the first sweep: originally 61ab9d32 at 21:39 ET, rebased to a1ae9008 with identical bytes (sha256 prefix below), before the first sweep at 21:55 ET. Any change is a new id.
watch.py reuses incentive-monitor/monitor.py's neutral helpers: credentials reader, HTTP counters, budget, retry, rate-limit floor. It has plan, once (dry run) and run modes and writes to a private 0700 run directory.
"Significant" combines three measures: overnight dollar volume from the BOATS dailyBar.v, the overnight share of SIP 20-day ADV, and the change against the prior regular SIP close. Thresholds were calibrated on the night of 09-27 to 09-28: 1,637 of 9,683 names traded; p90 dollar volume $1.83M; p90 ADV share 2.1%.
Universe: every asset whose attributes contain overnight_tradable and not overnight_halted, 9,683 names. The sweep uses 20 snapshot calls every 120 s, about 0.1% of the 10,000/min data limit.

Live run (2026-09-29, trading lane: ecosystem-roadmap-2026)

Unit overnight-volume-watch-20260929: systemd-run --user, nice 10, CPUWeight 20, MemoryMax 4G, RuntimeMaxSec to 04:05 ET. It is the external backstop the README names.
Credentials through the pointer --env-file "$PAPER_ENV_FILE" via bash -ic.
First sweep: 796 names with overnight volume, 17 significant, 5 notified. A sanitized receipt with totals follows after the 04:00 stop.

SOTA sources

Alpaca overnight trading, "8:00 PM ET Sunday to 4:00 AM ET Friday", Blue Ocean ATS; feed=boats for Algo Trader Plus: https://docs.alpaca.markets/docs/24-5-trading (documentation pages fetched 2026-09-29, updated 2026-07-07).
Snapshot feed enum (boats, overnight): https://docs.alpaca.markets/reference/stocksnapshots-1
Real-time stock streams (v1beta1/boats): https://docs.alpaca.markets/docs/real-time-stock-pricing-data
Daily-bar rules (extended-hours T trades do not update daily OHLC; odd lots add volume only): https://docs.alpaca.markets/docs/market-data-faq
Reused in-repo code: blueprints/us-equities/incentive-monitor/monitor.py (helpers at the lines cited in the README); the notifier pattern from scripts/host_requests.py send_notice and check_notify_url.

Checks

tests.test_overnight_volume: Ran 34, OK. Failing-first: ModuleNotFoundError on base, independently re-proved on a git archive of the base.
validate.py passed; the three registry tests OK.
A verifier grep of watch.py and monitor.py found no path for orders, positions, DELETE or PATCH.
An independent stack-verifier dry-run sweep: 9,683 symbols, 119 calls, 0 failed chunks.

Not verified

Whether paper accounts accept overnight orders. The documentation covers limit orders with extended_hours=true; no order was placed.
The BOATS dailyBar is session-keyed. The watch logs session_bar_rolled if SPY's changes at ET midnight.

🤖 Generated with Claude Code

…itted before the first sweep Monitoring thresholds only (not a trading rule): BOATS overnight session volume against SIP ADV20 and a 20-session BOATS baseline, significant and notify tiers, two top-N lists and the local ntfy topic. Any change is a new id. No order, cancel or position call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…(dry run) and run modes watch.py sweeps BOATS snapshots for every overnight_tradable, not overnight_halted US equity every 120 s until 04:00 ET and records names whose session volume is significant against SIP ADV20, a 20-session BOATS baseline or the prior regular close (thresholds-v1.json). - reuses monitor.py's credentials, Http, Budget, retry, constants and Sink (fixed-directory subclass); DailyBars follows AdvLoader - GET allow-list (paper /v2/assets, data snapshots and bars), refused before any request; per-source budget, plan refused above 5% of the data limit; rate-limit floor honoured only on fresh readings - session bar by the UTC date of dailyBar.t; prevDailyBar never used - alerts once per symbol and tier; notify tier pushed to the loopback ntfy topic (mirror of host_requests' URL rule and POST), capped - once is a dry run: never posts, dry-run alerts never restored - tests: synthetic fixtures, no network (34 tests) No order, cancel or position call. monitor.py, board_scan.py and broad-universe/scan.py are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- --no-push also holds back queued alerts restored from an earlier run: push() returns before due() while pushing is disabled - a failed or rate-limited probe (an error, a 429 or fewer than RATELIMIT_FLOOR calls left) stops the sweep's fan-out; GuardedHttp keeps an error answer's rate-limit headers; every fan-out request rechecks the floor first, as monitor.poll_option_oi does - plan_calls counts the busiest rolling minute: both daily-bar passes, then the first sweep (118 calls at 9,683 names; 45,400 names is refused at 545). DataBudget caps all data-host calls at the plan's 500 in any rolling minute, on the watch's clock - start-up checks a stop request and its 600 s deadline before every request and retry; its waits are interruptible and never pass the deadline (stop reason startup_deadline, exit 1) - each push attempt is fsynced as a push record before the POST, so restore() puts it into the dedup set and session cap; the outcome is a push_result record - an --out inside a git worktree (a .git entry at or above it, symlinks resolved, as data-lake/backfill.py) is refused before any file or request - README: every Alpaca request is a GET; the busiest-minute numbers; stop latency (a read in flight can take up to its 20 s timeout) - tests: 16 new and 3 updated (50); each new test fails on 6545d7a or kills one of the five named mutants, which survive the old suite thresholds-v1.json is unchanged. No order, cancel or position call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

seathatflowsinourveins · 2026-09-29T03:18:34Z

Repair round for the GPT-6 review (gpt-6-astra, effort max, read-only, head 6545d7ad, changes-needed with 3 must-fix, 4 should-fix and 1 nit). All of it is fixed in 04ab3308:

--no-push held after restart. Watch.push returns before Alerts.due() when pushes are off.
A failed or 429 probe stops the fan-out. Error-response rate headers are kept, and capacity is rechecked before every fan-out request.
Busiest-minute plan. The formula is 2*passes + chunks*ceil(60/sweep): 118 calls at 9,683 symbols, and 45,400 symbols is refused at 545. A new DataBudget caps all data-host calls together in a rolling minute.
Mutants. The review's five surviving mutants are each killed by new tests (runtime budget, sweep floor, notifier redirects, notifier proxies, SIP split adjustment). Ten further mutants of the repair are also killed.
Cancellation and deadline during startup. A Stopped checkpoint runs between requests, waits are interruptible, and the deadline starts before the asset load. The README's stop-latency claim is corrected.
Push attempt recorded (fsync) before the POST. A crash can no longer repeat a notification.
--out inside a git worktree is refused before any file or request.
README: "Every Alpaca request is a GET".

tests.test_overnight_volume: 50 OK, with 16 new and 3 updated tests, each failing on 6545d7a (independent stack-verifier: pass). validate.py passes, the registry tests pass, and thresholds-v1.json is byte-identical (8287b858…).

The live unit overnight-volume-watch-20260929 keeps running the 6545d7a code until 04:05 ET, and its receipt will say so. Under the bounded-review rule there is one review and one repair, with no second review round.

…ps, 75 alerts, 0 orders) Receipt for the BOATS overnight watch of trade date 2026-09-29 (22:09:58-04:00:04 ET, stop_time, account 1 data only): code identity from start.json (dca569ae, not on main; its watch.py is the same blob as 6545d7a in refs/pull/480/head history, and main's #480 squash differs after the review repair round), thresholds and plan, universe and reference coverage, 176 sweeps at 120 s with all 3,520 chunks ok, 3,619 requests with 0 failed (429: 0), 75 alerts (62 significant, 13 notify) with 6 posted to the loopback ntfy topic, the SPY session-bar roll probe, the no-order basis, and sha256/bytes/records of every private file. The data stays private; no alerted symbol is named. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…onitor and overnight watch (mechanics-only) (#564) * Incentive monitor 2026-09-29: sanitized run receipt (data only, 2,874 sweeps, 0 orders) Receipt for the incentive-monitor-20260929 unit (03:55-20:00 ET, account 2 key pair, data and asset-list reads only): code identity from the b528bb5 clone (monitor.py sha256 asserted across the clone, git show, the frozen SUMS and the process's own board.json hash), unit settings, journal and service-manager records (1 start, 0 restarts, Result=success, exited 0), sweep cadence, stream coverage, call budget use, errors (406: 0, 429: 0 recorded), the no-order basis with the partial account 2 listing, and sha256/bytes/records of every private file. The console is committed byte-identical; the data stays private. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Overnight volume watch 2026-09-28/29: sanitized run receipt (176 sweeps, 75 alerts, 0 orders) Receipt for the BOATS overnight watch of trade date 2026-09-29 (22:09:58-04:00:04 ET, stop_time, account 1 data only): code identity from start.json (dca569ae, not on main; its watch.py is the same blob as 6545d7a in refs/pull/480/head history, and main's #480 squash differs after the review repair round), thresholds and plan, universe and reference coverage, 176 sweeps at 120 s with all 3,520 chunks ok, 3,619 requests with 0 failed (429: 0), 75 alerts (62 significant, 13 notify) with 6 posted to the loopback ntfy topic, the SPY session-bar roll probe, the no-order basis, and sha256/bytes/records of every private file. The data stays private; no alerted symbol is named. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Watcher receipts 2026-09-29: repairs after the independent verification Overnight volume watch: the watch ran under the transient systemd user unit overnight-volume-watch-20260929.service, not from a shell without a unit. Its journal (queried from 2026-09-28, where the start falls) gives the start at 22:09:29 ET, the ExecStart with the $PAPER_ENV_FILE pointer, one start and no restart, the watch's start and summary lines on stdout, the resource record at 04:00:04 ET, no exit line, and the unit file gone by 04:09:59 ET. started_at now follows the template (the unit start). The output directory's birth time (.233 s) precedes the Started record by 50 ms and the watch's own start-up by 0.12 to 0.22 s, so its creator stays unrecorded. The start-up difference to main is stated precisely, the 14 missing names are marked as unrecorded, and the missing provenance entries are added. Incentive monitor: the monitor's output root gets its own placeholder, a sibling of <private state root>; the launch, unit and timer hashes are cited as recorded before the run in SERIES-INDEX.md; the start-up retry() is named as a second silent-retry surface, and the call counts show neither surface retried (1 + 1 calls before sweep 1, 68 daily-bar calls for 68 requests); the freeze notes' statements about the monitor are quoted and met; the missing provenance entries are added. Both receipts cite the 2026-09-29 freeze notes by sha256 (frozen_expectations, private_notes_sha256) and use defined placeholders for private paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Receipt: pre-20260929 pre-market mover paper series (3 of 3 trials pass, flat) Sanitized receipt of the 2026-09-29 PRE series on account 2 (alpaca-paper-2), engine b528bb5: receipt.json with per-trial frozen-acceptance verdicts, 18 orders asserted across engine receipt, ledger snapshot and broker reconciliation, withheld engine receipts (broker_order_ref only), the byte-identical console, frozen SUMS and reconciliation stdout, and README. Realized -2.05 USD; mechanics-only, no validated edge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Receipt: rth-20260929 regular-hours mover paper series (5 of 6 pass; trial 6 fails on a ledger halt) Sanitized receipt of the 2026-09-29 RTH series on account 1 (alpaca-paper), engine b528bb5. Trial 6 ended with ledger halted_reason gross_loss_cap_reached (lifetime gross loss 250.18 of 250 USD) although the engine reported passed, so it fails the frozen no-halt clause. Trial 5's two E2 average-invariant records are explained from native_adapter.py (broker 6-decimal average one unit high) and fall under no frozen clause. 46 orders asserted across engine receipt, ledger snapshot and broker reconciliation; withheld engine receipts, byte-identical console, SUMS and reconciliation stdout, and README. Realized -115.94 USD; mechanics-only, no validated edge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Receipt: ext-20260929 after-hours mover paper series (3 of 4 pass; trial 4 needs attention, IOVA 68 left open) Sanitized receipt of the 2026-09-29 POST series on account 2 (alpaca-paper-2), engine b528bb5. Trial 1 ended completed_no_signals (its 5 entries were canceled unfilled); trials 2 and 3 passed on controller_stop exits. Trial 4's exits were forced by transport_gap; the engine handed off to recovery (no_exit_progress), the recover command sold BEX 24 (+1.68) and its IOVA 68 sale was refused before HTTP (transport_proven_not_sent, SubmissionNotSent), so trial 4 is needs_attention: not flat, ledger recovery_only, pnl_consistent false. The durable budget never reserved the IOVA submit, which excludes the rate caps and the wire guard; the files cannot distinguish validate_pending's quote check from the 0.25 s submit-hook deadline. 22 engine and 2 recovery orders asserted across engine receipts, ledger snapshot and both broker reconciliation runs (0 foreign orders or fills). The 2026-09-30 residual recovery is recorded as pending: 17 attempts, 0 orders, each cash_mismatch_or_unmodeled_fees from -0.47 USD of 2026-09-29 fees. Realized -15.38 USD with IOVA 68 open at cost 983.28; mechanics-only, no validated edge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Receipts 2026-09-29: repair pre, rth and ext after the independent verification - Websocket lines: attempts[].errors.websocket_lines_without_close_frame now counts the lines with no close frame received (pre 1/1/1, rth 0/1/1/0/1/1, ext 1/2/1/3); the new websocket_lines_no_close_frame_either_way keeps ext's two trading-stream lines with no close frame either way (0/1/0/1). A new websocket_lines object defines both counters from the pinned websockets 17.1 and alpaca-py 0.44.0 sources. The READMEs no longer call the data-stream lines normal closes (they are ConnectionClosedError: the closing handshake did not complete). - The 2026-09-23 isolation check is described from its committed output, which records both forms after stop() (0.016 s and 0.018 s), not only the trading-stream line. - Offline gate: the private gate log and rc record are re-checked and cited by sha256 (402 tests, OK, rc 0), replacing "the gate log no longer exists". - Units: exit status and result come from the systemd user manager (units[].manager_state, read-only), and the journal statement is kept. - scan_log_sha256 renamed to scan_full_sha256 (same values); the config fields the receipt had left out are added; the ledger.snapshot pointer is fixed; the runtime basis is corrected for pre and ext; the reconciliation command line is limited to its recorded basis; field provenance is completed. No verdict and no total changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Receipts 2026-09-29: record the broker's FEE activities from the coordinator's FEE read - rth (account 1): fees lists REG -0.34, TAF -0.30 and CAT -0.01 USD (-0.65) from fee-activities-20260929.json (sha256 2d02c55b...), replacing "not read"; series_totals adds fees_usd -0.65 and realized_pnl_including_fees_usd -116.59 beside the unchanged -115.94. The read keeps no description, so the fee bases are not recomputed. - pre and ext (account 2): the REG -0.19, TAF -0.27 and CAT -0.01 rows (-0.47) are now asserted across the cash-gap output and the FEE read. The broker posts one activity per fee type per day, so no split between the series is made (series_totals fees_usd null); the fees appear only in a joint pre+ext figure: realized -17.43, including fees -17.90. - ext: the residual section and residual_recovery_20260930.sources cite both reads; status pending and the PENDING line are unchanged. - Every new number has a field_provenance entry; the limitations and engine_fee_model sentences now name the fee-inclusive figures; revised_at_utc and revision_note record this revision. No verdict and no existing figure changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Catalog: the September 28 paper series did not start; cite the September 29 receipts The stale sentence said the 2026-09-28 series still waited on a rejected paper key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Receipts 2026-09-29: complete field provenance and correct ext wording after the further verification ext-20260929: - README: scope the recover command's request count to its recovery start (17:10:04 ET to the IOVA reservation: 4 reads and 1 submit), as recovery_20260929.recover_command already was; name the IOVA leg's exit_wait_reason (last_wait_reason is null); label the 0.25 s hook deadline as a path beyond the two candidates the coordinator's brief named; note trial 4's null ref on the never-sent BEX exit. - FREEZE.md stays private because it names a private host path, not account fingerprints (none found with the checker's fingerprint set); the private series index names fingerprints (frozen_acceptance.source, private_notes_note, README Files). - field_provenance.asserted: ledger.snapshot, residual_recovery_20260930 .residual and the development study's 0 of 768 (limitations[]). pre-20260929 and rth-20260929: the same ledger.snapshot and 0-of-768 provenance entries (same clause-4 gap). No verdict and no figure changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Receipts 2026-09-29: remove an unsupported fill cause; say which branch left the dashboard alone Cross-family review (gpt-6.1-sol ultra, lens B) found two problems: - the RTH receipt attributed its 17 worse-than-reference fills to walking the book. Paper fills are simulated from real-time quotes (Alpaca, Paper Trading), so the cause is now left unrecorded; - the three series receipts said this branch does not touch the dashboard. The builder branches did not, but the integrated pull request does, in its last commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Grand dashboard: paper lane row at the 2026-09-29 receipts; register the receipts observability/grand-dashboard/state.json: the paper lane row (and the checkpoint time) now states the 2026-09-29 series results and cites ext-20260929/README.md. No other row changes. manifests/evidence.json: the receipt files and the changed catalog README, re-registered through the docs/lanes.md hot-file protocol (last commit, main's manifest). scripts/validate.py and scripts/evidence_manifest.py --check: passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Scout <scout@local> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Scout and others added 2 commits September 28, 2026 22:10

seathatflowsinourveins added the lane:trading Trading lane: sim, paper and live north star label Sep 29, 2026

seathatflowsinourveins enabled auto-merge (squash) September 29, 2026 03:18

seathatflowsinourveins disabled auto-merge September 29, 2026 03:19

seathatflowsinourveins enabled auto-merge (squash) September 29, 2026 03:19

seathatflowsinourveins merged commit 1b8d633 into main Sep 29, 2026
25 checks passed

seathatflowsinourveins deleted the claude/overnight-volume-watch-20260929 branch September 29, 2026 04:08

seathatflowsinourveins mentioned this pull request Sep 29, 2026

Roadmap update 2026-09-29 (later): trading lane owned, paper series frozen, foundation sweep running, ai-memory candidate only #496

Merged

Navigation Menu

Repository navigation

Overnight volume watch: data-only BOATS significant-volume monitor over the full overnight-tradable universe - #480

Overnight volume watch: data-only BOATS significant-volume monitor over the full overnight-tradable universe#480
seathatflowsinourveins merged 3 commits into
mainfrom
claude/overnight-volume-watch-20260929

seathatflowsinourveins commented Sep 29, 2026

Uh oh!

seathatflowsinourveins commented Sep 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

seathatflowsinourveins commented Sep 29, 2026

Summary

Live run (2026-09-29, trading lane: ecosystem-roadmap-2026)

SOTA sources

Checks

Not verified

Uh oh!

seathatflowsinourveins commented Sep 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant