Skip to content

test(verification): §5.1 measured local targets + Q9 defaults submission (#22) - #53

Merged
luongnv89 merged 2 commits into
mainfrom
refactor/22-3-8-measure-local-targets-and-obtain
Oct 5, 2026
Merged

luongnv89 merged 2 commits into
mainfrom
refactor/22-3-8-measure-local-targets-and-obtain

Conversation

@luongnv89

@luongnv89 luongnv89 commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

Closes #22

Summary

Measures every proposed §5.1 local target on the documented selected host (issue #22 / task 3.8) and submits the Q9 numerical defaults for owner acceptance — real wall-clock numbers through the real durable service boundaries, never invented.

Measured verdict: targets-passed — all 10 targets inside proposed bounds on the recorded run (docs/measurements/v1-targets-2026-10-05.json, 302.5 s elapsed).

How it was measured

tools/probes/local_targets.py drives the §8.2 World fixture stack — the same IntakeStore (SQLite, BEGIN IMMEDIATE + synchronous=FULL), intake/control/lane/recovery services, scripted ports. Because GitHub/model/Telegram/preview are scripted in-process ports, external network latency can never enter the measured path (reported as a separate 0 ms in-path term — the separation §5.1 requires). The intake soak was actually paced: 300 deliveries at a real 1 event/second for a measured 302.0 s wall.

Fixture (A1): one registered repo (luongnv89/money-mind), 100 pending tasks via the real accept path, exactly 10,000 retained events (200 real + 9,800 labelled benchmark_seed rows), ~1.5 MB store.

§5.1 target Bound Measured Verdict
Intake accept/reject p95, 1 ev/s × 5 min ≤ 2 s p95 11.73 ms (n=300) pass (~171×)
100 duplicate deliveries + restart → 1 task, ≤ 1 PR 10 s 52.3 ms → 1 row, 0 PRs pass
Local status p95 (100 queries) ≤ 1 s 0.24 ms pass
Control pause/cancel + fence persist ≤ 5 s handler p95 4.0 ms / commit p95 1.5 ms pass
Worker exit/quarantine + local notify ≤ 30 s 2.9 ms / 3.4 ms pass
Reconcile interval / dropped-work discovery ≤ 60 s / ≤ 120 s 60 s clamped / ≈1.0 s pass
Restart recovery of known active set ≤ 60 s 6.1 ms wall, 4/4 settled pass
Lost acknowledged tasks (crash tests) 0 0 / 57 recorded fault-matrix reps pass
Model calls in 30 idle minutes 0 0 pass
A6 mechanisms (observable state, no SLA claim, CI-wait exclusion, 24 h park, 5 min cap, stale-remote explicit) — all 10 checks pass

Decision record

docs/decisions/q9-acceptance.md — every proposed default (attempt/time budgets, 7-day logs/30-day audit retention, 24 h preview, 60 min cleanup/approval expiry, 10 min smoke freshness, retry/backoff bounds) with its enforcing constant and measured evidence. Status: submitted for owner acceptance — pending Luong's Q9 verdict. Nothing is silently finalized; unaccepted targets remain blockers per A7. The measured evidence supports accepting as proposed — no measured adjustment requested.

Test results

  • Full suite: 714 passed, 22 subtests (pytest -x)
  • New benchmark tests: 22 passed (tests/benchmarks/test_local_targets.py)
  • Probe --quick: targets-passed · full paced run archived: targets-passed (10/10 targets)
  • --fixture re-check of the archive: targets-passed
  • ruff check + ruff format: clean on changed files
  • gi-secscan --range origin/main: clean (7 scanned, 0 findings)

Acceptance criteria verification

  • A1 — 100 pending / 10,000 retained events on the selected host; environment + fixture identity + reproduce command recorded; network delay separate by construction. ✅ measured
  • A2 — paced 1 ev/s × 5 min p95 11.73 ms ≤ 2 s; ack-after-persist is structural (failed durable write → IntakeUnavailable, no ack); 100 dup + restart → 1 task, 0 PRs. ✅ measured
  • A3 — status p95 0.24 ms ≤ 1 s; stale remote explicitly labeled (forced unobserved pass >5 min → watermark + view stale, 1 dedup alert); control persist p95 4.0 ms ≤ 5 s, Telegram outside path. ✅ measured
  • A4 — confirm 2.9 ms / quarantine+alert 3.4 ms ≤ 30 s; replacement denied while uncertain; interval ≤ 60 s; discovery ≈1.0 s ≤ 120 s. ✅ measured
  • A5 — restart recovery 6.1 ms ≤ 60 s, 4/4 settled; crash-loss 0/57 (recorded §8.2 counters); idle 0 calls. ✅ measured
  • A6 — all mechanisms verified incl. forced-stale remote observation, no SLA wording, CI-wait excluded, 24 h park, 5 min cap w/ upstream override. ✅ verified
  • A7 — per-target pass/fail + uncertainty published; all defaults tabled; Q9 decision recorded as pending owner acceptance — no unaccepted target claimed achieved. ✅ submitted

Uncertainty: single host, single run, scripted ports — numbers are this host's local-boundary evidence, not end-to-end service claims.


qa-handoff: clean (1 review cycle, self-review + fix loop; no breaches)

…ion (#22)

Measure every proposed §5.1 local target on the documented selected
host through the real durable service boundaries (tests/faults World
fixture stack — scripted ports keep all GitHub/model/Telegram network
latency out of the measured path by construction):

- tools/probes/local_targets.py — paced intake (300 deliveries at a
  real 1 ev/s over 302 s wall → p95 11.7 ms vs 2 s), 100-delivery dup
  burst + mid-burst restart (52 ms → 1 task, 0 PRs), status p95
  0.20 ms, control persist p95 4.0 ms, cancel confirm 2.9 ms /
  quarantine+alert 3.4 ms, reconcile interval clamp 60 s + discovery
  ~1 s, restart recovery 6.1 ms (4/4 settled), 30-min idle 0 model
  calls, and 10 A6 mechanism checks incl. forced-stale remote labels.
  Crash-loss reads the recorded §8.2 audit counters (0 lost / 57 reps).
- tests/benchmarks/ — 22 fast mechanics tests of the probe.
- docs/measurements/v1-targets.md + dated JSON archive — per-target
  pass/fail, stats, margins, host/runtime/fixture identity.
- docs/decisions/q9-acceptance.md — every proposed default submitted
  for owner acceptance; pending, never silently finalized.

Verdict: targets-passed. Uncertainty (single host/run, scripted
ports) is recorded; unaccepted targets stay blockers per A7.
@luongnv89
luongnv89 merged commit a55b37e into main Oct 5, 2026
2 checks passed
@luongnv89
luongnv89 deleted the refactor/22-3-8-measure-local-targets-and-obtain branch October 5, 2026 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3.8: Measure local targets and obtain acceptance of proposed defaults

1 participant