Skip to content

fix(c7): C7 backfill slice C2 -- 11 durable-state findings in cron/plugins/CI scripts (t_779040d0) - #1371

Merged
Kyzcreig merged 6 commits into
mainfrom
daedalus/t_779040d0-c7-c2-backfill
Sep 28, 2026
Merged

Kyzcreig merged 6 commits into
mainfrom
daedalus/t_779040d0-c7-c2-backfill

Conversation

@ang-fleet-workers

@ang-fleet-workers ang-fleet-workers Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Re-verified 14 C7 instances at 533612e (then rebased on ab6b6d4).
Every fix has a RED-on-base test.

Fixed:

  • k87 cron/fork_ext/scheduler_ext.py: timeout_s "inf"/float('inf') raised
    OverflowError out of resolve_job_script_timeout; now falls back.
  • k88 cron/scheduler.py: host-down ledger row was written before the send;
    now written only once the demoted #logs delivery succeeds.
  • k116 plugins/blackbox/init.py: prefix guard got cache_read /
    prompt_tokens as measured numbers when buckets were unknown; now None.
  • k117 plugins/blackbox/store.py: turn_api_calls stored unknown buckets as
    0, so first_call_cache_miss judged an unmeasured read; now NULL.
  • k119 lcm/db_bootstrap.py: a trigger-only repair cleared a deep/parity
    corruption flag; now only a rebuild clears it (a structural flag is
    cleared when the structure verifies).
  • k120 lcm/db_bootstrap.py: repair rolled back a caller-owned transaction
    on failure; now rolls back only its own (captured at entry).
  • k122 mem0/capture_drain.py: the exactly-once shortcut (add committed,
    crash before routing) skipped Arm-B routing; now routes (staging is
    keyed by turn id, so a re-route overwrites in place).
  • k125 scripts/ci/live_comment.py: re-run dedupe kept the first-listed
    artifact; now newest (created_at, id) wins.
  • k127 scripts/ci_overflow_integration.py: an executed job with no PROBE
    line gave PASS; now UNVERIFIABLE.
  • k128 scripts/ci_overflow_integration.py: one matching slice certified a
    selective re-run as full; full_rerun_verdict now BLOCKs when any job
    was carried over from the prior attempt.
  • k136 plugins/blackbox/store.py: migration test passed on a table that
    could not take an insert; the migration now adds every base column
    insert_api_call names, and the test inserts after migrating.

Dropped:

  • k124 FIXED on main by fix(guards): C4 backfill slice -- 6 confirmed guard/ownership findings (t_9c9e757c) #1353 (live_comment.py:659 body = last_body,
    test_comment_lookup_outage_is_retried_not_fatal).
  • k126 no rate-limit bug to fix: nothing on main runs live_comment.py (no
    .github reference; ci-review-comment was retired in aff7aea).
  • k133 FALSE: the autouse tests/conftest.py::_hermetic_environment sets
    HERMES_HOME per test; a probe confirmed get_hermes_home() is
    <tmp_path>/hermes_test, so the real scripts dir is never written.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

@ang-prism

ang-prism Bot commented Sep 28, 2026

Copy link
Copy Markdown

FleetReview

Review: pre-merge · head 073339d0e698 · duration 4m 31s
Profile: light (merit: default light: lines 375<800, files 17<1000000, hunks 30<1000000, no hot path) · policy: below-size-and-path-gates
Roster: B-assert-ctx → gpt-6-sol (openai), C-assert-xhigh → claude-code-opus-5-5 (anthropic), F → gpt-6-sol (openai), L6 → gpt-6-sol (openai)

profile: light (rule: default light: lines 375<800, files 17<1000000, hunks 30<1000000, no hot path) · round 0 · members: B-assert-ctx, L6, C-assert-xhigh, F · families: anthropic,openai

Confidence: 3/5

Findings

  • P1 scripts/ci_overflow_integration.py:143 — Missing Probe · agreed: B-assert-ctx,C-assert-xhigh,F (openai, anthropic)
  • P1 cron/scheduler.py:4535 — False ledger entry · agreed: B-assert-ctx,C-assert-xhigh,F (openai, anthropic)
  • P1 plugins/context_engine/lcm/db_bootstrap.py:3264 — Partial repair · agreed: B-assert-ctx,L6,C-assert-xhigh,F (openai, anthropic)
  • P1 tests/test_ci_overflow_integration.py:136 — Missing Probe Coverage · agreed: L6 (openai)
  • P1 scripts/ci_overflow_integration.py:209 — Reject attempt 1 as evidence of a full re-run · agreed: C-assert-xhigh,F (anthropic, openai)
  • P1 plugins/memory/mem0/capture_drain.py:301 — Retry can overwrite or duplicate already-staged world facts · agreed: C-assert-xhigh,F (anthropic, openai)
  • P1 plugins/blackbox/store.py:360 — Migrate indexed API-call columns before creating their indexes · agreed: C-assert-xhigh,F (anthropic, openai)
  • P1 scripts/ci_overflow_integration.py:108 — Do not PASS a re-run containing only an unprobed planner · agreed: F (openai)

FleetReview provenance · models: B=gpt-6-sol, C=claude-code-opus-5-5, F=gpt-6-sol · cost: $4.80 · duration: 4m 29s · rounds: 1 · files examined: 17

@Kyzcreig
Kyzcreig force-pushed the daedalus/t_779040d0-c7-c2-backfill branch 2 times, most recently from f615e89 to 21d18c3 Compare September 28, 2026 00:45
@ang-prism

ang-prism Bot commented Sep 28, 2026

Copy link
Copy Markdown

FleetReview

Review: pre-merge · head 21d18c340f8f · duration 3m 53s
Profile: light (merit: default light: lines 596<800, files 19<1000000, hunks 39<1000000, no hot path) · policy: changed-lines>400
Roster: B-assert-ctx → gpt-6-sol (openai), F → gpt-6-sol (openai), L6 → gpt-6-sol (openai)

PARTIAL — ensemble escalated: family floor: too few distinct model families completed

This review did not reach a trusted verdict, so it is not a gate pass and the findings below may be incomplete. They are posted so they can be read rather than lost in a terminal record.

Reviewed with 1 of 2 model families — anthropic unavailable.

profile: light (rule: default light: lines 596<800, files 19<1000000, hunks 39<1000000, no hot path) · round 1 · members: B-assert-ctx, L6, C-assert-xhigh, F · families: openai

Confidence: 1/5

Findings

  • P1 scripts/ci_overflow_integration.py:142 — Missing jobs · agreed: B-assert-ctx,F,C-assert-xhigh (openai)
  • P1 plugins/memory/mem0/capture_router.py:496 — Incorrect deduplication · agreed: B-assert-ctx,F (openai)
  • P1 plugins/blackbox/store.py:925 — Successful calls without usage are still persisted as measured zero · agreed: F (openai)
  • P1 plugins/blackbox/store.py:969 — Unknown cache reads still produce a definite fallback cold verdict · agreed: F (openai)
  • P1 scripts/ci_overflow_integration.py:222 — Full-rerun probe check accepts echoed commands as execution evidence · agreed: F (openai)
  • P1 plugins/context_engine/lcm/db_bootstrap.py:3367 — Clear a stale failure flag after a successful explicit integrity check · agreed: F (openai)
  • P1 cron/scheduler.py:4535 — False ledger entry · agreed: B-assert-ctx,C-assert-xhigh,F (openai)
  • P1 plugins/context_engine/lcm/db_bootstrap.py:3264 — Partial repair · agreed: B-assert-ctx,L6,C-assert-xhigh,F (openai)
  • P1 tests/test_ci_overflow_integration.py:136 — Missing Probe Coverage · agreed: L6 (openai)
  • P1 scripts/ci_overflow_integration.py:209 — Reject attempt 1 as evidence of a full re-run · agreed: C-assert-xhigh,F (openai)
  • P1 plugins/memory/mem0/capture_drain.py:301 — Retry can overwrite or duplicate already-staged world facts · agreed: C-assert-xhigh,F (openai)
  • P1 plugins/blackbox/store.py:360 — Migrate indexed API-call columns before creating their indexes · agreed: C-assert-xhigh,F (openai)
  • P1 scripts/ci_overflow_integration.py:108 — Do not PASS a re-run containing only an unprobed planner · agreed: F (openai)

FleetReview provenance · models: B=gpt-6-sol, C=claude-code-opus-5-5, F=gpt-6-sol · cost: $7.46 (estimated) · duration: 3m 48s · rounds: 2 · files examined: 19

@ang-fleet-lander

Copy link
Copy Markdown

🤖 merged-by: apollo · lane: kanban-merge-pass · gate: ADVISORY (FleetReview not green for 21d18c3): fleetreview-advisory-20260927-standing.md · why: t_779040d0: C7 backfill fix: hermes-agent slice C2: cron, plugins, CI scripts (14 instances); Argus off card review (Ace 13:08), CI green

@Kyzcreig
Kyzcreig force-pushed the daedalus/t_779040d0-c7-c2-backfill branch 2 times, most recently from 788221e to ef35814 Compare September 28, 2026 03:47
@ang-prism

ang-prism Bot commented Sep 28, 2026

Copy link
Copy Markdown

FleetReview

Review: pre-merge · head ef358148c3fd · duration 4m 40s
Profile: light (merit: default light: lines 596<800, files 19<1000000, hunks 39<1000000, no hot path) · policy: changed-lines>400
Roster: B-assert-ctx → gpt-6-sol (openai), C-assert-xhigh → gpt-6-sol (openai), F → gpt-6-sol (openai), L6 → gpt-6-sol (openai)

PARTIAL — ensemble escalated: family floor: too few distinct model families completed

This review did not reach a trusted verdict, so it is not a gate pass and the findings below may be incomplete. They are posted so they can be read rather than lost in a terminal record.

Reviewed with 1 of 2 model families — anthropic unavailable.

profile: light (rule: default light: lines 596<800, files 19<1000000, hunks 39<1000000, no hot path) · round 1 · members: B-assert-ctx, L6, C-assert-xhigh, F · families: openai

Confidence: 1/5

Findings

  • P1 scripts/ci_overflow_integration.py:130 — Skipped Jobs · agreed: B-assert-ctx,F (openai)
  • P1 scripts/ci_overflow_integration.py:152 — Missing jobs · agreed: B-assert-ctx,C-assert-xhigh,F (openai)
  • P1 cron/scheduler.py:4535 — Lost ledger rows · agreed: B-assert-ctx,C-assert-xhigh,F (openai)
  • P1 plugins/memory/mem0/capture_router.py:496 — Glob collision · agreed: B-assert-ctx,C-assert-xhigh,F (openai)
  • P1 plugins/memory/mem0/capture_drain.py:63 — Do not classify connection resets as proof that an add was never sent · agreed: C-assert-xhigh (openai)
  • P1 scripts/ci_overflow_integration.py:115 — Block a stale execution even when planner jobs were rerun · agreed: C-assert-xhigh (openai)
  • P1 scripts/ci_overflow_integration.py:222 — Echoed commands can masquerade as executed PROBE evidence · agreed: C-assert-xhigh (openai)
  • P1 plugins/context_engine/lcm/db_bootstrap.py:3367 — Clear stale corruption flags after a successful explicit integrity check · agreed: C-assert-xhigh (openai)
  • P1 plugins/blackbox/store.py:937 — Do not discard measured buckets for aggregate-only unknown usage · agreed: F (openai)
  • P1 plugins/blackbox/__init__.py:160 — Preserve measured prompt-side telemetry when only the aggregate is unknown · agreed: F (openai)
  • P1 scripts/ci_overflow_integration.py:143 — Missing Probe · agreed: B-assert-ctx,C-assert-xhigh,F (openai)
  • P1 plugins/context_engine/lcm/db_bootstrap.py:3264 — Partial repair · agreed: B-assert-ctx,L6,C-assert-xhigh,F (openai)
  • P1 tests/test_ci_overflow_integration.py:136 — Missing Probe Coverage · agreed: L6 (openai)
  • P1 scripts/ci_overflow_integration.py:209 — Reject attempt 1 as evidence of a full re-run · agreed: C-assert-xhigh,F (openai)
  • P1 plugins/memory/mem0/capture_drain.py:301 — Retry can overwrite or duplicate already-staged world facts · agreed: C-assert-xhigh,F (openai)
  • P1 plugins/blackbox/store.py:360 — Migrate indexed API-call columns before creating their indexes · agreed: C-assert-xhigh,F (openai)
  • P1 scripts/ci_overflow_integration.py:108 — Do not PASS a re-run containing only an unprobed planner · agreed: F (openai)

FleetReview provenance · models: B=gpt-6-sol, C=claude-code-opus-5-5, F=gpt-6-sol · cost: $4.78 · duration: 4m 37s · rounds: 2 · files examined: 19

@Kyzcreig
Kyzcreig force-pushed the daedalus/t_779040d0-c7-c2-backfill branch from ef35814 to 137e189 Compare September 28, 2026 06:58
@ang-prism

ang-prism Bot commented Sep 28, 2026

Copy link
Copy Markdown

FleetReview

FleetReview's daily member-call budget is spent (1082/1200 for 2026-09-28 UTC); review skipped.


FleetReview · reviewKind: skipped-budget

Kyzcreig pushed a commit that referenced this pull request Sep 28, 2026
- cron/scheduler.py: flush host-down ledger rows the moment the #logs send
  succeeds, not after all targets.
- mem0 capture_router: match staged turn file literally (glob.escape).
- ci_overflow_integration full_rerun_verdict: never-started jobs are not
  reused/missing; a prior-attempt job absent from the re-run BLOCKs.
- blackbox store/_measured + prefix guard: gate each bucket on its own
  unknown flag; aggregate-only usage_unknown keeps measured buckets.

Six new tests, each RED on the pre-fix head, GREEN after.
@ang-prism

ang-prism Bot commented Sep 28, 2026

Copy link
Copy Markdown

FleetReview

FleetReview's daily member-call budget is spent (1082/1200 for 2026-09-28 UTC); review skipped.


FleetReview · reviewKind: skipped-budget

…ugins/CI scripts (t_779040d0)

Re-verified 14 C7 instances at 533612e (then rebased on ab6b6d4).
Every fix has a RED-on-base test.

Fixed:
- k87  cron/fork_ext/scheduler_ext.py: timeout_s "inf"/float('inf') raised
       OverflowError out of resolve_job_script_timeout; now falls back.
- k88  cron/scheduler.py: host-down ledger row was written before the send;
       now written only once the demoted #logs delivery succeeds.
- k116 plugins/blackbox/__init__.py: prefix guard got cache_read /
       prompt_tokens as measured numbers when buckets were unknown; now None.
- k117 plugins/blackbox/store.py: turn_api_calls stored unknown buckets as
       0, so first_call_cache_miss judged an unmeasured read; now NULL.
- k119 lcm/db_bootstrap.py: a trigger-only repair cleared a deep/parity
       corruption flag; now only a rebuild clears it (a structural flag is
       cleared when the structure verifies).
- k120 lcm/db_bootstrap.py: repair rolled back a caller-owned transaction
       on failure; now rolls back only its own (captured at entry).
- k122 mem0/capture_drain.py: the exactly-once shortcut (add committed,
       crash before routing) skipped Arm-B routing; now routes (staging is
       keyed by turn id, so a re-route overwrites in place).
- k125 scripts/ci/live_comment.py: re-run dedupe kept the first-listed
       artifact; now newest (created_at, id) wins.
- k127 scripts/ci_overflow_integration.py: an executed job with no PROBE
       line gave PASS; now UNVERIFIABLE.
- k128 scripts/ci_overflow_integration.py: one matching slice certified a
       selective re-run as full; full_rerun_verdict now BLOCKs when any job
       was carried over from the prior attempt.
- k136 plugins/blackbox/store.py: migration test passed on a table that
       could not take an insert; the migration now adds every base column
       insert_api_call names, and the test inserts after migrating.

Dropped:
- k124 FIXED on main by #1353 (live_comment.py:659 `body = last_body`,
       test_comment_lookup_outage_is_retried_not_fatal).
- k126 no rate-limit bug to fix: nothing on main runs live_comment.py (no
       .github reference; ci-review-comment was retired in aff7aea).
- k133 FALSE: the autouse tests/conftest.py::_hermetic_environment sets
       HERMES_HOME per test; a probe confirmed get_hermes_home() is
       <tmp_path>/hermes_test, so the real scripts dir is never written.
- cron/scheduler.py: flush host-down ledger rows the moment the #logs send
  succeeds, not after all targets.
- mem0 capture_router: match staged turn file literally (glob.escape).
- ci_overflow_integration full_rerun_verdict: never-started jobs are not
  reused/missing; a prior-attempt job absent from the re-run BLOCKs.
- blackbox store/_measured + prefix guard: gate each bucket on its own
  unknown flag; aggregate-only usage_unknown keeps measured buckets.

Six new tests, each RED on the pre-fix head, GREEN after.
@Kyzcreig
Kyzcreig force-pushed the daedalus/t_779040d0-c7-c2-backfill branch from 550a9a9 to 3e8ef74 Compare September 28, 2026 12:23
@ang-prism

ang-prism Bot commented Sep 28, 2026

Copy link
Copy Markdown

FleetReview

FleetReview's daily member-call budget is spent (1082/1200 for 2026-09-28 UTC); review skipped.


FleetReview · reviewKind: skipped-budget

@ang-prism

ang-prism Bot commented Sep 28, 2026

Copy link
Copy Markdown

FleetReview

Review: pre-merge · head 430c5ba9eb47 · duration 6m 52s
Profile: light (merit: default light: lines 693<800, files 19<1000000, hunks 38<1000000, no hot path) · policy: changed-lines>400
Roster: B-assert-ctx → gpt-6-sol (openai), C-assert-xhigh → claude-code-opus-5-5 (anthropic), F → gpt-6-sol (openai), L6 → gpt-6-sol (openai)

profile: light (rule: default light: lines 693<800, files 19<1000000, hunks 38<1000000, no hot path) · round 2 · members: B-assert-ctx, L6, C-assert-xhigh, F · families: anthropic,openai

Confidence: 3/5

Findings

  • P1 scripts/ci_overflow_integration.py:134 — Duplicate Job Names · agreed: B-assert-ctx (openai)
  • P1 scripts/ci_overflow_integration.py:148 — Skipped Job · agreed: B-assert-ctx,L6,C-assert-xhigh,F (openai, anthropic)
  • P1 plugins/memory/mem0/capture_router.py:497 — Incomplete staging · agreed: B-assert-ctx,C-assert-xhigh,F (openai, anthropic)
  • P1 plugins/blackbox/store.py:1022 — Unknown cache writes can produce falsely complete tier totals · agreed: F (openai)
  • P1 cron/scheduler.py:3799 — Write suppression ledger only after a demoted target succeeds · agreed: F (openai)
  • P1 scripts/ci_overflow_integration.py:130 — Skipped Jobs · agreed: B-assert-ctx,F (openai)
  • P1 cron/scheduler.py:4535 — Lost ledger rows · agreed: B-assert-ctx,C-assert-xhigh,F (openai, anthropic)
  • P1 plugins/blackbox/store.py:937 — Do not discard measured buckets for aggregate-only unknown usage · agreed: F (openai)
  • P1 plugins/blackbox/__init__.py:160 — Preserve measured prompt-side telemetry when only the aggregate is unknown · agreed: F (openai)
  • P1 scripts/ci_overflow_integration.py:143 — Missing Probe · agreed: B-assert-ctx,C-assert-xhigh,F (openai, anthropic)
  • P1 plugins/context_engine/lcm/db_bootstrap.py:3264 — Partial repair · agreed: B-assert-ctx,L6,C-assert-xhigh,F (openai, anthropic)
  • P1 tests/test_ci_overflow_integration.py:136 — Missing Probe Coverage · agreed: L6 (openai)
  • P1 scripts/ci_overflow_integration.py:209 — Reject attempt 1 as evidence of a full re-run · agreed: C-assert-xhigh,F (anthropic, openai)
  • P1 plugins/memory/mem0/capture_drain.py:301 — Retry can overwrite or duplicate already-staged world facts · agreed: C-assert-xhigh,F (anthropic, openai)
  • P1 plugins/blackbox/store.py:360 — Migrate indexed API-call columns before creating their indexes · agreed: C-assert-xhigh,F (anthropic, openai)
  • P1 scripts/ci_overflow_integration.py:108 — Do not PASS a re-run containing only an unprobed planner · agreed: F (openai)

FleetReview provenance · models: B=gpt-6-sol, C=claude-code-opus-5-5, F=gpt-6-sol · cost: $5.64 · duration: 6m 56s · rounds: 1 · files examined: 19

@Kyzcreig

Copy link
Copy Markdown
Collaborator

🤖 merged-by: apollo · lane: review · gate: ADVISORY (FleetReview not green for 430c5ba): fleetreview-advisory-20260927-standing.md · why: t_779040d0 C7 slice C2 rebased 3x; CI 24/24; armed but never queued

@Kyzcreig
Kyzcreig disabled auto-merge September 28, 2026 14:44
@Kyzcreig
Kyzcreig merged commit 667885f into main Sep 28, 2026
42 checks passed
@Kyzcreig
Kyzcreig deleted the daedalus/t_779040d0-c7-c2-backfill branch September 28, 2026 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant