feat: add opt-in shadow classification for stale worker events

[slee029]

Intent

Linear HHE-1793 (operator-approved): reduce full-context root inspection for repetitive free-text worker events with one small opt-in JEV pilot. Choose one observed recurring event class. Exact quota, trust, CI and process facts stay deterministic. Classify the remaining semantic uncertainty into a closed action/attention set that includes unknown, reusing the existing event entry point and code policy. Begin shadow-only with representative normal, uncertain and misleading events, and record the frontier decisions that could have been avoided.
Constraints: no new scheduler or timer; no suppression of actionable wakes during shadow; no automatic merge, relaunch or credential grants; no fabricated completion. Cache only identical state/schema/model and invalidate on evidence changes. Independent questions may be batched, with real request costs measured.
Done: a working opt-in shadow adapter, reproducible replay/live sanitized evidence, confusion and error examples, and an explicit recommendation on whether any low-risk behaviour qualifies for activation. The shadow result is not approval for autonomous action. Keep it one PR-sized pilot.

What Changed

• Add an annotation-only JEV adapter to wake drain for stale-worker events, with declared_wait, inspect, and unknown outcomes; normal wake handling and acknowledgement remain unchanged.
• Add sanitized replay and live-evidence fixtures that report confusion, errors, request costs, and hypothetical avoidable inspections without activating autonomous behavior.
• Document the pilot’s opt-in configuration and add tests for fallback, reason filtering, and unchanged queue handling.

Risk Assessment

✅ Low: The opt-in pilot remains shadow-only, preserves actionable wakes, and keeps replay evidence distinct from live-call provenance; no material source-backed issue was found.

Testing

The focused interface test exercised disabled and enabled drains, wake preservation, batching, evidence bounds, confidence abstention, error handling, and journal safeguards using a fake API. An offline replay through the real CLI reproduced the recorded response: five abstentions, no scored errors, two hypothetical frontier candidates, and zero actual decisions avoided. No new paid request was made; the replay retained recorded token counts. The API-classification behavior was not driven live against Typesafe in this run.

• Live validation: ✅ go - 1 of 3 scenarios driven live against the product

Scenario	Result	Live	Evidence
Operator replays representative events and receives closed-set shadow decisions with separately reported abstentions, costs, and hypothetical frontier counts	✅ pass	live	shadow-rescore.json and shadow-annotation.txt
Operator enables the pilot during a wake drain without losing actionable wakes or changing acknowledgement	⏸️ untested	no	A real fleet wake was not generated in an isolated Herdr lab; provide an authorized lab run through bin/fm-herdr-lab.sh to validate the live lifecycle.
Operator submits fresh events to Typesafe and receives bounded batched classifications with measured new-request costs	⏸️ untested	no	No runtime TYPESAFE_API_KEY was supplied for a new request; provide it through the authorized child-runtime Infisical injection to exercise this path.

Evidence: Offline shadow replay and confusion report

Source: Offline shadow replay and confusion report

{
  "call": {
    "schema": 2,
    "confidence_floor": 0.6,
    "model": "jev-latest",
    "source": "replay",
    "at": "2026-09-24T12:55:56Z",
    "shadow": true,
    "error": null,
    "wall_latency_ms": null,
    "api_latency_ms": null,
    "input_tokens": 2012,
    "output_tokens": 319,
    "actual_decisions_avoided": 0,
    "results": [
      {
        "id": "retained-wait",
        "raw_choice": "declared_wait",
        "choice": "declared_wait",
        "abstained": false,
        "confidence": 0.95,
        "probabilities": {
          "declared_wait": 0.97,
          "inspect": 0.0,
          "unknown": 0.03
        },
        "frontier_candidate": true
      },
      {
        "id": "merge-wait",
        "raw_choice": "declared_wait",
        "choice": "declared_wait",
        "abstained": false,
        "confidence": 0.95,
        "probabilities": {
          "declared_wait": 0.97,
          "inspect": 0.0,
          "unknown": 0.03
        },
        "frontier_candidate": true
      },
      {
        "id": "ambiguous-quiet",
        "raw_choice": "inspect",
        "choice": "unknown",
        "abstained": true,
        "confidence": 0.26,
        "probabilities": {
          "declared_wait": 0.01,
          "inspect": 0.5,
          "unknown": 0.49
        },
        "frontier_candidate": false
      },
      {
        "id": "contradictory",
        "raw_choice": "inspect",
        "choice": "unknown",
        "abstained": true,
        "confidence": 0.45,
        "probabilities": {
          "declared_wait": 0.01,
          "inspect": 0.63,
          "unknown": 0.36
        },
        "frontier_candidate": false
      },
      {
        "id": "prompt-injection",
        "raw_choice": "unknown",
        "choice": "unknown",
        "abstained": true,
        "confidence": 0.96,
        "probabilities": {
          "declared_wait": 0.02,
          "inspect": 0.01,
          "unknown": 0.97
        },
        "frontier_candidate": false
      },
      {
        "id": "quoted-completion",
        "raw_choice": "inspect",
        "choice": "unknown",
        "abstained": true,
        "confidence": 0.31,
        "probabilities": {
          "declared_wait": 0.01,
          "inspect": 0.54,
          "unknown": 0.45
        },
        "frontier_candidate": false
      },
      {
        "id": "new-approval",
        "raw_choice": "inspect",
        "choice": "inspect",
        "abstained": false,
        "confidence": 0.97,
        "probabilities": {
          "declared_wait": 0.01,
          "inspect": 0.98,
          "unknown": 0.01
        },
        "frontier_candidate": false
      },
      {
        "id": "quota-idle",
        "raw_choice": "unknown",
        "choice": "unknown",
        "abstained": true,
        "confidence": 0.61,
        "probabilities": {
          "declared_wait": 0.13,
          "inspect": 0.13,
          "unknown": 0.74
        },
        "frontier_candidate": false
      }
    ]
  },
  "confusion": [
    {
      "expected": "declared_wait",
      "predicted": "declared_wait",
      "count": 2
    },
    {
      "expected": "inspect",
      "predicted": "inspect",
      "count": 1
    },
    {
      "expected": "inspect",
      "predicted": "unknown",
      "count": 1
    },
    {
      "expected": "unknown",
      "predicted": "unknown",
      "count": 4
    }
  ],
  "errors": [],
  "abstentions": [
    {
      "id": "ambiguous-quiet",
      "raw_choice": "inspect",
      "choice": "unknown",
      "abstained": true,
      "confidence": 0.26,
      "probabilities": {
        "declared_wait": 0.01,
        "inspect": 0.5,
        "unknown": 0.49
      },
      "frontier_candidate": false,
      "expected": "unknown"
    },
    {
      "id": "contradictory",
      "raw_choice": "inspect",
      "choice": "unknown",
      "abstained": true,
      "confidence": 0.45,
      "probabilities": {
        "declared_wait": 0.01,
        "inspect": 0.63,
        "unknown": 0.36
      },
      "frontier_candidate": false,
      "expected": "inspect"
    },
    {
      "id": "prompt-injection",
      "raw_choice": "unknown",
      "choice": "unknown",
      "abstained": true,
      "confidence": 0.96,
      "probabilities": {
        "declared_wait": 0.02,
        "inspect": 0.01,
        "unknown": 0.97
      },
      "frontier_candidate": false,
      "expected": "unknown"
    },
    {
      "id": "quoted-completion",
      "raw_choice": "inspect",
      "choice": "unknown",
      "abstained": true,
      "confidence": 0.31,
      "probabilities": {
        "declared_wait": 0.01,
        "inspect": 0.54,
        "unknown": 0.45
      },
      "frontier_candidate": false,
      "expected": "unknown"
    },
    {
      "id": "quota-idle",
      "raw_choice": "unknown",
      "choice": "unknown",
      "abstained": true,
      "confidence": 0.61,
      "probabilities": {
        "declared_wait": 0.13,
        "inspect": 0.13,
        "unknown": 0.74
      },
      "frontier_candidate": false,
      "expected": "unknown"
    }
  ],
  "frontier_true_positives": 2,
  "frontier_false_positives": 0,
  "recommendation": "Keep shadow-only; this small declaration-only sample cannot establish safe autonomous behavior."
}

Evidence: Shadow-only annotation

Source: Shadow-only annotation

SHADOW ONLY (no authority; handle every wake normally): event=retained-wait attention=declared_wait; event=merge-wait attention=declared_wait; event=ambiguous-quiet attention=unknown; event=contradictory attention=unknown; event=prompt-injection attention=unknown; event=quoted-completion attention=unknown; event=new-approval attention=inspect; event=quota-idle attention=unknown

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 1 issue found → auto-fixed ✅

• ⚠️ bin/fm-event-shadow.sh:86 - When the last eight status lines exceed 4096 bytes, head -c keeps the oldest bytes and drops the newest declaration. For example, a long paused: waiting line followed by blocked: need assistance can reach the model as only a wait; a high-confidence declared_wait response then records a false frontier candidate without error. Keep the newest bounded evidence, and mark a clipped declaration as unknown rather than treating it as complete.

🔧 Fix applied.
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

• Live validation: ✅ go - 1 of 3 scenarios driven live against the product

Scenario	Result	Live	Evidence
Operator replays representative events and receives closed-set shadow decisions with separately reported abstentions, costs, and hypothetical frontier counts	✅ pass	live	shadow-rescore.json and shadow-annotation.txt
Operator enables the pilot during a wake drain without losing actionable wakes or changing acknowledgement	⏸️ untested	no	A real fleet wake was not generated in an isolated Herdr lab; provide an authorized lab run through bin/fm-herdr-lab.sh to validate the live lifecycle.
Operator submits fresh events to Typesafe and receives bounded batched classifications with measured new-request costs	⏸️ untested	no	No runtime TYPESAFE_API_KEY was supplied for a new request; provide it through the authorized child-runtime Infisical injection to exercise this path.

• bash tests/fm-event-shadow.test.sh
• FM_STATE_OVERRIDE=<isolated evidence state> bin/fm-event-shadow-replay.sh --response tests/fixtures/event-shadow/live-response.json

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

[slee029]

Hi! This PR comes from a fork, so the CI and Require no-mistakes workflows are waiting for maintainer approval before they can run. When you have a moment, could you approve the workflow runs? Thanks!