Native data (G2): token-tool savings into Loki and Grafana, one series per native scope

[seathatflowsinourveins]

Summary

The native-data adapter now publishes the token-tool savings to Loki as one series per native
scope. A new trend panel, 20, in the native-foundation dashboard draws them. On this workstation the
unchanged adapter left 5 measurement rows unknown. The changed adapter, run against the real tools
with a systemd-like environment and no publish, had all 9 measurement rows ok (6 savings,
3 memory), plus 68 coverage rows, with 0 unknown and 0 stale.

The adapter needed three fixes to get there:

• snapshot.py: optional report_scopes maps entity ids to this host's token-report scope labels.
  The labels stay private, and rows carry only fixed ids and titles.

• snapshot.py: optional ai_memory.server_url, an exact 127.0.0.1 origin, reaches the ai-memory
  2.4 status command only.

• native-data.service.example puts a Node 22+ directory on PATH for QMD's env node launcher.

• render.py, panel 20: each scope is its own series, grouped only by entity_id, never stacked
  or summed. A step shows a value only when the scope's newest row in that step is ok, so an
  earlier success never fills a failed or stale step. The legend carries no value.

• README.md: scheduled deployment (private paths, unit placeholders, stale bound). The native
  undo reads the timer's earlier enablement and activation separately. The section is marked
  changed after v2026.09.26.2.

• The adoption README and manifest list the recipe in the observability profile.

Scope

• What this PR changes: the native-data adapter (snapshot.py, render.py, the unit and config
  examples, the README), docs/native-dashboard-data.md, the observability profile's recipe paths
  in adoption/manifest.json and adoption/README.md, and tests/test_native_dashboard_data.py.
• Base commit: 803bc351. git merge-tree against origin/main dde28cc2 is clean.
• Lane: lane:foundation. manifests/evidence.json is only re-registered.
• Owned paths touched: observability/native-data/, docs/native-dashboard-data.md, adoption/
  (the observability profile, not a trading profile) and tests/.

What was verified

Claim	Evidence class	Command / receipt
Repository checks pass at 2b2b56e9	local integration	scripts/validate.py (6,917 hashed files, 159 receipts), evidence_manifest.py --check, component_matrix.py --check, new_host_grand_list.py --check and build_ecosystem.py --check all exit 0
Unit modules	synthetic	test_native_dashboard_data, test_adoption_docs_consistency, test_adoption_status and test_release_pin_contents: 162 OK, 1 skipped by design
Full suite	synthetic	unittest discover -s tests -q: 6,237 tests, 635 skipped, 1 failure: test_pre_commit_gate hit the per-user gitleaks lock while another worktree's suite ran. The single rerun of that module passed 3 of 3.
Panel 20 query shows no stale success	local integration	a sandboxed Loki 3.7.8 with a synthetic timeline, queried as Grafana does (step 180 s, [3m]). The old state="ok" query showed 3 earlier successes in failed or stale steps. The guarded query matches all 25 expected values and all 8 expected gaps.
Gaps render as gaps	source review	grafana-loki-datasource v13.2.0 pkg/loki/frame.go sets the step interval; Grafana v13.2.2 nullInsertThreshold.ts inserts nulls, drawn as gaps with spanNulls: false
Host scripts behave	synthetic	the sandbox harness (fake HOME, stateful systemctl stand-in, real coreutils) gave 20 PASS and 9 FAIL on the reviewed scripts and 29 of 29 on the repaired ones. ShellCheck 0.11.0 is clean and bash -n passes.
Host is untouched and ready	host observation	the read-only apply.sh --dry-run on this worktree exits 0: 77 rows, 0 unknown, 0 stale. On the live clone it refuses until merge. rollback.sh --dry-run reports nothing to undo.

Reviews (bounded to two rounds; the notes are in the session scratchpad):

• Round 1, adversarial review by a Claude Opus reviewer (p2rev/review-u2.md) found 10 defects,
  none blocking deployment. All 10 are fixed:

  • the trend panel showed an earlier success;
  • a Qdrant blip deployed without its row;
  • rollback acted on files changed after the apply;
  • apply never checked rows after the first run;
  • timer and --purge-state edge cases;
  • --repository could point at a /tmp worktree;
  • the only undo tool lived in the scratchpad;
  • prove.sh ignored panel settings and datasource;
  • a stale base;
  • a fallback read ai-memory's config.toml.

  Before the repair, 13 of 20 sandbox scenarios failed; after it, 20 of 20 passed. The resolution
  is u2-p2/REPAIR-1.md, commit b7e299fe.

• Round 2, GPT-6 cross-family review (codex-rev/review-p2-u2.md) found 4 medium issues, and
  all held against the reviewed code:

  1. rollback wrote through a symbolic link;
  2. prove.sh accepted overridden, hidden or foreign-datasource queries;
  3. an enabled but inactive timer was started without a record;
  4. the Loki evidence discarded native responses.

  All four are fixed. Findings 1, 2 and 4 are in the host scripts; finding 3 is in the README's
  native undo and a new contract test, commit 2b2b56e9. The resolution is u2-p2/REPAIR-2.md.

Host-apply plan (after merge)

apply.sh --apply runs only against the live clone, which follows origin/main, or a clean
checkout at origin/main outside temporary directories. So the host step waits until this PR has
merged and the live clone has synced.

1. apply.sh --dry-run is read-only. It runs a no-publish preflight snapshot and lists the actions.
2. apply.sh --apply writes the private config (0600) and the state directory (0700), then the
   ecosystem-native-data.{service,timer} user units and the provisioned dashboard file. It records
   each file's sha256 and a backup of any file it replaces. It also records the timer's earlier
   enablement and activation, and copies rollback.sh and prove.sh beside those records.
3. prove.sh runs one rtk ls in a fresh temporary directory, waits for the next timer run and
   checks the rows in Loki and the deployed panel through Grafana's anonymous Viewer API.
4. Rollback: rollback.sh --apply refuses, with nothing undone, if a recorded file changed after
   the apply. It undoes the timer's enablement and activation only where a recorded run changed
   them.

Residuals

• The host scripts are not in the tree. apply.sh, rollback.sh, prove.sh and their
  sandbox evidence live in the session scratchpad. The apply copies rollback.sh and prove.sh
  into its private state directory, but there is no committed receipt for the sandbox runs yet.
• Host apply and prove.sh on the host have not run. The exact-equality check is backed by a
  served-panel probe on the two existing dashboards, not yet on native-foundation-data.
• Only enabled, disabled and not-found timer states were exercised.
• Two fail-closed edge cases were reasoned from source and not run: rollback --force on a path
  that became a directory, and a string datasource reference in prove.sh.
• Unchanged from before:
  • Grafana keeps an unprovisioned copy of the dashboard after rollback (disableDeletion);
  • rows already pushed expire with Loki's 72 h retention;
  • panels 10 and 19 still read the Prometheus counters that the writer-identity change replaces;
  • the link panel still points to the original machine's URLs;
  • gaps are not yet rendered in a browser.
• Merge order: the writer-identity branch also re-registers manifests/evidence.json. Whichever
  merges second takes main's copy and re-registers its own files (hot-file protocol in
  docs/lanes.md). render.py and the tests merge textually.

SOTA sources

• grafana/grafana v13.2.2:
  • packages/grafana-data/src/transformations/transformers/nulls/nullInsertThreshold.ts and
    public/app/plugins/panel/timeseries/utils.ts (gap insertion; spanNulls);
  • docs/sources/administration/provisioning/index.md (file provisioning, disableDeletion).
• grafana/grafana-loki-datasource v13.2.0: pkg/loki/frame.go (the time field interval is the
  query step).
• grafana/loki v3.7.8: docs/sources/reference/loki-http-api.md (push and query APIs) and
  docs/sources/query/metric_queries.md (last_over_time, vector matching).
• systemd/systemd v255 (the host's systemd 255): man/systemd.timer.xml and man/systemctl.xml
  (enable --now, UnitFileState and ActiveState).

Decision record

No separate record. The deployment contract is in observability/native-data/README.md
("Scheduled deployment") and docs/native-dashboard-data.md.

🤖 Generated with Claude Code