Skip to content

Port privacy-filter-canary into trace-commons-operator-client - #127

Merged
zmanian merged 7 commits into
mainfrom
trace-commons-operator-privacy-filter-canary
May 20, 2026
Merged

zmanian merged 7 commits into
mainfrom
trace-commons-operator-privacy-filter-canary

Conversation

@zmanian

@zmanian zmanian commented May 19, 2026

Copy link
Copy Markdown
Collaborator

Context

Stacks on #126. Removes the last stub from the operator-binaries migration: trace-commons-tenant privacy-filter-canary now runs the real local sidecar canary, with the adapter ported into trace-commons-operator-client.

What lands

trace-commons-operator-client::privacy_filter (new module, 625 LOC)

Ported from Ironclaw's ironclaw_reborn_traces::contribution:

  • pub trait PrivacyFilterAdapter + NoopPrivacyFilterAdapter (tests) + CommandPrivacyFilterAdapter (subprocess runner with input/stdout/stderr size limits, timeout, cleared env except PATH/LANG/LC_ALL).
  • SafePrivacyFilterRedaction, SafePrivacyFilterSummary, PrivacyFilterSidecarRequest — wire shapes byte-identical to Ironclaw's so existing sidecar binaries plug in unchanged.
  • safe_privacy_filter_redaction_from_output parser.
  • adapter_from_env() -> Option<Arc<dyn PrivacyFilterAdapter>>.
  • canary_leaked_tokens(original, redacted) -> Vec<String> — hash-only output (never raw tokens), matches Ironclaw's hash-only logging discipline.

Env-var precedence (with backwards compat)

Canonical Legacy fallback
TRACE_COMMONS_PRIVACY_FILTER_COMMAND IRONCLAW_TRACE_PRIVACY_FILTER_COMMAND
TRACE_COMMONS_PRIVACY_FILTER_ARGS IRONCLAW_TRACE_PRIVACY_FILTER_ARGS
TRACE_COMMONS_PRIVACY_FILTER_TIMEOUT_MS IRONCLAW_TRACE_PRIVACY_FILTER_TIMEOUT_MS
TRACE_COMMONS_PRIVACY_FILTER_MAX_INPUT_BYTES IRONCLAW_TRACE_PRIVACY_FILTER_MAX_INPUT_BYTES
TRACE_COMMONS_PRIVACY_FILTER_MAX_STDOUT_BYTES IRONCLAW_TRACE_PRIVACY_FILTER_MAX_STDOUT_BYTES
TRACE_COMMONS_PRIVACY_FILTER_MAX_STDERR_BYTES IRONCLAW_TRACE_PRIVACY_FILTER_MAX_STDERR_BYTES

TRACE_COMMONS_* wins when both are set. Verified by tests adapter_from_env_prefers_canonical_over_legacy and adapter_from_env_falls_back_to_legacy.

trace-commons-tenant privacy-filter-canary handler

Stub replaced with the real implementation: build adapter from env, run redact_text with --timeout-seconds, compute leaked tokens, render JSON envelope (via format::emit_json with method: \"LOCAL\") or human-readable summary. Bails with non-zero exit code if any canary token leaks through.

Runbook

docs/operator/operator-binaries.md updated: removed the "currently stubbed" note, documented the env-var matrix with canonical + legacy names, added a worked example for invoking the canary against a Python sidecar.

Tests

trace-commons-operator-client::privacy_filter: 30 → 39 (+9 new) — parser ok-case, parser missing-redacted-text, canary leak detector, NoopAdapter, adapter_from_env unset / canonical / legacy / overrides, end-to-end command-adapter round-trip via /bin/sh, canonical hash format.

trace-commons-tenant: 26 → 27 (+1 — privacy_filter_canary_errors_when_unconfigured).

Verification

RUSTFLAGS=\"-D warnings\" cargo check --workspace --all-targets   # clean
RUSTFLAGS=\"-D warnings\" cargo test --workspace --no-run         # clean
cargo test -p trace-commons-operator-client                       # 39 passed
cargo test -p trace-commons-server --bins                         # all green
cargo clippy --workspace --all-targets -- -D warnings              # clean

Deviations from Ironclaw reference

  1. Returns anyhow::Result rather than Ironclaw's TraceContributionError. The operator-client has no need to depend on the trace-contribution error taxonomy. Wire shapes unchanged.
  2. canary_leaked_tokens is whitespace-token comparison (≥4 chars) returning sha256:<hex> hashes only. Matches Ironclaw's hash-only logging discipline and the operator-CLI's user-supplied-text shape.
  3. Human-readable + JSON rendering synthesized to match operator-binary conventions (format::emit_json envelope with method: \"LOCAL\" since this isn't an HTTP request).

Net diff

6 files, +837 / -54.

Out of scope

  • Removing the legacy IRONCLAW_TRACE_PRIVACY_FILTER_* env-var compat reads — defer until operators are migrated. The tracing::warn! on legacy-fallback usage will signal when adoption is complete.

🤖 Generated with Claude Code

zmanian added 6 commits May 19, 2026 14:48
New workspace member that holds the shared HTTP transport + foundational
types for the upcoming operator binaries (trace-commons-review,
trace-commons-worker, trace-commons-admin, trace-commons-tenant). These
binaries take over the operator-side CLI surface that was pruned from
Ironclaw's reborn trace client in nearai/ironclaw#3738.

Surface in this PR:
- Client with bearer-token resolution from an env var (never logged),
  host-allowlist enforcement, and typed error mapping for the server's
  {"error": "<Label>"} refusal envelope.
- Error taxonomy: BearerMissing, InvalidEndpoint, HostNotAllowed,
  Transport, HttpFailure, ServerLabel, MalformedResponse. Each carries
  a user_diagnostic() that strips query strings + fragments.
- HostAllowlist with exact-match semantics, case-insensitive, sourced
  from --allowed-hosts or TRACE_COMMONS_ALLOWED_HOSTS.
- Output helpers: JsonEnvelope wrapper for --json mode, ASCII table
  renderer, key-value renderer for single-record commands.

Per-endpoint typed request/response structs land in PR #2 (reviewer +
admin) and PR #3 (worker + tenant). When a struct turns out to be
shared across binaries it migrates back here.

Verification:
- cargo check --workspace --all-targets under -D warnings: clean
- cargo test -p trace-commons-operator-client: 30 passed
- cargo clippy -p trace-commons-operator-client --all-targets -- -D warnings: clean
Wire up the reviewer-audience (8 subcommands) and admin-audience
(12 subcommands) CLI binaries on top of the trace-commons-operator-client
foundation crate. Each binary takes a top-level --endpoint, --bearer-token-env,
--allowed-hosts, and --json flag and delegates per-endpoint typing to the
foundation Client. A shared operator_common module exposes table / kv / map
rendering helpers consumed by both bins via #[path = ...].

Coverage matches the Ironclaw reference handlers in
crates/ironclaw_reborn_cli/src/commands/traces.rs at the same paths
(/v1/review/*, /v1/admin/*, /v1/benchmarks/*, /v1/datasets/replay,
/v1/analytics/summary).
Adds a privacy_filter module to the operator-client foundation crate
holding the subprocess sidecar adapter, safe-summary parser, env-var
configuration reader, and canary-leak helper. The wire shapes
(PrivacyFilterSidecarRequest, SafePrivacyFilterRedaction, RedactionReport,
SafePrivacyFilterSummary) match the Ironclaw types byte-for-byte so
operators can repoint existing sidecar binaries unchanged.

Environment variables prefer the canonical TRACE_COMMONS_PRIVACY_FILTER_*
names and fall back to the Ironclaw-era IRONCLAW_TRACE_PRIVACY_FILTER_*
names with a one-shot warn log on fallback.

The trace-commons-tenant privacy-filter-canary subcommand now drives the
real adapter: it spawns the configured sidecar, verifies no canary token
survives redaction, and renders either a human-readable result or the
operator JSON envelope. The old anyhow::bail! stub is gone.

operator-binaries.md drops the deferred-port note and documents the env
vars (canonical + legacy) plus the failure modes.
CI on PR #124 surfaced fmt diffs across the operator-client foundation
and the review/admin/worker binaries. Applied cargo fmt --all to bring
the stack into compliance. Pure formatting; zero behavior change.
@zmanian
zmanian changed the base branch from trace-commons-operator-binaries-worker-tenant to main May 20, 2026 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant