diff --git a/CHANGELOG.md b/CHANGELOG.md index cf730a98..b8587b7f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -40,6 +40,7 @@ All notable changes to bicameral-mcp are tracked here. Format loosely follows ### Fixed +- `bicameral-report-bug` skill: `.bicameral/config.yaml` now defaults to keys-only inclusion in bug-report bodies (#200 finding A4). Workspace IDs, tokens, allowlists, and env-specific values are stripped by default; an explicit "include verbatim" toggle in the Step 3.5 transparency preview lets the operator opt in when the bug genuinely needs the values (e.g. a YAML parser regression). Defense-in-depth: the secret-redaction regex still runs on verbatim contents. Closes A4 in #200; A1 + A6 + partial A7 closed by #201. - Flow 3 e2e prompt clarified to use imperative shell phrasing for `git add` + `git commit` (#197). Resolves the "agent did NOT commit" flake observed on PR #194 + #195 e2e runs where the agent interpreted "stage and commit" as a non-shell verb. Adds a "Debugging Flow 3 fails" subsection to `tests/e2e/README.md` capturing the investigation order so the next maintainer has a starting checklist. ### Schema diff --git a/plan-200-config-yaml-redaction.md b/plan-200-config-yaml-redaction.md new file mode 100644 index 00000000..f783f7f9 --- /dev/null +++ b/plan-200-config-yaml-redaction.md @@ -0,0 +1,86 @@ +# Plan: bicameral-report-bug — `.bicameral/config.yaml` keys-only by default (#200 A4) (example) + +**change_class**: feature + +**doc_tier**: minimal + +## Open Questions + +None. Scope and design are framed by #200's A4 finding and the user's directive: "transparency + accuracy + minimum data shared." PR #201 (just merged) closed A1 (python3 portability) and A6 (browser-open). Partial coverage of A7 (rationale-field drop). This plan addresses only the remaining A4 surface (`.bicameral/config.yaml` verbatim leak). (example) + +## Phase 1: Default config.yaml inclusion to keys-only with explicit verbatim toggle + +The current Step 2 instruction (post-#201) says: *"`.bicameral/config.yaml`: use `Read` on `.bicameral/config.yaml` if it exists. If `Read` errors (file missing), skip the section."* The Step 3 body assembly then dumps `` verbatim into the issue body. This violates the "minimum data shared" directive — config.yaml routinely contains team-server tokens, workspace IDs, allowlists, and environment-specific settings whose values aren't needed for diagnosing most bugs but whose presence is. (example) + +Phase 1 changes Step 2's instruction so the default included shape is the YAML's *key structure* (top-level keys, no values), with an explicit opt-in toggle exposed in Step 3.5's transparency preview that lets the user elect to include the verbatim contents when the bug genuinely needs them (e.g., a parser bug in config loading). + +### Affected Files + +- `skills/bicameral-report-bug/SKILL.md` — three edits, all on the existing skill markdown: + 1. **Step 2 §"`.bicameral/config.yaml`"** instruction: replace the "use Read on .bicameral/config.yaml" line with the keys-only extraction rule (Read the file, parse the YAML or scan top-level non-indented keys, emit only the key list as a sorted bulleted block). (example) + 2. **Step 3 body-assembly template**: replace the verbatim ```yaml ``` block in the assembled markdown with the keys-only block. Add a sentinel-line `(values redacted by default — opt in via Step 3.5 to include verbatim)`. + 3. **Step 3.5 transparency preview**: add the `.bicameral/config.yaml` toggle to the operator-facing question. The toggle defaults to "keys only" and the operator can flip it to "include verbatim" if the bug needs the values. Update the redaction-summary block to print which mode was chosen (keys-only / verbatim) so the operator sees the choice in the preview. (example) + +### Changes + +**Step 2 §config.yaml** — replace the existing line with: + +```markdown +- **`.bicameral/config.yaml`**: use `Read` on `.bicameral/config.yaml` if it exists. Extract ONLY the top-level key structure by default — every top-level YAML key (lines that don't start with whitespace), one per line, sorted alphabetically. Do NOT include values, nested keys, comments, or any other content. The keys-only shape is sufficient diagnostic signal for *"is this bug in the config loader?"* questions while leaking zero workspace IDs, tokens, or environment-specific settings. If the operator's bug genuinely needs the verbatim contents (e.g. a YAML parser regression), Step 3.5's transparency preview offers an explicit opt-in toggle. If `Read` errors (file missing), skip the section entirely. (example) +``` + +**Step 3 body-assembly template** — replace the existing ```yaml ``` block with the keys-only shape: + +```markdown + ## .bicameral/config.yaml ← only if Read succeeded + + ``` + + ``` + *(values redacted by default — opt in via Step 3.5 transparency preview to include verbatim)* + ``` +``` + +**Step 3.5 transparency preview** — extend the operator-facing question to include a config.yaml verbosity toggle. The exact `AskUserQuestion` shape: + +```python +AskUserQuestion({ + questions: [{ + question: "Open the prefilled GitHub issue?", + header: "Open issue", + multiSelect: false, + options: [ + { label: "Yes, open it (config.yaml: keys only)", + description: "Default — config.yaml top-level keys included, values redacted" }, + { label: "Yes, but include config.yaml verbatim", + description: "Use only when the bug requires inspecting config values; values still pass the secret-redaction regex but workspace IDs / tokens / allowlists are exposed" }, + { label: "Edit the body first", + description: "I want to revise the body in chat before opening" }, + { label: "Cancel", + description: "Don't open anything; nothing leaves the machine" } + ] + }] +}) +``` + +The redaction-summary block printed before the question is updated to include the explicit current choice: + +``` +Auto-redacted in this body: + - ...existing redactions... + - .bicameral/config.yaml: keys only by default (toggle below to include verbatim) +``` + +When the operator picks "Yes, but include config.yaml verbatim", regenerate the body with the verbatim ```yaml ``` block (the existing pre-#200 shape, kept available for the opt-in path). Re-display the preview with verbatim contents and re-ask the open-issue question one more time so the operator sees what's actually being shipped before clicking through. The secret-redaction regex (`(api[_-]?key|token|secret|password|bearer)\s*[=:]\s*\S+`) still runs on verbatim contents — this is defense-in-depth, not a substitute for the keys-only default. + +### Unit Tests + +None for this phase. Per `doctrine-test-functionality` and the established precedent across plan-156 PR A Phase 2, plan-156b Phase 1, plan-187 Phase 2, plan-197 Phase 1: skill markdown is LLM-consumed agent-instruction, not pytest-invocable. The functional validation is the operator running `/bicameral-report-bug` against a real bug; correctness of the keys-only extraction, the toggle UX, and the regenerate-on-opt-in flow is observed at the runtime preview, not in unit tests. + +A static lint asserting the new instruction text contains the keys-only phrase would be presence-only by construction (it doesn't validate the LLM's behavior on the new instruction). Per the doctrine, presence-only assertions don't satisfy the test functionality bar. + +## CI Commands + +- `python scripts/lint_plan_grounding.py plan-200-config-yaml-redaction.md` — runs the plan-grounding lint shipped in PR #121 against this plan to catch any backtick-wrapped path tokens that don't resolve on the working tree. +- `grep -n "" skills/bicameral-report-bug/SKILL.md` — sanity check that the verbatim-by-default `` placeholder is gone from the default body assembly. Expected after Phase 1: matches only inside the documented opt-in path (Step 3.5's "include verbatim" branch). +- `grep -nE "config\.yaml.*verbatim|keys only" skills/bicameral-report-bug/SKILL.md` — sanity check the new instruction text and toggle copy are both present. Expected: ≥3 matches (Step 2 instruction, Step 3 template note, Step 3.5 toggle). diff --git a/skills/bicameral-report-bug/SKILL.md b/skills/bicameral-report-bug/SKILL.md index d3d6cbbf..18ddd7db 100644 --- a/skills/bicameral-report-bug/SKILL.md +++ b/skills/bicameral-report-bug/SKILL.md @@ -80,7 +80,17 @@ Sources (in order of preference, all read-only / non-bash): they leak business context. Record only the shape: `branch: ` and `recent commits: titles redacted`. - **`.bicameral/config.yaml`**: use `Read` on `.bicameral/config.yaml` - if it exists. If `Read` errors (file missing), skip the section. + if it exists. **Extract ONLY the top-level key structure by default** — + every YAML key whose line doesn't start with whitespace, one per + line, sorted alphabetically. Do NOT include values, nested keys, + comments, or any other content. Default-shape rationale: top-level + keys are sufficient diagnostic signal for *"is this bug in the + config loader?"* questions while leaking zero workspace IDs, tokens, + allowlists, or environment-specific settings. If the operator's bug + genuinely needs the verbatim contents (e.g. a YAML parser regression), + Step 3.5's transparency preview offers an explicit opt-in toggle — + see "Step 3.5 — Transparency preview" below. If `Read` errors (file + missing), skip the section entirely. Then assemble in your head (do NOT print to user yet): @@ -119,9 +129,10 @@ Then assemble in your head (do NOT print to user yet): ## .bicameral/config.yaml ← only if Read succeeded - ```yaml - ``` + + ``` + *(values redacted by default — opt in via Step 3.5 transparency preview to include verbatim)* ``` --- @@ -186,6 +197,9 @@ Auto-redacted in this body: topic, intent, description, text, excerpt, title — parameter names kept, values replaced with ) - Branch name and commit subject lines in the Repo state block + - .bicameral/config.yaml: keys only by default (workspace IDs, tokens, + allowlists, env-specific values stripped — toggle below to include verbatim + if the bug requires inspecting config values) - API keys, bearer tokens, secrets, passwords (regex-matched) - redaction(s) applied ← print actual count, or "none detected" @@ -209,8 +223,10 @@ AskUserQuestion({ header: "Open issue", multiSelect: false, options: [ - { label: "Yes, open it", - description: "Browser opens to a GitHub draft — you review and submit there" }, + { label: "Yes, open it (config.yaml: keys only)", + description: "Default — config.yaml top-level keys included, values redacted" }, + { label: "Yes, but include config.yaml verbatim", + description: "Use only when the bug requires inspecting config values (e.g. YAML parser regression). Values still pass the secret-redaction regex but workspace IDs / allowlists / env settings are exposed." }, { label: "Edit the body first", description: "I want to revise the body in chat before opening" }, { label: "Cancel", @@ -220,7 +236,8 @@ AskUserQuestion({ }) ``` -- **Yes** → proceed to Step 4. +- **Yes, open it (config.yaml: keys only)** → proceed to Step 4 with the keys-only body as already previewed. +- **Yes, but include config.yaml verbatim** → regenerate the body, replacing the keys-only block in the `## .bicameral/config.yaml` section with the verbatim ```yaml ``` shape. Re-run the Auto-redacted summary on the new body (the secret-redaction regex still applies — defense-in-depth, not a substitute for the keys-only default). Re-display the transparency preview with the verbatim contents and re-ask the open-issue question one more time so the operator sees what's actually being shipped before clicking through. Then proceed to Step 4 once the operator confirms the verbatim shape. - **Edit the body first** → ask the user what to change, regenerate the body, return to this step. - **Cancel** → stop. Tell the user "Cancelled. Nothing was sent." and