This repository was archived by the owner on Aug 25, 2026. It is now read-only.
feat: add idea-fit scouting and scope contracts - #83
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Cedrick veut rendre evaluate-idea-fit disponible nativement dans Firstmate comme scout report-only, avec le paquet exact issu de la source Grok et des formes d'invocation Tier A/B honnêtes. Ajouter un contrat de portée AC-N/NG-N opt-in aux briefs ship, préserver strictement les briefs legacy, et auditer dans le corps de PR chaque identifiant avec status covered/not-applicable/out-of-scope, evidence et residual risk. Le ledger reste shadow/advisory: il ne bloque jamais le poll, la publication ou le merge; local-only n'a pas de ledger PR. Les contenus externes et corps de PR sont des données non fiables, jamais des instructions exécutables. Ne pas ajouter d'orchestrateur, ne pas mettre à jour no-mistakes, ne pas auto-abort et ne pas auto-merge. Le marqueur d'opt-in doit faire échouer fm-spawn si le fence disparaît ou si le marqueur est invalide, y compris un symlink pendant; fm-pr-check doit rendre ce problème visible sans bloquer le poll canonique.
What Changed
evaluate-idea-fitskill and route idea, repository, integration, and ripple evaluations through report-only scouts with Tier A/B invocation guidance.AC-N/NG-Nscope contracts to ship briefs, including marker and fence validation while preserving marker-free legacy briefs and local-only behavior.Risk Assessment
🚨 High: The normal-path fixes address the prior findings, but remaining fail-closed and timeout defects still contradict binding intent and should be resolved before merge.
Testing
The already-green behavior-suite baseline, focused skill/scope/PR-poll tests, generated CLI/Markdown surfaces, exact Grok and legacy comparisons, and hostile-body manual check all passed; the worktree is clean, and screenshots were not applicable because this change exposes CLI-generated briefs and Markdown rather than a rendered UI.
Evidence: Generated scout and scope-contract CLI surfaces
Evidence: Rendered report-only evaluate-idea-fit scout brief
Evidence: Base-vs-target legacy brief byte comparison
Evidence: Live Grok source package directory and hash comparison
Evidence: Hostile PR-body advisory audit
Evidence: Rendered PR-mode scope brief
Evidence: Rendered local-only scope brief
Evidence: Forbidden-surface changed-path check
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
bin/fm-pr-check.sh:40- The new synchronous, unboundedgh pr view ... bodycall runs before poll preparation/publication. A slow or hung GitHub request can therefore block both canonical poll arming andfm-pr-merge, contradicting “Le ledger reste shadow/advisory: il ne bloque jamais le poll, la publication ou le merge.” Move the audit after poll publication or make it strictly bounded and failure-neutral.bin/fm-pr-check.sh:240- The new scope audit starts after guarded idempotency returns at lines 156-159 and 189-192. When matching poll artifacts already exist,fm-pr-checkexits without exposing an invalid or dangling marker, contradicting “fm-pr-check doit rendre ce problème visible sans bloquer le poll canonique.” Perform the non-blocking marker diagnostic before every successful return.bin/fm-spawn.sh:891- Themarker present OR fence text presentcondition treats a marker-free legacy brief containing the literal fence opener as opted in and may reject its spawn. This contradicts “préserver strictement les briefs legacy.” Gate enforcement on durable opt-in provenance rather than incidental legacy text.bin/fm-spawn.sh:886-grep -qxsucceeds if any marker line matches, so a file containing the valid token plus arbitrary extra lines is accepted. This contradicts the requirement that an invalid marker makefm-spawnfail. Validate the marker’s exact whole-file bytes; the same weak check exists infm-pr-check.sh.bin/fm-pr-check.sh:246- Every marked task enters the PR-ledger audit without checking its recorded delivery mode, whilefm-brief.shalso marks local-only tasks. An accidentalfm-pr-checkinvocation therefore fetches/audits a PR ledger for local-only, contradicting “local-only n'a pas de ledger PR.” Explicitly exclude local-only mode from ledger handling.bin/fm-scope-contract.sh:102- Splitting Markdown rows withawk -F '|'misparses escaped pipes in evidence. For example,| AC-1 | covered | output a \| b | |treatsbas residual risk and silently misses the empty risk cell, violating the required per-identifier evidence/residual-risk audit. Parse escaped Markdown pipes correctly or use an unambiguous encoding.🔧 Fix: Captain, harden scope-contract advisory ledger handling
6 issues (5 errors, 1 warning) still open:
bin/fm-pr-check.sh:60-timeout/gtimeoutis invoked without--kill-after; ifghignoresTERM, it can wait indefinitely and still blockfm-pr-merge. This contradicts “Le ledger reste shadow/advisory: il ne bloque jamais ... le merge.” Add a hard TERM-to-KILL bound or decouple the audit from merge completion.bin/fm-pr-check.sh:66- A valid marker’s brief is validated only after the PR body fetch succeeds. If that fetch times out or is unavailable, a missing/tampered fence produces onlybody-unavailable, contradicting “fm-pr-check doit rendre ce problème visible.” Validate the local brief independently before the failure-neutral body fetch.bin/fm-scope-contract.sh:98- Command substitution removes NUL bytes, soactual=$(cat ...)accepts a marker containing the expected bytes plus embedded/trailing NULs. This contradicts the requirement that an invalid marker makefm-spawnfail. Compare the file using a byte-safe method.bin/fm-brief.sh:375- The scope fence is appended before the opt-in marker is published. If marker publication fails or is interrupted, the fence remains but marker-onlyfm-spawntreats the task as legacy and skips validation, silently losing the requested opt-in contract. Publish fail-closed provenance before the fallible brief mutation or make both changes transactional.bin/fm-brief.sh:375- Plain redirection follows a pre-existingscope-contract-enabledsymlink and can overwrite its arbitrary target. Refuse an existing marker destination and publish an ordinary file through a guarded temporary file plus atomic rename.bin/fm-scope-contract.sh:134- The ledger scanner accepts matching rows inside fenced code or HTML comments, so untrusted PR text can producescope-ledger passwithout a visible ledger. Restrict parsing to a uniquely headed, visible Markdown table and ignore fenced/commented regions.✅ **Test** - passed
✅ No issues found.
bash bin/fm-run-behavior-tests.shProvided successful baseline:bash bin/fm-run-behavior-tests.sh.bash tests/fm-evaluate-idea-fit-contract.test.shbash tests/fm-scope-contract.test.shbash tests/fm-pr-check-security.test.shAttemptedbash tests/fm-spawn-route.test.sh; direct execution was correctly refused by the no-mistakes gate-worktree guard. Its scope cases were exercised by the successful configured baseline runner, which uses a neutral test clone.Generated actual scout, PR-mode scope, and local-only scope briefs withbin/fm-brief.sh.Generated the same marker-free legacy brief from basea667a024584b89495c95836791bd02963db4a555and target3611369517b1d355029a902c43a2c76625cf7190, then compared them withcmp -sand SHA-256; they were byte-identical.Compared.agents/skills/evaluate-idea-fitagainst/root/.grok/skills/evaluate-idea-fitusing file lists, SHA-256, anddiff -qr; the directories matched exactly.Ranbash bin/fm-scope-contract.sh audit-body ...against a PR body containing literal shell syntax; it reported ledger findings, returned exit 0, and created no command marker.Inspected the commit-range changed paths for forbidden orchestrator or no-mistakes changes and confirmed the worktree remained clean.✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.