docs(directive-detection): iteration spec for #374 H1 gate - #466
Conversation
Successor memo to docs/v2_enforcement.md § H1. Does not re-decide the deferral or the gate; documents three concrete iteration paths to raise detector precision from the measured 0.664 to the >=0.80 floor while keeping recall above 0.60. Recommends Path A (intent-prefix filter on imperative coding-task verbs at the sentence head). Path B (deontic-anchor partition) and Path C (LLM classifier) are listed with tradeoffs; Path C is flagged non-deterministic and out of scope without an explicit ratification. Memo also defines the public-tree vs lab-tree work split so #374 stops bouncing in the work queue: detector source + public-CI sanity tests ship from this repo; corpus authoring and the bench-gate run stay lab-side per directory-of-origin rules. Closing #374 still requires a lab-side run demonstrating P>=0.80 ^ R>=0.60 ^ n>=200. Refs #374, #199 (umbrella), PR #377 (harness).
Reviewer's GuideAdds a new directive-detection iteration spec memo and wires it into the v2 re-entry queue, defining concrete iteration paths (and a recommended one) for clearing the H1 directive-detection gate without changing any code or tests. File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThis PR documents the directive detection enforcement H1 feature as deferred with bench-gate criteria, adding a re-entry queue row and a detailed iteration specification memo covering measured performance, identified failure modes, and three candidate implementation paths. ChangesDirective Detection H1 Re-entry Documentation
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- Consider factoring the intent-prefix verbs and deontic/rule-marker lists into a single normative source (or explicitly referencing where they will live in code) so the eventual
directive_detector.pyimplementation doesn’t silently diverge from the spec. - The decision asks around head-position anchors and connective lists are quite detailed; you might tighten them into a short, definitive table or checklist so future implementers can see the exact default configuration at a glance without rereading the narrative.
- Since Path C is explicitly out of scope for now due to determinism, you could add a short note on what conditions would justify revisiting it, to avoid future ambiguity about when it becomes a realistic option.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Consider factoring the intent-prefix verbs and deontic/rule-marker lists into a single normative source (or explicitly referencing where they will live in code) so the eventual `directive_detector.py` implementation doesn’t silently diverge from the spec.
- The decision asks around head-position anchors and connective lists are quite detailed; you might tighten them into a short, definitive table or checklist so future implementers can see the exact default configuration at a glance without rereading the narrative.
- Since Path C is explicitly out of scope for now due to determinism, you could add a short note on what conditions would justify revisiting it, to avoid future ambiguity about when it becomes a realistic option.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
[claim:review:Gylf:2026-05-07T18:46:24Z] |
|
[claim:review:Toug:2026-05-07T18:46:38Z] |
|
[claim:review:Kulili:2026-05-07T18:46:41Z] |
|
[release:review:Toug:2026-05-07T18:46:43Z] |
|
[release:review:Kulili:2026-05-07T18:46:46Z] |
|
[release:review:Gylf:2026-05-07T18:47:59Z] |
What
Adds
docs/v2_directive_detection.md— an iteration spec for the deferred H1 directive-detection gate (issue #374). Cross-links fromdocs/V2_REENTRY_QUEUE.mdto the new memo.Does not touch
src/ortests/. Does not re-decide the deferral or the gate numbers (those stay owned bydocs/v2_enforcement.md§ H1). Does not claim H1 is unblocked.Why
Issue #374 has bounced ten times across sessions because the queue surface tells autonomous workers "deferred + bench-gated" without saying which detector iteration to attempt next. Lab corpus v0.1 (285 rows) has been measured against the candidate detector at P=0.664 / R=0.937 — under the 0.80 precision floor, with 23 percentage points of recall headroom. The iteration is a precision problem, not a corpus problem; the next session that lands here needs a chosen path, not a re-derivation.
Changes
docs/v2_directive_detection.md(new, 103 lines) — failure-mode analysis (the FP cluster is imperative-grammar one-shot coding tasks), three iteration paths (intent-prefix filter / deontic-anchor partition / LLM classifier) with tradeoffs, recommendation (Path A), decision asks, public-tree vs lab-tree work split, unchanged re-entry trigger.docs/V2_REENTRY_QUEUE.md— adds anIteration specline to the H1 row pointing at the new memo.Out of scope
feat(directive_detection):PR after the decision asks land.docs/v2_enforcement.md§ H1.Verification
git diff github/main...HEADclean against the discretion grep.docs(directive-detection):anddocs(reentry-queue):.Closes / refs
Refs #374, #199 (umbrella). Does not close #374 — that requires a passing bench-gate run, which this PR does not produce.
Summary by Sourcery
Add an iteration spec for the deferred directive-detection gate and cross-link it from the v2 re-entry queue.
Documentation:
Summary by CodeRabbit