chore: /orchestrate asks instead of assuming, and manual steps come from the work - #724
Conversation
…rom the work Two behaviour changes Thomas asked for on 2026-08-13, plus the tool gap they exposed. Ask when unsure, at any step, in any mode. The worker prompt's "choose the reading a careful colleague would" made silent assumptions the instructed behaviour. Now ambiguity has two tiers: mechanical choices land in a PR-body ## Assumptions section the orchestrator adjudicates with Thomas at handover, and a decision that belongs to Thomas ends the worker with NEEDS_DECISION: <question> instead of a guess. Step 7 reads it from the worker log; attended runs ask and relaunch with the answer commented on the ticket, sleep runs defer with the question in the report. Step 2b states that the classifier and the needs:conversation label are detection aids, not the boundary of asking. Manual steps from three sources, not one ticket section. Step 10 merges the ticket's own sections (complete-ticket.mjs --preflight), the PR body's ## Manual steps section the prompt now requires, and the orchestrator's own diff read for new env keys, secrets and vendor-console dependencies, then comments the merged list on the ticket. tools/label-ticket.mjs is new: the only sanctioned label mutation on an existing ticket, validated against the live label list. Measured the same day: needs:conversation could not be applied to #36 through any tool, and the raw-mutation hook correctly blocked gh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
|
@pullfrog review |
There was a problem hiding this comment.
Important
The decision channel and readiness ordering still allow unanswered questions to reach a ready handoff, and the new external CLI contract needs the repository-required evidence before merge.
Reviewed changes I reviewed the full initial diff at 5d8074a, covering the orchestration contract and ticket-label tooling.
- Ambiguity handoff: The worker prompt now separates mechanical assumptions from Thomas-owned decisions, and Step 7 adds attended and sleeping-run recovery behavior.
- Manual operations: Step 10 now combines ticket, PR-body, and diff-derived manual steps and carries unresolved assumptions into handoff reporting.
- Ticket labels: A validated
label-ticket.mjscommand andeditLabelsadapter add and remove existing ticket labels. - Harness coverage: Prompt assertions, adapter export coverage, label mutation cases, tool registration, and documentation were updated for the new surfaces.
GPT Sol | 𝕏
There was a problem hiding this comment.
Important
The new decision channel can bypass its own question on an otherwise delivered branch, and the label mutation still lacks the required proof of its external gh contract.
Reviewed changes I reviewed the orchestration prompt and lifecycle updates, structured handoff sections, and the new existing-ticket label mutation with its harness coverage.
- Decision handling Splits ambiguity into recorded mechanical assumptions and human-owned
NEEDS_DECISIONquestions. - Structured handoff Collects manual steps from the ticket, PR body, and diff while adjudicating PR assumptions before readiness.
- Ticket labels Adds validated add/remove mutations for existing tickets through the shared GitHub adapter.
- Harness coverage Extends prompt, adapter-surface, tool-registration, and label-command cases.
GPT Sol | 𝕏
The NEEDS_DECISION and ## Assumptions read now happens at EVERY worker exit, DELIVERED included, because verify-delivery.mjs reads artifacts and never the log, so a delivered branch proves nothing about an unanswered question. A delivered PR with an open question keeps its ticket In Progress under --sleep, never In Review. Assumptions are adjudicated at the step 7 worker-exit read, before the step 8 and 9 loops, so an invalidating answer is ordinary bounded-fixer work rather than a revoked READY; step 10 only prints outcomes. The gh issue edit --add-label/--remove-label contract is now proven by execution per code standard 8: three real invocations against gh 2.97.0 on orbit-tickets#316 (single add, combined add plus remove in one call, re-add), each confirmed by the follow-up label read; the evidence lives in the editLabels comment. The label-ticket success cases now consume a removePath marker on the stubbed edit command, so deleting the editLabels write fails the test instead of passing on a preloaded issue view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes I reviewed the fd9711c follow-up, which addressed the decision-ordering, mutation-test, and external-interface evidence findings from the prior Pullfrog review.
- Decision precedence Made
NEEDS_DECISIONand PR-body assumption inspection unconditional at every worker exit and moved attended adjudication ahead of review and readiness. - Sleeping-run handoff Kept delivered PRs with unanswered human decisions In Progress and carried open decisions into consolidated reporting.
- Mutation proof Added side-effect markers showing both label add and remove commands reached
gh issue edit. - External contract evidence Documented real
gh 2.97.0add, remove, and combined invocations with observed successful exits and read-back label sets.
GPT Sol | 𝕏




Two behaviour changes Thomas asked for on 2026-08-13, plus the tool gap they exposed.
Ask when unsure, at any step, in any mode
The worker prompt's "choose the reading a careful colleague would" made silent assumptions the instructed behaviour, and asking was wired only into the step 2b gate and the needs:conversation label. Now:
Manual steps from three sources, not one ticket section
Step 10 merges: the ticket's own sections (complete-ticket.mjs --preflight), the PR body's "Manual steps" section the prompt now requires of every worker, and the orchestrator's own diff read for new env keys, secrets, flags and vendor-console dependencies. The merged list is commented on the ticket so it survives scrollback.
tools/label-ticket.mjs
New tool, the only sanctioned label mutation on an existing ticket, validated against the live label list like creation. Measured the same day: needs:conversation could not be applied to #36 through any tool, and the raw-mutation hook correctly blocked gh issue edit.
Both harnesses pass: node tools/test-tools.mjs (ORBIT TOOLS GATE OK) and node .claude/hooks/test-hooks.mjs (ORBIT HOOKS OK).
🤖 Generated with Claude Code
External-interface evidence (code standard 8)
The
gh issue edit --add-label/--remove-labelcontract is proven by execution against gh 2.97.0 (2026-07-31), three real invocations on orbit-tickets#316 on 2026-08-13, each followed by a label read:--add-label harness: result setrepo:ui, Improvement, parity:no, harness.--add-label needs:no-conversation --remove-label harnessin one call: result setrepo:ui, Improvement, parity:no, needs:no-conversation.--add-label harness: result setrepo:ui, Improvement, parity:no, harness, needs:no-conversation.Exit 0 on each. The same record lives in the
editLabelscomment intools/lib/github-issues.mjs.