Skip to content

chore: /orchestrate asks instead of assuming, and manual steps come from the work - #724

Merged
thomasluizon merged 2 commits into
mainfrom
chore/orchestrate-asks-not-assumes
Aug 14, 2026
Merged

chore: /orchestrate asks instead of assuming, and manual steps come from the work#724
thomasluizon merged 2 commits into
mainfrom
chore/orchestrate-asks-not-assumes

Conversation

@thomasluizon

@thomasluizon thomasluizon commented Aug 14, 2026

Copy link
Copy Markdown
Owner

Two behaviour changes Thomas asked for on 2026-08-13, plus the tool gap they exposed.

Ask when unsure, at any step, in any mode

The worker prompt's "choose the reading a careful colleague would" made silent assumptions the instructed behaviour, and asking was wired only into the step 2b gate and the needs:conversation label. Now:

  • The composed prompt splits ambiguity into two tiers. Mechanical choices land in a PR-body "Assumptions" section with the rejected alternative named. A decision that belongs to Thomas (product, brand, copy, price, design, a tool contradiction, a missing dependency) ends the worker with "NEEDS_DECISION: " instead of a guess.
  • Step 7 reads the worker log tail for NEEDS_DECISION on every non-DELIVERED verdict. Attended runs ask Thomas, comment the answer on the ticket, and relaunch with the new work order; --sleep runs defer the ticket with the question in the step 11 decision list.
  • Step 2b now states that the classifier and the label are detection aids, not the boundary of asking: an attended run asks any question at the moment it appears, one topic at a time.
  • Step 10 adjudicates the PR body's Assumptions with Thomas before READY; step 11 gains one consolidated decision list.

Manual steps from three sources, not one ticket section

Step 10 merges: the ticket's own sections (complete-ticket.mjs --preflight), the PR body's "Manual steps" section the prompt now requires of every worker, and the orchestrator's own diff read for new env keys, secrets, flags and vendor-console dependencies. The merged list is commented on the ticket so it survives scrollback.

tools/label-ticket.mjs

New tool, the only sanctioned label mutation on an existing ticket, validated against the live label list like creation. Measured the same day: needs:conversation could not be applied to #36 through any tool, and the raw-mutation hook correctly blocked gh issue edit.

Both harnesses pass: node tools/test-tools.mjs (ORBIT TOOLS GATE OK) and node .claude/hooks/test-hooks.mjs (ORBIT HOOKS OK).

🤖 Generated with Claude Code

External-interface evidence (code standard 8)

The gh issue edit --add-label / --remove-label contract is proven by execution against gh 2.97.0 (2026-07-31), three real invocations on orbit-tickets#316 on 2026-08-13, each followed by a label read:

  1. --add-label harness: result set repo:ui, Improvement, parity:no, harness.
  2. --add-label needs:no-conversation --remove-label harness in one call: result set repo:ui, Improvement, parity:no, needs:no-conversation.
  3. --add-label harness: result set repo:ui, Improvement, parity:no, harness, needs:no-conversation.

Exit 0 on each. The same record lives in the editLabels comment in tools/lib/github-issues.mjs.

…rom the work

Two behaviour changes Thomas asked for on 2026-08-13, plus the tool gap they exposed.

Ask when unsure, at any step, in any mode. The worker prompt's "choose the reading a
careful colleague would" made silent assumptions the instructed behaviour. Now ambiguity
has two tiers: mechanical choices land in a PR-body ## Assumptions section the
orchestrator adjudicates with Thomas at handover, and a decision that belongs to Thomas
ends the worker with NEEDS_DECISION: <question> instead of a guess. Step 7 reads it from
the worker log; attended runs ask and relaunch with the answer commented on the ticket,
sleep runs defer with the question in the report. Step 2b states that the classifier and
the needs:conversation label are detection aids, not the boundary of asking.

Manual steps from three sources, not one ticket section. Step 10 merges the ticket's own
sections (complete-ticket.mjs --preflight), the PR body's ## Manual steps section the
prompt now requires, and the orchestrator's own diff read for new env keys, secrets and
vendor-console dependencies, then comments the merged list on the ticket.

tools/label-ticket.mjs is new: the only sanctioned label mutation on an existing ticket,
validated against the live label list. Measured the same day: needs:conversation could
not be applied to #36 through any tool, and the raw-mutation hook correctly blocked gh.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
orbit-ui-mobile-web Ignored Ignored Aug 14, 2026 2:34am

Request Review

@thomasluizon

Copy link
Copy Markdown
Owner Author

@pullfrog review

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The decision channel and readiness ordering still allow unanswered questions to reach a ready handoff, and the new external CLI contract needs the repository-required evidence before merge.

Reviewed changes I reviewed the full initial diff at 5d8074a, covering the orchestration contract and ticket-label tooling.

  • Ambiguity handoff: The worker prompt now separates mechanical assumptions from Thomas-owned decisions, and Step 7 adds attended and sleeping-run recovery behavior.
  • Manual operations: Step 10 now combines ticket, PR-body, and diff-derived manual steps and carries unresolved assumptions into handoff reporting.
  • Ticket labels: A validated label-ticket.mjs command and editLabels adapter add and remove existing ticket labels.
  • Harness coverage: Prompt assertions, adapter export coverage, label mutation cases, tool registration, and documentation were updated for the new surfaces.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using GPT Sol𝕏

Comment thread .claude/skills/orchestrate/SKILL.md Outdated
Comment thread .claude/skills/orchestrate/SKILL.md Outdated
Comment thread tools/lib/github-issues.mjs

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The new decision channel can bypass its own question on an otherwise delivered branch, and the label mutation still lacks the required proof of its external gh contract.

Reviewed changes I reviewed the orchestration prompt and lifecycle updates, structured handoff sections, and the new existing-ticket label mutation with its harness coverage.

  • Decision handling Splits ambiguity into recorded mechanical assumptions and human-owned NEEDS_DECISION questions.
  • Structured handoff Collects manual steps from the ticket, PR body, and diff while adjudicating PR assumptions before readiness.
  • Ticket labels Adds validated add/remove mutations for existing tickets through the shared GitHub adapter.
  • Harness coverage Extends prompt, adapter-surface, tool-registration, and label-command cases.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using GPT Sol𝕏

Comment thread .claude/skills/orchestrate/SKILL.md Outdated
Comment thread tools/lib/github-issues.mjs
Comment thread tools/__tests__/label-ticket.mjs Outdated
The NEEDS_DECISION and ## Assumptions read now happens at EVERY worker exit, DELIVERED
included, because verify-delivery.mjs reads artifacts and never the log, so a delivered
branch proves nothing about an unanswered question. A delivered PR with an open question
keeps its ticket In Progress under --sleep, never In Review. Assumptions are adjudicated
at the step 7 worker-exit read, before the step 8 and 9 loops, so an invalidating answer
is ordinary bounded-fixer work rather than a revoked READY; step 10 only prints outcomes.

The gh issue edit --add-label/--remove-label contract is now proven by execution per code
standard 8: three real invocations against gh 2.97.0 on orbit-tickets#316 (single add,
combined add plus remove in one call, re-add), each confirmed by the follow-up label
read; the evidence lives in the editLabels comment. The label-ticket success cases now
consume a removePath marker on the stubbed edit command, so deleting the editLabels write
fails the test instead of passing on a preloaded issue view.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes I reviewed the fd9711c follow-up, which addressed the decision-ordering, mutation-test, and external-interface evidence findings from the prior Pullfrog review.

  • Decision precedence Made NEEDS_DECISION and PR-body assumption inspection unconditional at every worker exit and moved attended adjudication ahead of review and readiness.
  • Sleeping-run handoff Kept delivered PRs with unanswered human decisions In Progress and carried open decisions into consolidated reporting.
  • Mutation proof Added side-effect markers showing both label add and remove commands reached gh issue edit.
  • External contract evidence Documented real gh 2.97.0 add, remove, and combined invocations with observed successful exits and read-back label sets.

Pullfrog  | View workflow run | Using GPT Sol𝕏

@thomasluizon
thomasluizon merged commit d999d0e into main Aug 14, 2026
33 checks passed
@thomasluizon
thomasluizon deleted the chore/orchestrate-asks-not-assumes branch August 14, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant