Repository navigation
Worker processing comm - #3
Merged
Merged
Conversation
…pending from dict to list[EventAndPointers], added request_id to EventAndPointers, made NameAndRequestId frozen/hashable, guarded mooncake import
…t when waiting is None
…f.subgraph.section) instead of deepcopy(self.subgraph) (Subgraph vs GraphSection); (2) SubgraphQueues.is_done: now checks waiting is None AND ready is empty (was prematurely marking subgraphs complete); (3) SubgraphsManager.process_stage_outputs: skip back_to_conductor pointers and unknown stages when routing to workers
…er, and MooncakeCommunicationManager
…n loop, SubgraphsManager integration, MicroScheduler, and prefill graph
…ll/decode, pause/resume
…approach would be to raise an error or log a warning instead of silently skipping
Collaborator
|
What's the distinction of engine vs worker? |
shg8
added a commit
that referenced
this pull request
May 5, 2026
PR #78 review issue #3. _apply_pending_stops_to_batch built the "loop-back inputs to drop from this iter's routing" set as every self-loop edge on the running node: stopped_loop_backs[rid] = { edge.name for edge in node.outputs if edge.next_node == node.name } That works for in-tree models (each AR node owns exactly one decode loop, so {self-loop edges} == {this loop's loop-back edges}), but contradicts the docstring's "the stopped loop's loop-back" promise. A node that participates in two distinct loops with disjoint loop- back edges would lose the SURVIVING loop's loop-back tensor when the OTHER loop stopped, since the broad self-loop filter doesn't know which edge belongs to which loop. Fix: thread per-loop attribution through stop_loops. - WorkerGraphQueues.stop_loops now returns the (edge.name, edge.next_node) pairs of the loop-back signals belonging to the loops that actually matched loop_names. The walker already finds matching DynamicLoops to call register_finished on; collect their _loop_back_signals at the same point. - WorkerGraphsManager.stop_loops aggregates and returns the union across worker graphs. Existing void callers (_drain_orphan_pending_stops) ignore the return — backward- compatible. - _apply_pending_stops_to_batch intersects node.outputs against the returned set: now drops only edges that are BOTH self-loops AND loop-back of a stopped loop. The self-loop guard preserves the existing consumer's filter shape (kept_for_routing only drops e.next_node == node.name && e.name in consumed_names). No in-tree model has the multi-loop-per-node shape today, so the visible behavior on Orpheus / BAGEL / Q3-Omni is unchanged. The fix tightens the docstring contract to match the code so the next model with that shape doesn't silently drop loop-back tensors. test/modular/test_stop_loops_filter.py drives WorkerGraphQueues .stop_loops directly with a hand-built section to verify the attribution. Five cases: - returns only stopped loops' loop-back signals - returns union when multiple loops stopped - empty when no loops match - register_finished only fires on matched loops - shared-node-name case from the PR comment: two loops, same GraphNode name "n", disjoint self-loop edges {a} and {b}; stopping loop_a returns {("a","n")} and never {("b","n")}
shg8
added a commit
that referenced
this pull request
May 5, 2026
PR #78 review issue #3. _apply_pending_stops_to_batch built the "loop-back inputs to drop from this iter's routing" set as every self-loop edge on the running node: stopped_loop_backs[rid] = { edge.name for edge in node.outputs if edge.next_node == node.name } That works for in-tree models (each AR node owns exactly one decode loop, so {self-loop edges} == {this loop's loop-back edges}), but contradicts the docstring's "the stopped loop's loop-back" promise. A node that participates in two distinct loops with disjoint loop- back edges would lose the SURVIVING loop's loop-back tensor when the OTHER loop stopped, since the broad self-loop filter doesn't know which edge belongs to which loop. Fix: thread per-loop attribution through stop_loops. - WorkerGraphQueues.stop_loops now returns the (edge.name, edge.next_node) pairs of the loop-back signals belonging to the loops that actually matched loop_names. The walker already finds matching DynamicLoops to call register_finished on; collect their _loop_back_signals at the same point. - WorkerGraphsManager.stop_loops aggregates and returns the union across worker graphs. Existing void callers (_drain_orphan_pending_stops) ignore the return — backward- compatible. - _apply_pending_stops_to_batch intersects node.outputs against the returned set: now drops only edges that are BOTH self-loops AND loop-back of a stopped loop. The self-loop guard preserves the existing consumer's filter shape (kept_for_routing only drops e.next_node == node.name && e.name in consumed_names). No in-tree model has the multi-loop-per-node shape today, so the visible behavior on Orpheus / BAGEL / Q3-Omni is unchanged. The fix tightens the docstring contract to match the code so the next model with that shape doesn't silently drop loop-back tensors. test/modular/test_stop_loops_filter.py drives WorkerGraphQueues .stop_loops directly with a hand-built section to verify the attribution. Five cases: - returns only stopped loops' loop-back signals - returns union when multiple loops stopped - empty when no loops match - register_finished only fires on matched loops - shared-node-name case from the PR comment: two loops, same GraphNode name "n", disjoint self-loop edges {a} and {b}; stopping loop_a returns {("a","n")} and never {("b","n")}
4 tasks
Merged
1 task done
NSagan271
added a commit
that referenced
this pull request
Sep 29, 2026
…ontract (#324) * Move worker graph state behind a GraphRuntime contract The worker's graph-shaped state and logic move behind an abstract GraphRuntime (mstar/graph/runtime/base.py). PythonGraphRuntime (runtime/python.py) implements it with the same logic, moved rather than rewritten. That state is the per-request worker graph queues, routing, per-request sharding, loop stops, speculation prep, and the request id <-> handle table. - WorkerGraphsManager becomes RequestStateManager. It keeps only what cannot live behind the contract: forward-pass info, a wire object the runtime forwards opaquely, and stream buffers, which hold tensors. - Ingest, pop and speculation prep exchange uuids (EdgeSpec) rather than GraphNode objects. Completion is one batched complete_and_route_batch followed by send_outputs, which builds and sends INPUT_SIGNALS, RESULT_TENSORS and WORKER_GRAPHS_DONE. - The micro-scheduler asks the runtime for ready nodes and pops through it. ScheduledBatch carries uuid inputs and the node's output signals instead of node objects. - Loop stop times move off CurrentForwardPassInfo into the runtime, since only the worker ever read them. Peers now learn a stop only from STOP_LOOPS, not also from the fwd_info riding on INPUT_SIGNALS. This is a move: no fixes to pre-existing behavior. Two regressions the move itself would otherwise introduce are kept at main's behavior, with tests: - Output refcounts still settle through the tensor manager, so an output routed to no consumer is freed at routing, not at request teardown. - A new-token signal routed to two destinations is still counted once. Also retypes PartitionState.wg_rank_completions to int keys, which the worker-graph id change missed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * register by uuid instead of by index to also register past tensors from loop outputs / acc outputs * respond to PR comments --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2 tasks done
dghosef
added a commit
that referenced
this pull request
Nov 2, 2026
Reports appended to persist_signals, so a name persisted while an earlier value was still held (held from the start, or persisted by an earlier step that nothing read in between) reached its reader as both tensors: Whisper's ts_rules with the language detected. A report now replaces what another pass left held under the name, releasing the sent tensors at their reader counts. Reports of the same pass (TP ranks, several worker graphs) still accumulate, keyed on the (partition, pass) that wrote the name, kept on RequestData. Review findings: r1-code-review #3, r1-reviewer P3-5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dghosef
added a commit
that referenced
this pull request
Nov 2, 2026
) `SeqShardSpec.derive` filtered its node's specs by `attends_split_steps`, then type-switched to ask a paged attention `resolve_exchange(specs)` and a ragged one `.sp_exchange`: a hook in disguise. `resolve_exchange(specs)` is now the one `NodeResourceSpec` hook (default "none"; the paged spec resolves its `sp_exchange` against its pool's storage, the ragged self-attention returns its declared one), and the derive asks every spec over its nodes. Deletes `attends_split_steps`. Also deletes the base `splits_steps` hook, which only `SeqShardSpec` answered (with its `splits` field) and only the engine still read; the engine now reads `isinstance(spec, SeqShardSpec) and spec.splits`, as the rest of the shard code does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Here's a summary of everything implemented:
Files Created
Files Modified
mooncake import
- SubgraphQueues.add_request/reset: used deepcopy(self.subgraph.section) instead of deepcopy(self.subgraph) (Subgraph vs GraphSection)
- SubgraphQueues.is_done: now checks waiting is None AND ready is empty (was prematurely marking subgraphs complete)
- SubgraphsManager.process_stage_outputs: skip back_to_conductor pointers and unknown stages when routing to workers