feat(capture): graph-safe global capture for the filesystem consumer - #195
Merged
Conversation
…/dot-list; default positions all_prompt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Gives
FilesystemConsumera workingglobal_capture_spec(). When capture hooks/positions are configured at the consumer level (global_hooks,global_positions,default_tagparams), every request is captured uniformly via the CUDA-graph-safe persistent-buffer path, and per-request output files are still written (keyed by the engine request id + the configured tag).The engine request id was already threaded through the global dispatch path, so no manager plumbing was needed — only the consumer-side spec + naming fallback.
Why
Global capture specs ride a graph-baked persistent-buffer copy and never force eager. The filesystem consumer, being per-request-only, always took the dynamic-gather path that forces a captured decode step out of the CUDA graph (~4x/step; ~+285% at
all_generated). For workloads that capture the same layers for all traffic (dataset/reward collection), routing the filesystem consumer through the global path removes that penalty.Result (Qwen3-0.6B, bs16, cudagraph):
all_generatedcapture overhead +287% → +11%.Limitations
SamplingParams.capturestill overrides per request.per_file(default) is the intended layout for global capture.Sibling PRs
Part of a set addressing capture's force-eager-under-cudagraph cost from different angles (graph-safe per-request allowlist; piecewise-eager fallback). These touch overlapping capture files (manager/step_gate/runner/config) and will conflict if merged together — resolve at merge.