Skip to content

docs: refresh the READMEs against the work merged since May - #129

Merged
yeahdongcn merged 1 commit into
mainfrom
xd/readme-refresh-2026-10
Oct 10, 2026
Merged

yeahdongcn merged 1 commit into
mainfrom
xd/readme-refresh-2026-10

Conversation

@yeahdongcn

Copy link
Copy Markdown
Collaborator

The two READMEs have not kept up with the work merged since May. This PR
scans every merged PR from #61 (2026-05-07) to #128 (2026-10-09), documents
what is missing, and fixes claims that no longer match the repository. Both
languages are updated together.

Docs only: no source, test, or config change.

What was missing

Merged PR Now documented as
#61 CUDA memory-pool aliases "Memory management" row — torch.cuda.memory.* and the CUDA memory-pool APIs
#65 unsupported CUDA APIs → no-ops C++ extension symbol table — cudaGridDependencySynchronize() / cudaTriggerProgrammaticLaunchCompletion()
#72 graph executable rotation CUDA Graphs section plus the graph-rotation environment variables
#95 nested <torch/cuda.h> porting Building C++ Extensions
#97, #104 torch.accelerator.get_memory_info() Unified Accelerator API
#98 torch.cuda.streams "Synchronization" row and the compatibility table
#100 stable-ABI STABLE_TORCH_LIBRARY_IMPL rekeying and stream helpers Building C++ Extensions and the symbol table
#103 FA3 _flash_attn_forward FlashAttention Providers section
#106 graph-safe ops, Inductor heuristics, MUSA_VISIBLE_DEVICES torch.compile section and the compatibility table
#108 torch.cuda._get_device_index for TorchDynamo torch.compile section
#115 generated MoE tables Triton Fused MoE Tuning section
#121 include_paths / library_paths signatures Building C++ Extensions
#124 FX device builtin torch.compile section
#128 stale JIT build lock recovery Building C++ Extensions

Structural changes

  • torch_musa Compatibility (new section). Every version-gated shim was
    previously a paragraph-sized cell in "What Works". They now live in one table
    with the release each is installed on: the four < 2.11.0.post2 patches, the
    < 2.13.0 mm/bmm out_dtype= backport, asynchronous isfinite, the
    stable-ABI header backport, and torch.cuda.streams. _patch.py already
    refers to "this repository's compatibility table" — this is that table, and it
    is now readable as one.
  • Environment Variables (new section). Only two of the ten user-facing
    TORCHADA_* variables were documented; the graph-rotation knobs, the
    operator-override switches, and TORCHADA_PLATFORM were only discoverable by
    reading the source.
  • "What Works" goes back to one line per feature, so the table is scannable
    again.
  • The verbose out_dtype risk paragraph is kept verbatim, and the remaining
    "Not covered" scope note moved with it.

Stale claim corrected

The Performance section listed the 0.1.94 / torch_musa 2.7.1 medians
(130-155 ns) and stated that all frequently-called operations complete within
200 ns. The checked-in benchmarks/benchmark_history.json already has a newer
0.1.95 entry (2026-10-09, torch_musa 2.11.0.post2+musa5.2.0) measuring
170-260 ns for the same fast paths. The section now quotes the newest
measurement, keeps the 0.1.94 figures as the comparison, and no longer claims a
sub-200 ns bound the data does not support.

Notes

  • Relative links, anchors, table widths and codespell were checked; both files
    end with a newline and have no trailing whitespace.
  • torchada>=0.1.95 and the mapping-rule count (400+) still match
    pyproject.toml and src/torchada/_mappings/.
  • Everything added here was verified against the current tree rather than the PR
    descriptions; for example fix: generalize stable ABI extension porting for MUSA #100's description mentions CUDA_VERSION/USE_MUSA
    mappings that the merged commit does not contain, so only the rekeying and
    stream helpers it did merge are documented.

The English and Chinese READMEs have not kept up with the May-October work.
This documents what merged, moves the version-gated shims into one table, and
refreshes the measured numbers that had gone stale.

Feature table
- CUDA memory-pool APIs, `torch.cuda.streams`, CUDA-graph executable rotation,
  `torch.cuda._get_device_index`, `get_memory_info()`, and the FlashAttention
  provider shims (#61, #98, #103, #106, #108, #115)
- the "What Works" table goes back to one line per feature; the paragraph-sized
  `log_` / `isfinite` / `out_dtype` cells move into the new section below

New "torch_musa Compatibility" section
- one table of every version-gated shim with the release it is installed on:
  the four `< 2.11.0.post2` patches (#106, #113, #124), the `< 2.13.0`
  `mm`/`bmm` `out_dtype=` backport (#116), the stable-ABI header backport (#86,
  #96), asynchronous `isfinite` (#120), and `torch.cuda.streams` (#98)

New "Environment Variables" section
- the graph-rotation knobs (#72), `TORCHADA_PLATFORM`, the C++ operator-override
  switches (#61, #128), and the two variables that were already documented

Corrected and extended details
- torch.compile: FX `device` builtin (#124), Dynamo's device-index helper (#108),
  `MUSA_VISIBLE_DEVICES` mirroring (#106)
- C++ extensions: nested `<torch/cuda.h>` porting (#95), stable-ABI
  `STABLE_TORCH_LIBRARY_IMPL` rekeying and stream helpers (#100), torch 2.6+
  `include_paths`/`library_paths` signatures (#121), stale JIT build locks (#128)
- MoE tables are generated from checked-in recipes (#115)
- unsupported CUDA runtime APIs as no-ops (#65)
- Performance: replace the 0.1.94 / torch_musa 2.7.1 numbers with the checked-in
  0.1.95 / 2.11.0.post2 entry, and stop claiming every fast path is under 200ns

@yeahdongcn yeahdongcn left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge verdict: no blocking findings — ready to merge.
Docs-only diff at 384ef94a0cf8; every new claim I could check against the tree holds, and no earlier round raised findings.

Findings: 0 blocking · 3 nit

nit location problem fix
nit README.md:456, README_CN.md:417 the 400-600ns bound is exceeded by the file it cites (606 and 626 ns medians) quote ~400-630ns, or drop the bound
nit README.md:441, README_CN.md:404 numbers are attributed to MTT S5000, which benchmark_history.json never records drop the model, or add the device to the JSON
nit README.md:196, README_CN.md:181 mirroring is not conditional on both variables being set: an unset MUSA_VISIBLE_DEVICES clears CUDA_VISIBLE_DEVICES describe both branches
What changed · what I could not verify

Diff map. README.md +147/-23, README_CN.md +124/-21. Docs only: no source, test,
or config file is touched, so no build, import, or runtime path changes. Head commit is
384ef94a0cf8bbdccf023c95bc4c5ccf06428097 (xd/readme-refresh-2026-10), unchanged from
dispatch; mergeStateStatus: CLEAN.

Earlier rounds. gh pr view 129 --json reviews returns [] and
gh api repos/MooreThreads/torchada/pulls/129/comments returns []. Nothing earlier to
re-check; this is round 1.

Checks. gh pr checks 129 --repo MooreThreads/torchada prints no checks reported on the 'xd/readme-refresh-2026-10' branch. Per the round rules, that does not block.

Verified against the head commit (read-only; nothing was executed and no build or test
was run):

  • Performance table: benchmarks/benchmark_history.json 0.1.95 (2026-10-09) medians are
    172 / 172 / 182 / 202 / 262 ns for Stream, Event, _translate_device('cuda'),
    device_count(), is_built() — matching ~170/170/180/200/260ns and the stated order.
    0.1.94 medians 121/122/136/139/159 ns support ~120-160ns; its torch_musa_version is
    2.7.1+5ee0a64.
  • Environment-variable table: all ten variables exist with the documented defaults —
    TORCHADA_PLATFORM (cuda/musa/cpu), TORCHADA_EXCLUDE_DIRS,
    TORCHADA_CUDA_GRAPH_DEBUG_DUMP_PATH, TORCHADA_GRAPH_ROTATION (!= "0"),
    TORCHADA_GRAPH_EXEC_CAP (_DEFAULT_CAP = 1900), TORCHADA_GRAPH_AUTOPROBE,
    TORCHADA_GRAPH_EXEC_MARGIN (_DEFAULT_MARGIN = 128), TORCHADA_CPP_OPS_VERBOSE,
    TORCHADA_DEBUG_CPP_OPS (read in csrc/ops.h), TORCHADA_DISABLE_OP_OVERRIDE_<op>.
  • Graph rotation prose: ~2048 live-executable cap, ~40 layers, template retention, LRU cap
    1900, ~0.3 ms re-instantiation, zero-cost below the cap, capture_sizes * num_layers
    instantiation — all stated in src/torchada/_graph_rotation.py.
  • Compatibility table gates: _patch.py returns early from the Inductor heuristic fill-in,
    Tensor.log_, and the torch.accelerator memory overrides at
    >= "2.11.0.post2"; _cpp_ops.py:145-147 disables multinomial/log/log_ the same
    way; the func(*args, **kwargs) signature of _flash_attn_forward receives window_size
    as a 2-tuple built from window_size_left/right, matching the shim's window_size=(...)
    call. The out_dtype < 2.13.0 gate matches the >= "2.13.0" return.
  • New feature rows: CUDA memory-pool API claim matches the _cuda_beginAllocateCurrentThreadToPool
    / _cuda_endAllocateToPool / _cuda_releasePool injection into torch.musa.memory;
    _get_device_index → core._utils._get_musa_device_index; get_memory_info() →
    mem_get_info; torch.cuda.streams ← torch_musa.core.stream in sys.modules.
  • FA3 shim: adapter over flash_attn_func(..., return_softmax_lse=True), returns
    None, None for dropout auxiliaries, raises when output+LSE are not both returned, and is
    installed only when the provider lacks _flash_attn_forward — as documented.
  • Symbol map: #include <torch/cuda.h> → torch/musa.h (cpp_extension.py:371-372);
    STABLE_TORCH_LIBRARY_IMPL(<ns>, CUDA, ...) rekey regex allows any namespace and
    whitespace layout; cudaGridDependencySynchronize() /
    cudaTriggerProgrammaticLaunchCompletion() → ((void)0) in
    _mappings/cuda_runtime.py:38-39, and they are the only two ((void)0) mappings, so
    "the one CUDA feature" holds.
  • MoE recipe: --materialize and --merge-configs exist in tune_moe.py;
    tests/test_tune_moe_recipe.py has test_materialize_reproduces_the_shipped_row and
    test_checked_in_recipe_targets_the_shipped_table; ci/shapes.json and ci/models.json
    sit beside tune_moe.py; docs/tune_triton_moe.md exists.
  • Links and structure: anchors #torch_musa-compatibility, #environment-variables,
    #torch_musa-兼容性, #环境变量 all resolve; every relative link resolves, including
    examples/migrate_existing_project.md#important-note-on-gpu-detection and both docs/
    targets; pyproject.toml is 0.1.95; src/torchada/_mappings/ holds ~422 rule lines, so
    400+ holds; the decode_jpegs_cuda.h example is the one in the porter's own comment;
    both files end with a newline and have no trailing whitespace; code fences pair up.

Assumed, not verified. That the 0.1.94/0.1.95 runs were on MTT S5000 — the JSON records
only "platform": "MUSA", which is the basis for nit 2. I did not execute anything, did not
re-measure, and did not check the vendor claim that 2.13.0 fixes out_dtype (the PR keeps
that framing as an accepted risk, matching the source comments).

auto-review round 1/3 · verdict: clean

@yeahdongcn
yeahdongcn merged commit 174cda1 into main Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant