Repository navigation
docs: refresh the READMEs against the work merged since May - #129
Conversation
The English and Chinese READMEs have not kept up with the May-October work. This documents what merged, moves the version-gated shims into one table, and refreshes the measured numbers that had gone stale. Feature table - CUDA memory-pool APIs, `torch.cuda.streams`, CUDA-graph executable rotation, `torch.cuda._get_device_index`, `get_memory_info()`, and the FlashAttention provider shims (#61, #98, #103, #106, #108, #115) - the "What Works" table goes back to one line per feature; the paragraph-sized `log_` / `isfinite` / `out_dtype` cells move into the new section below New "torch_musa Compatibility" section - one table of every version-gated shim with the release it is installed on: the four `< 2.11.0.post2` patches (#106, #113, #124), the `< 2.13.0` `mm`/`bmm` `out_dtype=` backport (#116), the stable-ABI header backport (#86, #96), asynchronous `isfinite` (#120), and `torch.cuda.streams` (#98) New "Environment Variables" section - the graph-rotation knobs (#72), `TORCHADA_PLATFORM`, the C++ operator-override switches (#61, #128), and the two variables that were already documented Corrected and extended details - torch.compile: FX `device` builtin (#124), Dynamo's device-index helper (#108), `MUSA_VISIBLE_DEVICES` mirroring (#106) - C++ extensions: nested `<torch/cuda.h>` porting (#95), stable-ABI `STABLE_TORCH_LIBRARY_IMPL` rekeying and stream helpers (#100), torch 2.6+ `include_paths`/`library_paths` signatures (#121), stale JIT build locks (#128) - MoE tables are generated from checked-in recipes (#115) - unsupported CUDA runtime APIs as no-ops (#65) - Performance: replace the 0.1.94 / torch_musa 2.7.1 numbers with the checked-in 0.1.95 / 2.11.0.post2 entry, and stop claiming every fast path is under 200ns
yeahdongcn
left a comment
There was a problem hiding this comment.
Merge verdict: no blocking findings — ready to merge.
Docs-only diff at 384ef94a0cf8; every new claim I could check against the tree holds, and no earlier round raised findings.
Findings: 0 blocking · 3 nit
| nit | location | problem | fix |
|---|---|---|---|
| nit | README.md:456, README_CN.md:417 |
the 400-600ns bound is exceeded by the file it cites (606 and 626 ns medians) |
quote ~400-630ns, or drop the bound |
| nit | README.md:441, README_CN.md:404 |
numbers are attributed to MTT S5000, which benchmark_history.json never records |
drop the model, or add the device to the JSON |
| nit | README.md:196, README_CN.md:181 |
mirroring is not conditional on both variables being set: an unset MUSA_VISIBLE_DEVICES clears CUDA_VISIBLE_DEVICES |
describe both branches |
What changed · what I could not verify
Diff map. README.md +147/-23, README_CN.md +124/-21. Docs only: no source, test,
or config file is touched, so no build, import, or runtime path changes. Head commit is
384ef94a0cf8bbdccf023c95bc4c5ccf06428097 (xd/readme-refresh-2026-10), unchanged from
dispatch; mergeStateStatus: CLEAN.
Earlier rounds. gh pr view 129 --json reviews returns [] and
gh api repos/MooreThreads/torchada/pulls/129/comments returns []. Nothing earlier to
re-check; this is round 1.
Checks. gh pr checks 129 --repo MooreThreads/torchada prints no checks reported on the 'xd/readme-refresh-2026-10' branch. Per the round rules, that does not block.
Verified against the head commit (read-only; nothing was executed and no build or test
was run):
- Performance table:
benchmarks/benchmark_history.json0.1.95 (2026-10-09) medians are
172 / 172 / 182 / 202 / 262 ns for Stream, Event,_translate_device('cuda'),
device_count(),is_built()— matching~170/170/180/200/260nsand the stated order.
0.1.94 medians 121/122/136/139/159 ns support~120-160ns; itstorch_musa_versionis
2.7.1+5ee0a64. - Environment-variable table: all ten variables exist with the documented defaults —
TORCHADA_PLATFORM(cuda/musa/cpu),TORCHADA_EXCLUDE_DIRS,
TORCHADA_CUDA_GRAPH_DEBUG_DUMP_PATH,TORCHADA_GRAPH_ROTATION(!= "0"),
TORCHADA_GRAPH_EXEC_CAP(_DEFAULT_CAP = 1900),TORCHADA_GRAPH_AUTOPROBE,
TORCHADA_GRAPH_EXEC_MARGIN(_DEFAULT_MARGIN = 128),TORCHADA_CPP_OPS_VERBOSE,
TORCHADA_DEBUG_CPP_OPS(read incsrc/ops.h),TORCHADA_DISABLE_OP_OVERRIDE_<op>. - Graph rotation prose: ~2048 live-executable cap, ~40 layers, template retention, LRU cap
1900, ~0.3 ms re-instantiation, zero-cost below the cap,capture_sizes * num_layers
instantiation — all stated insrc/torchada/_graph_rotation.py. - Compatibility table gates:
_patch.pyreturns early from the Inductor heuristic fill-in,
Tensor.log_, and thetorch.acceleratormemory overrides at
>= "2.11.0.post2";_cpp_ops.py:145-147disablesmultinomial/log/log_the same
way; thefunc(*args, **kwargs)signature of_flash_attn_forwardreceiveswindow_size
as a 2-tuple built fromwindow_size_left/right, matching the shim'swindow_size=(...)
call. Theout_dtype< 2.13.0gate matches the>= "2.13.0"return. - New feature rows: CUDA memory-pool API claim matches the
_cuda_beginAllocateCurrentThreadToPool
/_cuda_endAllocateToPool/_cuda_releasePoolinjection intotorch.musa.memory;
_get_device_index→core._utils._get_musa_device_index;get_memory_info()→
mem_get_info;torch.cuda.streams←torch_musa.core.streaminsys.modules. - FA3 shim: adapter over
flash_attn_func(..., return_softmax_lse=True), returns
None, Nonefor dropout auxiliaries, raises when output+LSE are not both returned, and is
installed only when the provider lacks_flash_attn_forward— as documented. - Symbol map:
#include <torch/cuda.h>→torch/musa.h(cpp_extension.py:371-372);
STABLE_TORCH_LIBRARY_IMPL(<ns>, CUDA, ...)rekey regex allows any namespace and
whitespace layout;cudaGridDependencySynchronize()/
cudaTriggerProgrammaticLaunchCompletion()→((void)0)in
_mappings/cuda_runtime.py:38-39, and they are the only two((void)0)mappings, so
"the one CUDA feature" holds. - MoE recipe:
--materializeand--merge-configsexist intune_moe.py;
tests/test_tune_moe_recipe.pyhastest_materialize_reproduces_the_shipped_rowand
test_checked_in_recipe_targets_the_shipped_table;ci/shapes.jsonandci/models.json
sit besidetune_moe.py;docs/tune_triton_moe.mdexists. - Links and structure: anchors
#torch_musa-compatibility,#environment-variables,
#torch_musa-兼容性,#环境变量all resolve; every relative link resolves, including
examples/migrate_existing_project.md#important-note-on-gpu-detectionand bothdocs/
targets;pyproject.tomlis0.1.95;src/torchada/_mappings/holds ~422 rule lines, so
400+holds; thedecode_jpegs_cuda.hexample is the one in the porter's own comment;
both files end with a newline and have no trailing whitespace; code fences pair up.
Assumed, not verified. That the 0.1.94/0.1.95 runs were on MTT S5000 — the JSON records
only "platform": "MUSA", which is the basis for nit 2. I did not execute anything, did not
re-measure, and did not check the vendor claim that 2.13.0 fixes out_dtype (the PR keeps
that framing as an accepted risk, matching the source comments).
auto-review round 1/3 · verdict: clean
The two READMEs have not kept up with the work merged since May. This PR
scans every merged PR from
#61(2026-05-07) to#128(2026-10-09), documentswhat is missing, and fixes claims that no longer match the repository. Both
languages are updated together.
Docs only: no source, test, or config change.
What was missing
torch.cuda.memory.*and the CUDA memory-pool APIscudaGridDependencySynchronize()/cudaTriggerProgrammaticLaunchCompletion()<torch/cuda.h>portingtorch.accelerator.get_memory_info()torch.cuda.streamsSTABLE_TORCH_LIBRARY_IMPLrekeying and stream helpers_flash_attn_forwardMUSA_VISIBLE_DEVICEStorch.cuda._get_device_indexfor TorchDynamoinclude_paths/library_pathssignaturesdevicebuiltinStructural changes
torch_musa Compatibility(new section). Every version-gated shim waspreviously a paragraph-sized cell in "What Works". They now live in one table
with the release each is installed on: the four
< 2.11.0.post2patches, the< 2.13.0mm/bmmout_dtype=backport, asynchronousisfinite, thestable-ABI header backport, and
torch.cuda.streams._patch.pyalreadyrefers to "this repository's compatibility table" — this is that table, and it
is now readable as one.
Environment Variables(new section). Only two of the ten user-facingTORCHADA_*variables were documented; the graph-rotation knobs, theoperator-override switches, and
TORCHADA_PLATFORMwere only discoverable byreading the source.
again.
out_dtyperisk paragraph is kept verbatim, and the remaining"Not covered" scope note moved with it.
Stale claim corrected
The Performance section listed the 0.1.94 / torch_musa 2.7.1 medians
(130-155 ns) and stated that all frequently-called operations complete within
200 ns. The checked-in
benchmarks/benchmark_history.jsonalready has a newer0.1.95 entry (2026-10-09, torch_musa 2.11.0.post2+musa5.2.0) measuring
170-260 ns for the same fast paths. The section now quotes the newest
measurement, keeps the 0.1.94 figures as the comparison, and no longer claims a
sub-200 ns bound the data does not support.
Notes
end with a newline and have no trailing whitespace.
torchada>=0.1.95and the mapping-rule count (400+) still matchpyproject.tomlandsrc/torchada/_mappings/.descriptions; for example fix: generalize stable ABI extension porting for MUSA #100's description mentions
CUDA_VERSION/USE_MUSAmappings that the merged commit does not contain, so only the rekeying and
stream helpers it did merge are documented.