Skip to content

docs: hipDNN route design (decisions open) - #10

Draft
SamuelReeder wants to merge 22 commits into
mainfrom
users/sareeder/hipdnn-design
Draft

SamuelReeder wants to merge 22 commits into
mainfrom
users/sareeder/hipdnn-design

Conversation

@SamuelReeder

@SamuelReeder SamuelReeder commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Design doc, no code: docs/hipdnn-route.md. It covers shipping the FlyDSL decode megakernels through hipDNN to vLLM via hipdnn_vllm (ROCm/rocm-libraries#14255).

  • No RFC cycle. Each part lands as a small PR behind a default-OFF flag. Only two items need sign-off from another team: the hipdnn_vllm generic op and its strict-on-execute semantics (dpeabody-amd), and write access to the DVC ingestor remote (hip-kernel-provider owners).
  • Fixed point and invariants H1-H7: bitwise equal, decline goes to vLLM's path, TP agreement, graph capture, co-residency, no plan or launch change by default (checked on every family wheel), no TP rank divergence after a launch failure.
  • Binary key: src is vLLM's own compile_key(cfg.compiled(), kernels/), one per code object. If no width has a plan, the spec's warmup raises at startup, so host/binary drift is an error, not wrong bits.
  • Geometry: vLLM's kernel copies have no rt2, so every model ships rt0 (14 objects per max_model_len) until a vLLM PR ports rt2 (step 1a). Then rt2 for Qwen3.6 and Qwen3.8 TP1; Qwen3.8 TP2 and Gemma4 (+1.81 % at M = 1) stay rt0.
  • Packaging: export here, DVC ingestor remote, sha256 lock checked by the pack step.
  • Risks a flag does not remove once enabled: co-residency hangs, in-place state and fallback re-run under TP2, three-repository version lock.
  • Every citation is file:line at a stated commit (#14255 cb84aeb, rocm-libraries develop 2d1e830e1bb, vLLM branches, #61311 b426bf7be, megakernels main 10bd202).

…1-H7, binary key, D1-D11 with recommendations, packaging on hipdnn_vllm (#14255), TP2, upstream sequence, risks
…ict on execute only, H6/H7 restated, no RFC gate, rt_geom from main, sha256 check at pack, citation fixes
@SamuelReeder
SamuelReeder force-pushed the users/sareeder/hipdnn-design branch from 319244f to 03d41f4 Compare October 30, 2026 05:51
… everywhere until then, src per object, H6 on every wheel, startup raise when no width has a plan, citations at develop 2d1e830e1bb and #61311 b426bf7be
…Alola login node); e2e_plan.sh: q6/g CLI, top-5 logprobs, PASS/FAIL verdicts, dvc add, info/cmd for the pipeline
…_wheel estimate from the local test; prep extracts over old bundles
… failed on the node), pins torchvision 0.28.0; e2e set = megakernel-engine 18c6ef46d60 lock (5b83b31430d7), RL_SHA 9bb279d30d0
…un-to-run spread for H1); check prints H3 and the spread; pipeline: e_q8tp2 estimate 65
…ull fetch broke off); ERROR stages get 3 attempts
… read-only: dvc add and build_wheel.sh's dvc pull failed); dvc add again records the set
…24 wheel's backend needs GLIBC_2.38 and fell back to rocm-sdk's 0.3.0 backend), image python as VENV, cone sparse checkout; RL_SHA adeef272211
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant