Skip to content

feat(s-series): device Philox RNG (rng_uniform/normal + dropout) on x86 + ROCm (P6) - #207

Merged
gstoner merged 4 commits into
mainfrom
s2/p6-philox-rng
Jun 29, 2026
Merged

gstoner merged 4 commits into
mainfrom
s2/p6-philox-rng

Conversation

@gstoner

@gstoner gstoner commented Jun 28, 2026

Copy link
Copy Markdown
Owner

Summary

Counter-based Philox-4x32-10 RNG as a real device lane on both native targets. A device RNG can't bit-match numpy's Generator (numpy uses Philox-4x64 + its own Lemire/ziggurat transforms), so Tessera's device lane uses the standard Philox-4x32-10 (Salmon et al. 2011 — the JAX/cuRAND algorithm): embarrassingly parallel (element i depends only on key + counter=i), reproducible, validated BIT-EXACTLY against a numpy reference of the identical algorithm. A separate deterministic stream from tessera.rng (the host numpy-Generator path).

What lands

  • avx512_philox_f32.cpp — scalar Philox-4x32-10 C-ABI uniform kernel.
  • GenerateROCMPhiloxKernel.cpp + ROCM_PhiloxKernelOp + pass registration — the gfx1151 MLIR Philox kernel (10 unrolled rounds, i64 umulhi); lowers to a valid hsaco, HIP-launched.
  • rng_device.py — the numpy Philox-4x32-10 reference + uniform/normal/dropout transforms (the bit-exact target).
  • runtime.py_rng_compute + _execute_{x86,rocm}_compiled_rng: device uniform bits + host transform (uniform-scale / Box-Muller normal / dropout mask).
  • backend_manifest / execution_matrix: fused x86 + compiled rocm entries, rows, descriptions, fixture map.

Proof

test_{x86,rocm}_rng_compiled.py — uniform/normal bit-exact vs the reference + statistics (mean≈0.5/var≈1/12; normal mean≈0/std≈1) + dropout rate/scale/eval + determinism. Both executed on hardware (AVX-512 + gfx1151): 9 passed; uniform/normal bit-identical across x86, gfx1151, and the numpy reference. Manifest/matrix/coverage/doc-drift gates green; ruff + mypy clean.

Note

The device stream is a standalone deterministic Philox-4x32-10, distinct from tessera.rng's host numpy-Generator stream — RNG streams are implementation-defined per backend; the contract is correctness + reproducibility, proven by bit-exact agreement with the documented reference. Dashboard: rng_uniform/rng_normal/dropout backend_kernel reference → partial.

🤖 Generated with Claude Code

…86 + ROCm (P6)

Counter-based Philox-4x32-10 RNG as a real device lane on both native targets.
A device RNG can't bit-match numpy's Generator (numpy uses Philox-4x64 + its own
Lemire/ziggurat transforms), so Tessera's device lane uses the STANDARD
Philox-4x32-10 (Salmon et al. 2011 — the JAX/cuRAND algorithm): embarrassingly
parallel (element i depends only on key+counter=i), reproducible, and validated
BIT-EXACTLY against a numpy reference of the identical algorithm. A SEPARATE
deterministic stream from tessera.rng (the host numpy-Generator path).

- avx512_philox_f32.cpp: scalar Philox-4x32-10 C-ABI kernel (uniform f32).
- GenerateROCMPhiloxKernel.cpp + ROCM_PhiloxKernelOp (TesseraROCMOps.td) +
  pass registration: the gfx1151 MLIR Philox kernel (10 unrolled rounds, i64
  umulhi); lowers to a valid hsaco. HIP-launched via _rocm_philox_uniform.
- rng_device.py: the numpy Philox-4x32-10 reference + uniform/normal/dropout
  transforms (the bit-exact validation target).
- runtime.py: `_rng_compute` + `_execute_{x86,rocm}_compiled_rng` — device
  uniform bits + host transform (uniform scale / Box-Muller normal / dropout
  mask); registered in `_executor_table`.
- backend_manifest / execution_matrix: fused x86 + compiled rocm entries, rows,
  descriptions, fixture map.
- tests: test_{x86,rocm}_rng_compiled.py — uniform/normal bit-exact vs the
  reference + statistics (mean/var) + dropout rate/scale/eval + determinism.
  Both executed on hardware (AVX-512 + gfx1151): 9 passed; uniform/normal
  bit-identical across x86, gfx1151, and the numpy reference.

Dashboard: rng_uniform/rng_normal/dropout backend_kernel reference -> partial.
Drift gate clean; ruff + mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a0f427f78c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread python/tessera/runtime.py
Comment thread python/tessera/runtime.py Outdated
@gstoner

gstoner commented Jun 28, 2026

Copy link
Copy Markdown
Owner Author

Note: Tessera's device Philox-4x32-10 is the same standard algorithm rocRAND ships (rocRAND PRNG list includes Philox 4x32-10; Salmon et al. 2011), so this device stream is consistent with rocRAND's Philox by construction — validated bit-exact against the in-repo numpy reference of the identical rounds. Per Decision #23 it's a reimplementation in Tessera codegen, not a rocRAND/hipRAND wrap. Future RNG generators (ThreeFry, Sobol QRNG, philox_4x64) can mirror rocRAND's set.

gstoner and others added 3 commits June 28, 2026 16:27
# Conflicts:
#	docs/audit/generated/test_coverage.md
#	python/tessera/runtime.py
…gs (PR #207 review)

- P1: add avx512_philox_f32.cpp to both x86 CMake source lists
  (tessera_x86_elementwise + tessera_x86_backend). Without it a clean source
  build produced a libtessera_x86_elementwise.so missing
  tessera_x86_philox_uniform_f32, so every x86 RNG op would fail the ctypes
  symbol lookup at runtime (CI passed only because the rng tests skip-clean when
  the .so is absent).
- P2: rng_uniform now reads the public-API range names `lo`/`hi` (used by
  tessera.ops.rng_uniform + the CPU reference), falling back to low/high — a
  non-default range no longer silently returns [0,1) for traced/compiled calls.
- test: rng_uniform lo/hi alias case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts:
#	docs/audit/generated/e2e_op_coverage.md
#	docs/audit/generated/test_coverage.md
#	python/tessera/runtime.py
@gstoner
gstoner merged commit 65ec632 into main Jun 29, 2026
17 checks passed
@gstoner
gstoner deleted the s2/p6-philox-rng branch June 29, 2026 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant