[subodh-dubey-amd]

ISSUE ID : #9559

Motivation

• On ci(jax): serve the JAX wheel build from the EngFlow Bazel cache #7912, Scott asked for each build's cache to be documented next to that build, as ci(jax): serve the JAX wheel build from the EngFlow Bazel cache #7912 does for JAX (comment). The PyTorch README didn't cover its cache.

Technical Details

• Adds a "Compiler cache" section to external-builds/pytorch/README.md:
  • local builds: --use-sccache / --use-ccache and --sccache-no-wrap;
  • CI and release builds: S3 bucket, IAM role and key prefix per release type;
  • fork PRs build uncached (compute_pytorch_cache_type.py).
• Once ci(jax): serve the JAX wheel build from the EngFlow Bazel cache #7912 merges, the PyTorch row of its cache table in ccache_troubleshooting.md can point here.

Test Plan

• pre-commit run --files external-builds/pytorch/README.md.
• Checked each statement against build_prod_wheels.py, setup_sccache_rocm.py, compute_pytorch_cache_type.py, the Linux and Windows PyTorch wheel workflows, and s3_buckets.md.

Test Result

• pre-commit passes, and every link in the new section resolves.

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/CONTRIBUTING.md.

[therock-pr-bot]

✅ All Checks Passed — Ready for Review

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	PR does not contain code files — Unit Test auto-passed
🔎 pre-commit	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—
🤖 therock-pr-bot	✅ Pass	—

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

[therock-pr-bot]

🎉 All checks passed! This PR is ready for review.

[ScottTodd]

Ok to document what currently exists, but we should set -DCMAKE_HIP_COMPILER_LAUNCHER=ccache instead now, on both Linux and Windows. See #8824

We're missing out of a large number of potential cache hits with the current code here:

TheRock/build_tools/setup_sccache_rocm.py

Lines 93 to 98 in b690ead

	env = {
	"CMAKE_C_COMPILER_LAUNCHER": str(sccache_path),
	"CMAKE_CXX_COMPILER_LAUNCHER": str(sccache_path),
	}
	if hip_launcher and not is_windows:
	env["HIP_CLANG_LAUNCHER"] = str(sccache_path)

See also the "Optimizing the PyTorch build" section on this recent comment of mine: #7346 (comment)

[ScottTodd]

I have a fix for this ready, preparing a PR now.

We'll just set all three compiler launchers on both platforms, as that works now. I'll also delete build_tools/setup_sccache_rocm.py for now since there's no meaningful logic left in the file now, but we can add it back later if we want to configure the remote cache server based on release type, job triggers, etc. in there.

[ScottTodd]

Ok to document what exists, but we should complete the work and limit to always sscache. Keeping the multiple options was an experiment, and the experiment is over now :)