fix(diffusion): reuse warm HF cache instead of re-downloading models

[pthombre]

What does this PR do ?

Stops the diffusion recipes (and their preprocessing step) from re-fetching models from the HuggingFace Hub on every run even when the model is already cached in HF_HOME.

The issue

The diffusers bridge and the preprocessing processors pass a bare repo id straight to diffusers' from_pretrained:

• NeMoAutoDiffusionPipeline.from_pretrained / from_config → DiffusionPipeline.from_pretrained(model_id, ...)
• tools/diffusion/processors/{flux,flux2,hunyuan,qwen_image,wan}.py → <Pipeline>.from_pretrained(model_name, ...)

With a bare repo id, huggingface_hub resolves in online mode by default: for every file it issues a network request to the Hub to revalidate the commit hash / ETag before deciding whether to reuse the cache. So even a warm HF_HOME is re-validated over the network, and any ETag drift, partial cache, or unreachable-but-not-offline state turns into a fresh download. A diffusion pipeline loads ~6 sub-components (VAE, two text encoders, two tokenizers, transformer) in one call, which multiplies the round-trips.

The behavior depended entirely on the ambient HF_HUB_OFFLINE env var being set in every stage/subprocess. The diffusion launcher never sets it itself — it only relies on the outer CI template — so any context where it isn't propagated (local dev, a runner without it, a subprocess that drops it) re-downloads.

The transformers bridge does not have this problem: it pre-resolves the repo to a local snapshot directory in code (_resolve_model_dir → snapshot_download(..., local_files_only=True) in nemo_automodel/_transformers/model_init.py, plus an explicit HF_HUB_OFFLINE short-circuit) and hands that local path to HF — which then performs zero network I/O. The diffusers bridge is a separate integration layer and never went through those helpers, so the cache discipline was simply missing on the diffusion path.

The fix

Port the same discipline to the diffusion path. Once HF from_pretrained receives a local directory, it does no network I/O regardless of env-var propagation, which is what removes the per-run re-download.

Changelog

• Add nemo_automodel/_diffusers/_hf_cache.py with resolve_diffusion_model_dir(): returns local paths unchanged; for repo ids it downloads the snapshot once only when the cache is cold and the process is online (HF_HUB_OFFLINE unset), then resolves the local snapshot with local_files_only=True so the subsequent from_pretrained performs no network I/O. Guarded with safe_import('huggingface_hub') and passes through unchanged when the lib is unavailable.
• Use the helper in NeMoAutoDiffusionPipeline.from_pretrained (finetune) and from_config (pretrain).
• Use the helper in the flux, flux2, hunyuan, qwen_image, and wan preprocessing processors (wan resolves once and reuses the dir across its three subfolder= loads).
• Add unit tests in tests/unit_tests/_diffusers/test_hf_cache.py covering local-path passthrough, offline (single cache-only call), online (download-then-resolve), and hub-unavailable passthrough.

Cache state	HF_HUB_OFFLINE	Behavior after fix
Warm	unset	local dir resolved, no network, no re-download
Warm	set	local dir resolved, no network
Cold	unset	download once, then resolve local dir
Cold	set	clean "not in cache" error

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?

Additional Information

• Independent of the flux nightly CI fixes in fix(diffusion): resolve flux nightly CI failures #2529.
• Not addressed here: the video-dataset snapshot_download('modal-labs/dissolve') in diffusion_finetune_launcher.sh has the same online-by-default gap, but only runs in CI where HF_HUB_OFFLINE is already set. Can harden separately if desired.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[pthombre]

/ok to test f6a139a

[pthombre]

/claude review

[claude]

nit: This assertion verifies the kwargs of the second snapshot_download call but doesn't check that "some/repo-id" was also passed as the positional arg. If someone accidentally dropped the model_id argument from the second call, this test would still pass.

Suggested change

	assert mock_sd.call_args_list[1].kwargs == {"local_files_only": True}
	assert mock_sd.call_args_list[1] == (("some/repo-id",), {"local_files_only": True})

[pthombre]

/ok to test a75a683