[ROCm][CI] Set "highest" matmul precision for reference hf_runner in test_bert_for_masked_lm

[micah-wil]

#48463 added test_bert_for_masked_lm which fails on ROCm due to subtle accuracy differences:

>           torch.testing.assert_close(hf_output, vllm_output, atol=3.2e-2, rtol=1e-3)
E           AssertionError: Tensor-likes are not close!
E           
E           Mismatched elements: 2 / 702006 (0.0%)
E           Greatest absolute difference: 0.049376606941223145 at index (7, 1011) (up to 0.032 allowed)
E           Greatest relative difference: 0.10987269133329391 at index (7, 1010) (up to 0.001 allowed)
models/language/pooling/test_token_classification.py:255: AssertionError

https://buildkite.com/vllm/ci/builds/77892/canvas?jid=019f5db5-b9e5-4728-86b6-2f2b8ef7e7c3&tab=output

This is because the vLLM runner uses matmul precisoin "highest" by default:

vllm/vllm/envs.py

Lines 597 to 602 in ecf4aa5

	"VLLM_FLOAT32_MATMUL_PRECISION": env_with_choices(
	"VLLM_FLOAT32_MATMUL_PRECISION",
	"highest",
	["highest", "high", "medium"],
	case_sensitive=False,
	),

vllm/vllm/v1/worker/gpu_worker.py

Lines 147 to 149 in ecf4aa5

	# configure float32 matmul precision according to vLLM env.
	precision = envs.VLLM_FLOAT32_MATMUL_PRECISION
	torch.set_float32_matmul_precision(precision)

However, for this test group, we explicitly set torch.set_float32_matmul_precision("high") on ROCm to workaround a separate issue (see #31820).

vllm/tests/models/language/pooling/conftest.py

Line 23 in ecf4aa5

torch.set_float32_matmul_precision("high")

This gets picked up by the reference hf_runner, so the accuracy of the reference is less stable than on CUDA which does not set torch.set_float32_matmul_precision("high").

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

Hi @micah-wil, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.