New stable abi cleanup

[cleonard530]

Purpose

This is to cleanup a bit after migrating vLLM to the libtorch stable ABI. This includes:

• moving all headers that are only used by files in the libtorch_stable directory to that directory.
• Adding device guards that we didn't add during migration because they weren't there before, but probably should be there.
• removing an import warning for _qutlass_C in cuda.py

Other things that could be improved that are not cleaned up here:

• A lot of variables that could be const are not. This PR scope is already large (50 files changed) and this update would make it a lot larger without a huge benefit, so I did not update them here.
• Some header file names in csrc/ are duplicated in csrc/libtorch_stable/ (e.g. dispatch_utils.h and torch_utils.h) which can cause confusion when importing. For instance, a file in csrc/libtorch_stable/ with #include "torch_utils.h" will include csrc/libtorch_stable/torch_utils.h, but if it was in any other directory it would include csrc/torch_utils.h. This should be resolved when we move the CPU and ROCm builds to the stable ABI as well.

cc @Harry-Chen @janeyx99

Test Plan

The changes are small but it covers a lot of files so the test plan is broad.

pytest tests/kernels/core
pytest tests/kernels/moe
pytest tests/kernels/quantization
pytest tests/kernels/test_top_k_per_row.py
pytest tests/kernels/test_fp32_router_gemm.py
pytest tests/kernels/test_fused_minimax_m3_qknorm_rope_kv_insert.py

Test Result

16,824 Test Executed

• 16,588 passed
• 236 failures

The same failures also failed on the main branch.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[janeyx99]

LGTM though we can do more and move headers that are used at all in libtorch_stable AND are stable themselves to libtorch_stable too right?

In other words, a file doesn't need to only be used in libtorch_stable to be migrated; it can also be used outside of libtorch_stable.

[cleonard530]

LGTM though we can do more and move headers that are used at all in libtorch_stable AND are stable themselves to libtorch_stable too right?

In other words, a file doesn't need to only be used in libtorch_stable to be migrated; it can also be used outside of libtorch_stable.

Oh, I misunderstood what you were saying in the comments on #44013, but you're right. Any header that is used to by a libtorch_stable file could be moved with little risk of regressing. I'll add a quick update to move over a few more headers.

[janeyx99]

@cleonard530 If this PR is already locally validated, it's ok to land this and then move more files in a new PR too. The important thing is for the files that are migrated to truly be ABI stable. Technically they can include unstable things but if those functions/structs never get used from within the libtorch_stable extension, then the TORCH_TARGET_VERSION flag won't catch that. But your lint might!

[cleonard530]

@cleonard530 If this PR is already locally validated, it's ok to land this and then move more files in a new PR too. The important thing is for the files that are migrated to truly be ABI stable. Technically they can include unstable things but if those functions/structs never get used from within the libtorch_stable extension, then the TORCH_TARGET_VERSION flag won't catch that. But your lint might!

Sounds good, I will leave this as is then and move the other headers over in future PRs

[mergify]

Hi @cleonard530, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[cleonard530]

@Harry-Chen, I checked the failure logs and they don't seem to be related to this PR. Can you rerun the failed test?

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @cleonard530.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

Hi @cleonard530, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[youkaichao]

stamp on behalf of @Harry-Chen

[Harry-Chen]

• A lot of variables that could be const are not. This PR scope is already large (50 files changed) and this update would make it a lot larger without a huge benefit, so I did not update them here.

Does this imply any potential performance improvement, or only semantics and safety guarantee?

[cleonard530]

• A lot of variables that could be const are not. This PR scope is already large (50 files changed) and this update would make it a lot larger without a huge benefit, so I did not update them here.

Does this imply any potential performance improvement, or only semantics and safety guarantee?

It would mostly be for semantics/safety. If there were any performance improvements, I think they would be negligible.