Add Tuna model support scaffolding

[zeel2104]

Purpose

Adds initial Tuna/Tuna-2 model support scaffolding for #3303.

This PR makes vLLM-Omni recognize Tuna/Tuna-2 model metadata and route it through a dedicated Tuna diffusion pipeline entrypoint. It includes:

• Tuna/Tuna-2 model type aliases for stage config resolution.
• Diffusion model detection for Tuna/Tuna-2 configs.
• TunaExternalPipeline registry entry and placeholder pipeline.
• Legacy and new-style Tuna stage configs.
• Unit coverage for model config resolution and detection.

Note: this is not full end-to-end Tuna inference yet. Upstream facebookresearch/tuna-2 currently uses its own Hydra-based inference flow and .pt checkpoint format, and full released weights / a stable HF-style loading contract are not available yet. The placeholder pipeline gives users a clear actionable error instead of an unknown-model failure.

Test Plan

Run lint on touched files:

python -m ruff check --no-cache vllm_omni/entrypoints/utils.py vllm_omni/diffusion/utils/hf_utils.py vllm_omni/diffusion/data.py vllm_omni/diffusion/registry.py vllm_omni/diffusion/models/tuna/pipeline_tuna.py tests/entrypoints/test_utils.py tests/diffusion/models/test_tuna_detection.py tests/test_config_factory.py

Run focused tests:

pytest tests/entrypoints/test_utils.py::TestResolveModelConfigPath::test_tuna_transformers_format_resolution tests/diffusion/models/test_tuna_detection.py tests/test_config_factory.py::TestArchitectureFallback::test_tuna_aliases_in_pipeline_models tests/test_diffusion_config_fields.py -q

Test Result

Ruff passed:

All checks passed!

Focused pytest passed in WSL with Python 3.12:

.......                                                                                                          [100%]
=================================================== warnings summary ===================================================
<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

.venv/lib/python3.12/site-packages/torch/jit/_script.py:362: 14 warnings
  /mnt/d/vllm-omni/.venv/lib/python3.12/site-packages/torch/jit/_script.py:362: DeprecationWarning: `torch.jit.script_method` is deprecated. Please switch to `torch.compile` or `torch.export`.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
7 passed, 16 warnings in 2.92s

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[hsliuustc0106]

BLOCKER scan:

Category	Result
Correctness	PASS
Reliability/Safety	PASS
Breaking Changes	PASS
Test Coverage	PASS
Documentation	ISSUES
Security	PASS

BLOCKING ISSUES:

1. [Documentation] Missing documentation updates: This PR adds Tuna model recognition scaffolding but does not update any documentation. Even for placeholder integration, the documentation should be updated to:

   • Mention Tuna/Tuna-2 as a recognized model in docs/models/supported_models.md (or equivalent)
   • Note the current status: recognized but runtime integration not yet available
   • Link to the upstream repository (facebookresearch/tuna-2) for users who want to track progress

   Without this documentation, users may be confused by the "not ready yet" error message.

Non-blocking suggestions:

2. Missing e2e validation of error message: The tests validate model detection and config resolution, but don't actually verify that the error message is actionable for users. Consider adding a test that instantiates TunaExternalPipeline and validates the error message contains the key information users need (upstream repo reference, current status).

3. Consider adding integration TODO: The error message is clear, but consider adding a TODO comment or issue reference in the code pointing to #3303 for future maintainers working on full Tuna integration.

4. Type annotation consistency: In pipeline_tuna.py, the forward and load_weights methods raise the same error but have different signatures. Consider whether load_weights should accept *args, **kwargs for consistency with other pipelines.

The scaffolding approach is reasonable given upstream constraints. Good unit test coverage for model detection.

[zeel2104]

@hsliuustc0106
Thanks for the review. I updated the PR to address the documentation blocker and the non-blocking suggestions:

• Added Tuna/Tuna-2 to docs/models/supported_models.md
• Documented the current status as recognized-only, with runtime integration not yet available
• Linked the upstream facebookresearch/tuna-2 repo
• Added an actionable error-message test for TunaExternalPipeline
• Added a TODO referencing [New Model]: [Meta] Tuna series #3303 for future full integration
• Updated load_weights to accept *args, **kwargs

[zhangj1an]

Hey @zeel2104, the convention is to include the entire model in one single PR, and then you can ping code maintainers like @princepride for review. The PR description should contain some visual examples like #2861. There is also a claude code skill vllm-omni/.claude/skills/add-diffusion-model/SKILL.md that might give more tips on what to do next. Hope this helps > <

[zeel2104]

@zhangj1an
@hsliuustc0106
@princepride

Thanks for the guidance! I’ll continue this PR as the full Tuna/Tuna-2 integration instead of only recognition scaffolding.

Before I start porting the runtime path, could you please confirm the expected checkpoint/model artifact to validate against? Should I use the upstream facebookresearch/tuna-2 release assets directly, or is there a preferred HuggingFace repo / checkpoint format reviewers expect?

Also, is it acceptable for the initial integration to depend on the upstream Tuna repo utilities, or should the required model/pipeline code be ported into vllm_omni/diffusion/models/tuna/ directly?

Once I have that clarified, I’ll add actual inference support, offline examples, docs, tests, and visual outputs in the PR description similar to #2861.

[zhangj1an]

Hey @zeel2104, you should start with official model weights (but I have not seen facebook released it yet...), like models listed in examples/offline_inference/text_to_image/README.md. In case the released model weights name have some mismatch or unnecessary nested structure, you will have to rename them and upload a custom HF model weights to adhere to vllm-omni format. (This is common for models from academic papers, so I think would not happen to this one).

The PR should be purely runnable on its own, without having to depend on upstream repo at all. If it used packages outside of vllm-omni/requirements/commons.txt, you should try to implement a custom version, the principle is to try import as little extra packages as possible to keep the repo lightweight.

[zeel2104]

Hey @zeel2104, you should start with official model weights (but I have not seen facebook released it yet...), like models listed in examples/offline_inference/text_to_image/README.md. In case the released model weights name have some mismatch or unnecessary nested structure, you will have to rename them and upload a custom HF model weights to adhere to vllm-omni format. (This is common for models from academic papers, so I think would not happen to this one).

The PR should be purely runnable on its own, without having to depend on upstream repo at all. If it used packages outside of vllm-omni/requirements/commons.txt, you should try to implement a custom version, the principle is to try import as little extra packages as possible to keep the repo lightweight.

Thanks @zhangj1an
Since official weights do not seem to be released yet, I’ll treat this as blocked for now.
Do maintainers prefer that I close this PR and reopen once Tuna/Tuna-2 weights are available, or keep it as draft until then?

[princepride]

Thanks for your contribute! Let's keep this PR open for now. Once they release the weights, we can move forward.

[hsliuustc0106]

Hi @zeel2104, friendly reminder — this PR hasn't had any activity (commits or reviews) in the past 12 days. 🕐

Could you please provide an update?

• If you're still working on it, that's great — just let us know.
• If you're blocked on something, feel free to ask for help.
• If this PR is no longer being pursued, please consider closing it so we can keep the review queue manageable.

Thanks for your contribution! 🙏