[Frontend] Split OpenAIServingModels into OpenAIModelRegistry + OpenAIServingModels

[sagearc]

Purpose

After #36166, OpenAIServingRender (a CPU-only, engine-free handler) was receiving a bare list[str] of model names and reimplementing model lookup logic (_check_model, _is_model_supported, show_available_models).

This PR extracts OpenAIModelRegistry — a lightweight, engine-free class for base-model verification — and wires it into the render path via composition.

Changes

• OpenAIModelRegistry (new): read-only base-model registry with no engine/LoRA dependency. Provides check_model, show_available_models, is_base_model, model_name.
• OpenAIServingModels: composes OpenAIModelRegistry via self.registry; delegates base-model ops to it, layers LoRA adapter CRUD on top.
• OpenAIServing._check_model: unchanged — retains the full LoRA-aware verification logic (static + runtime resolution).
• OpenAIServingRender: accepts model_registry: OpenAIModelRegistry instead of served_model_names: list[str]; removes duplicate _check_model, _is_model_supported, and show_available_models.
• /v1/models endpoint: return type widened to OpenAIModelRegistry | OpenAIServingModels.
• init_render_app_state: constructs OpenAIModelRegistry directly for the render-only server.

Test Plan

Existing tests cover the affected code paths.

Test Result

Pre-commit checks pass.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR
• The test plan
• The test results
• (Optional) Documentation update
• (Optional) Release notes update

[mergify]

Hi @sagearc, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[gemini-code-assist]

Code Review

The code refactors the OpenAI API entrypoints by introducing an OpenAIModelRegistry class to centralize base model management and checks, with OpenAIServingModels now inheriting from it to handle LoRA-specific logic. This change streamlines model validation and listing across various API components, including api_server, engine/serving, generate/api_router, and render/serving, by delegating these operations to the new model_registry object. A review comment pointed out a potential IndexError in OpenAIModelRegistry.model_name if base_model_paths is empty, suggesting an explicit check for improved robustness.

[DarkLight1337]

I prefer composition over inheritance here

[sagearc]

Since OpenAIServingModels requires an engine client, OpenAIServingRender relies on the base OpenAIModelRegistry to stay engine-free. Inheritance allows the overridden check_model to still pick up loras automatically during serving. How would you recommend structuring this dependency with composition?

[DarkLight1337]

I mean that OpenAIServingModels can contain an instance of OpenAIModelRegistry. There is no need to change OpenAIServingRender itself.

[sagearc]

I understand using composition there. My main hesitation is how that interacts with the renderer. If my understanding is correct, passing the contained OpenAIModelRegistry to OpenAIServingRender would mean the renderer only checks for base models.

I originally structured it this way to delegate both preprocessing and the model check from the serving layer to the renderer, to avoid having two separate entry points for preprocessing (one with the check, and one without, ref). I might be missing a cleaner way to wire this up though

[DarkLight1337]

LoRA doesn't affect Renderer itself, though I understand from the client's perspective they should be able to use the LoRA model for both components. Perhaps we need to integrate this logic even in the engine-less case then.

[sagearc]

After taking a further look, choosing composition here won't reduce the need for a duplicate check_model in both OpenAIServingRender and the serving layers, since check_model can dynamically load LoRAs.
What if, instead of the current approach, we revert the changes introduced in this PR and allow the engine client to be None in OpenAIServingModels?

[DarkLight1337]

Can you open a new PR to show what that looks like?

[sagearc]

Sure

[sagearc]

Something like that?
#36655
@DarkLight1337

[mergify]

Hi @sagearc, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[mergify]

Hi @sagearc, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @sagearc.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[sagearc]

@DarkLight1337 kept changes minimal, registry handles the base models while keeping lora related logic untouched

[DarkLight1337]

This looks better, thanks