Skip to content

fix: add missing return in load_model_with_fallback#272

Closed
mikepixelmagic-dev wants to merge 1 commit intowaybarrios:mainfrom
mikepixelmagic-dev:fix/load-model-missing-return
Closed

fix: add missing return in load_model_with_fallback#272
mikepixelmagic-dev wants to merge 1 commit intowaybarrios:mainfrom
mikepixelmagic-dev:fix/load-model-missing-return

Conversation

@mikepixelmagic-dev
Copy link
Copy Markdown
Contributor

Summary

load_model_with_fallback() in utils/tokenizer.py is missing a return statement on the happy path. When mlx_lm.load() succeeds without raising a ValueError, the function falls through the try block and implicitly returns None.

This causes a TypeError in MLXLanguageModel.load():

self.model, self.tokenizer = load_model_with_fallback(...)
TypeError: cannot unpack non-iterable NoneType object

The fix is a single line — adding return model, tokenizer after the successful load() call.

How to reproduce

import asyncio
from vllm_mlx.models.llm import MLXLanguageModel

async def test():
    m = MLXLanguageModel('mlx-community/Qwen3.5-35B-A3B-4bit', trust_remote_code=True)
    m.load()  # TypeError: cannot unpack non-iterable NoneType object

asyncio.run(test())

Note: the bug only manifests when called inside an async context (e.g., asyncio.run()), which is how the server starts the engine via SimpleEngine.start(). Direct calls outside async may work due to Python caching behaviour.

Related issues

Fixes #252, #249

Test

Verified with Qwen3.5-35B-A3B (4-bit and 8-bit) on Apple M5 Max. Server starts and serves requests successfully after the fix.

When mlx_lm.load() succeeds without raising a ValueError, the function
falls through the try block without returning, implicitly returning None.
This causes a TypeError in MLXLanguageModel.load() which tries to unpack
the result:

    self.model, self.tokenizer = load_model_with_fallback(...)
    TypeError: cannot unpack non-iterable NoneType object

Add the missing `return model, tokenizer` after the successful load.

Fixes waybarrios#252, waybarrios#249
@mikepixelmagic-dev
Copy link
Copy Markdown
Contributor Author

Closing as duplicate of #243 which has the same fix. 👍 on getting it merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

load_model_with_fallback missing return statement — model load succeeds but returns None

1 participant