Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] add pytorch kvint testcase into function regresstion #2584

Merged
merged 43 commits into from
Oct 16, 2024

Conversation

zhulinJulia24
Copy link
Collaborator

@zhulinJulia24 zhulinJulia24 commented Oct 11, 2024

after #2438

  1. [tested] add pytorch kvint testcase
  2. [tested] review test config according to support models, most models supported by turbomind are compatible with kvint and 4-bit quantization. Modify the configuration YAML file to make it easy to maintain.
  3. [testing] add llama3.2 model into pytorch backend

@zhulinJulia24
Copy link
Collaborator Author

@lvhan028 Qwen2-VL does not supported by turbomind backend

error is

lmdeploy convert qwen /nvme/qa_test_models/Qwen/Qwen2-VL-2B-Instruct --dst-path /nvme/qa_test_models/autotest_model/workspace_Qwen/Qwen2-VL-2B-Instruct

2024-10-12 09:44:10,786 - lmdeploy - WARNING - converter.py:324 - The argument `<model_name>` is deprecated and unused now. It will be removed on 2024.12.31. It was originally used to specify the name of the built-in chat template, but now it is substituted with a clearer parameter `--chat-template`
Unrecognized keys in `rope_scaling` for 'rope_type'='default': {'mrope_section'}
Traceback (most recent call last):
  File "/opt/py3/bin/lmdeploy", line 8, in <module>
    sys.exit(run())
  File "/opt/py3/lib/python3.10/site-packages/lmdeploy/cli/entrypoint.py", line 42, in run
    args.run(args)
  File "/opt/py3/lib/python3.10/site-packages/lmdeploy/cli/cli.py", line 165, in convert
    main(**kwargs)
  File "/opt/py3/lib/python3.10/site-packages/lmdeploy/turbomind/deploy/converter.py", line 334, in main
    assert is_supported(model_path), (
AssertionError: turbomind does not support /nvme/qa_test_models/Qwen/Qwen2-VL-2B-Instruct. Plz try pytorch engine instead.

Should we update this table
image

autotest/config.yaml Outdated Show resolved Hide resolved
@lvhan028 lvhan028 merged commit b689cbc into InternLM:main Oct 16, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants