Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hotfix] Fix get_ppl #3023

Merged
merged 4 commits into from
Jan 14, 2025
Merged

[hotfix] Fix get_ppl #3023

merged 4 commits into from
Jan 14, 2025

Conversation

lvhan028
Copy link
Collaborator

OC test with the following config failed

from copy import deepcopy
from mmengine.config import read_base
with read_base():
    # choose a list of datasets
    from opencompass.configs.datasets.ARC_c.ARC_c_few_shot_ppl import \
        ARC_c_datasets  # noqa: F401, E501
    from opencompass.configs.models.hf_internlm.lmdeploy_internlm2_5_7b import \
        models as lmdeploy_internlm2_5_7b  # noqa: F401, E501
turbomind_internlm2_5_7b = deepcopy(*lmdeploy_internlm2_5_7b)
for model in [v for k, v in locals().items() if k.endswith('_4bits')]:
    model['engine_config']['model_format'] = 'awq'
    model['abbr'] = model['abbr'] + '_4bits'
    model['path'] = model['path'] + '-inner-4bits'
for model in [v for k, v in locals().items() if '_batch1' in k]:
    model['abbr'] = model['abbr'] + '_batch1'
    model['engine_config']['max_batch_size'] = 1
    model['batch_size'] = 1
models = [turbomind_internlm2_5_7b]
datasets = [*ARC_c_datasets]
File "/nvme/envs/vllmtest/lib/python3.10/site-packages/opencompass/tasks/openicl_infer.py", line 161, in <module>
    inferencer.run()
  File "/nvme/envs/vllmtest/lib/python3.10/site-packages/opencompass/tasks/openicl_infer.py", line 89, in run
    self._inference()
  File "/nvme/envs/vllmtest/lib/python3.10/site-packages/opencompass/tasks/openicl_infer.py", line 139, in _inference
    inferencer.inference(retriever,
  File "/nvme/envs/vllmtest/lib/python3.10/site-packages/opencompass/openicl/icl_inferencer/icl_ppl_inferencer.py", line 159, in inference
    sub_res = self.model.get_ppl_from_template(sub_prompt_list).tolist()
  File "/nvme/envs/vllmtest/lib/python3.10/site-packages/opencompass/models/base.py", line 174, in get_ppl_from_template
    return self.get_ppl(inputs, mask_length)
  File "/nvme/envs/vllmtest/lib/python3.10/site-packages/opencompass/models/turbomind.py", line 196, in get_ppl
    results.append(self.pipe.get_ppl(input_ids[i:i + 128]))
  File "/nvme/envs/vllmtest/lib/python3.10/site-packages/lmdeploy/serve/utils.py", line 98, in get_ppl
    res = self._get_ppl(
  File "/nvme/envs/vllmtest/lib/python3.10/site-packages/lmdeploy/serve/utils.py", line 199, in _get_ppl
    _logits = _logits.float()
AttributeError: 'NoneType' object has no attribute 'float'

@lvhan028 lvhan028 merged commit b77e8b4 into InternLM:main Jan 14, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants