Skip to content

Conversation

AnikiFan
Copy link
Contributor

I have moved multichoice_continuations_start_space from generation parameter to base_params, since it is one of the arguments of TransformersModelConfig instead of GenerationParameters.
Before changing it, run the command
'lighteval accelerate <path to lighteval/examples/model_configs/transformers_model.yaml> "leaderboard|arc:challenge|0|0"' will get below error:

[2025-03-11 09:19:30,141] [    INFO]: PyTorch version 2.6.0 available. (config.py:54)
[2025-03-11 09:19:33,984] [ WARNING]: Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher. (other.py:335)
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ lighteval/src/lighteval/main_accelerate.py:162 in accelerate                │
│                                                                                                  │
│   159 │   │   # We extract the model args                                                        │
│   160 │   │   args_dict = {k.split("=")[0]: k.split("=")[1] for k in config["base_params"]["mo   │
│   161 │   │                                                                                      │
│ ❱ 162 │   │   args_dict["generation_parameters"] = GenerationParameters.from_dict(config)        │
│   163 │   │                                                                                      │
│   164 │   │   # We store the relevant other args                                                 │
│   165 │   │   args_dict["base_model"] = config["merged_weights"]["base_model"]                   │
│                                                                                                  │
│ lighteval/src/lighteval/models/model_input.py:60 in from_dict               │
│                                                                                                  │
│    57 │   │   │   │   }                                                                          │
│    58 │   │   │   }                                                                              │
│    59 │   │   """                                                                                │
│ ❱  60 │   │   return GenerationParameters(**config_dict.get("generation", {}))                   │
│    61 │                                                                                          │
│    62 │   @classmethod                                                                           │
│    63 │   def from_model_args(cls, model_args: str):                                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: GenerationParameters.__init__() got an unexpected keyword argument 'multichoice_continuations_start_space'

AnikiFan-copliot and others added 2 commits March 11, 2025 09:00
Copy link
Member

@NathanHB NathanHB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix !

@HuggingFaceDocBuilderDev
Copy link
Collaborator

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@NathanHB NathanHB merged commit d9d9086 into huggingface:main Mar 24, 2025
3 checks passed
@AnikiFan
Copy link
Contributor Author

🤗

@AnikiFan AnikiFan deleted the fix_multichoice_continuations_start_space branch March 24, 2025 15:38
hynky1999 pushed a commit that referenced this pull request May 22, 2025
move multichoice_continuations_start_space from generation
parameter to base_params
NathanHB pushed a commit that referenced this pull request Sep 19, 2025
move multichoice_continuations_start_space from generation
parameter to base_params
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants