[Bug] model_init_kwargs not used in InvernVLModule. get_model_class

[YongchengYAO]

Code:
https://github.com/om-ai-lab/VLM-R1/blob/00755a23ed1848641d2d14130824d843a080a09c/src/open-r1-multimodal/src/open_r1/vlm_modules/internvl_module.py#L26C2-L39C25

def get_model_class(self, model_id: str, model_init_kwargs: dict):
        assert "InternVL" in model_id, f"model_id must contain 'InternVL', but got {model_id}"
        self.model_config = AutoConfig.from_pretrained(model_id, trust_remote_code=True)
        # The model class of InternVL when being mapped has been determined by its config
        model_cls = AutoModel
        # InternVL should be inputted with "trust_remote_code=True"
        model_init_kwargs["trust_remote_code"] = True
        # "use_cache" should be removed
        model_init_kwargs.pop("use_cache", None)
        # "flash_attention_2" should be modified to "use_flash_attn" in InternVL
        if "flash_attention_2" in model_init_kwargs.get("attn_implementation", ""):
            model_init_kwargs["use_flash_attn"] = True
            model_init_kwargs.pop("attn_implementation")
        return model_cls

[YongchengYAO]

I guess you wanna modify model_init_kwargs and use it for model loading.

Could return the kwargs and modify this

model_init_kwargs , model_cls = self.vlm_module.get_model_class(model_id, model_init_kwargs)
model = model_cls.from_pretrained(model_id, **model_init_kwargs)