[janus] Fix failing tests on mi3XX

[remi-or]

This PR fixes a few test fails of the janus model on MI3XX:

• an AttributeError when trying to retrieve BOI token in a generation_kwargs attribute that is not guaranteed to be in the config;
• three multiple devices errors (device map start with the wrong module, module split when it should not be, inputs on different devices)
• a test fail due to numerical difference, fixed that with Expectations

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[yaswanth19]

Hi @remi-or , The other changes look logical to me, thanks for fixing them 🤗 . Can you expand on why boi_token_id won't be present in the generation_kwargs coz AFAIK I have added it explicitly in conversion file; hence should be present in checkpoints 🤔

[remi-or]

Hi @yaswanth19 , it seems like it is missing from the checkpoint used in testing (deepseek-community/Janus-Pro-1B) .

[yaswanth19]

I can see in the checkpoints: https://huggingface.co/deepseek-community/Janus-Pro-1B/blob/main/generation_config.json

[zucchini-nlp]

same question

[remi-or]

It seems like generation_config is either 1. not loaded, which would be weird because guidance_scale is equal to 5 (but idk if that's usual) or 2. the generation_kwargs attribute is dropped at some point. Before model.generate is called in the test, if I check the value of model.generation_config I get:

model.generation_config = GenerationConfig {
  "bos_token_id": 100000,
  "eos_token_id": 100001,
  "pad_token_id": 100002
}

[zucchini-nlp]

@remi-or do you have a snippet reproducing the issue, would be nice to add it in the PR body as well 🤗

I wonder if the issue you describe appears only on certain hardware (which is very unlikely) or the inference script is doing smth unexpected

[remi-or]

Sure! It's taken from tests/models/janus/test_modeling_janus.py::JanusIntegrationTest::test_model_generate_image :

from transformers import AutoProcessor, JanusForConditionalGeneration

if __name__ == "__main__":

    model_id = "deepseek-community/Janus-Pro-1B"
    model = JanusForConditionalGeneration.from_pretrained(model_id, device_map="auto")
    processor = AutoProcessor.from_pretrained(model_id)

    inputs = processor(
        text=["A portrait of young girl. masterpiece, film grained, best quality."],
        padding=True, generation_mode="image", return_tensors="pt",
    ).to(model.device)

    out = model.generate(**inputs, generation_mode="image", do_sample=False)

I tried changing device_map to CPU and it still crashed with AttributeError: 'GenerationConfig' object has no attribute 'generation_kwargs' so I dont think it's device-related.

[zucchini-nlp]

indeed, weird since I'd assume the config from the hub would be picked up. At least that was try for Whisper in the past. Let me check why this isn't loaded, we better make sure the pre-saved config values are used when running inference

[zucchini-nlp]

Fixed, the issue was in the config saved in the hub. One of the flags was set to True thus overwriting config values from scratch

I think now the only issue is the multi-device inferece. @remi-or can you update the PR so we can merge?

[remi-or]

Done!

[yaswanth19]

CC: @zucchini-nlp

[zucchini-nlp]

same question

[zucchini-nlp]

Great thanks!