can anyone share the finetune procedure of Janus-pro?

[hl0737]

I implement the finetune code myself according to the paper, but

when i sft the janus-pro 7b or 1b, The loss started at around 5, dropped to approximately 4.7, and then stopped decreasing further. I use about 500k data samples, pure t2i, not blend of understanding data

Worse still, the fine-tuned model drops a lot on the geneval, especially for 'two objects', 'position', and 'color' attributes.

anyone has some insight ? thx ~~ 😭😭

[husnejahan]

https://github.com/sunshine-JLU/deepseek-janus-pro-lora/blob/main/deepseek-janus-pro-lora.ipynb