[llama-vision] Remove token_idx_cpu parameter

[ugolowic]

It turns out that in the configuration with hpu_graphs, an int-type parameter token_idx_cpu passed to mllama's forward() method causes cache misses in https://github.com/habana-internal/pytorch-integration/blob/GAUDI_v1.21.0_555/python_packages/habana_frameworks/torch/hpu/graphs.py#L618. It's due to the fact that the integer's value changes at each forward call, and the hashing function returns different hash. This leads to the graph not being reused from cache and, in consequence, a performance drop.

Example command:

PT_HPU_LAZY_MODE=1  python3  examples/image-to-text/run_pipeline.py --model_name_or_path meta-llama/Llama-3.2-11B-Vision-Instruct --use_hpu_graphs --limit_hpu_graphs --use_kv_cache --max_new_tokens 2048 --bf16 --batch_size 28  --use_flash_attention --flash_attention_recompute    --sdp_on_bf16 --logits_bf16  --max_input_tokens 128    --ignore_eos --trim_logits

@kalyanjk

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[astachowiczhabana]

Hi @IlyasMoutawwakil can we merge this one for 1.18?