fix sequence parallelism conflict in kimiVL

[ShareLer]

Checklist Before Starting

• Search for similar PR(s).

What does this PR do?

Fix sequence parallelism conflict in kimiVL patch.

Background:
A recent VLM-related PR(#1739 ) has modified the sequence parallelism logic of VLM: Split inputs_embeds after the model's embedding layer instand of spliting input_ids and position_ids before forward.
However, the SP logic I implemented in KimiVL's PR(#1639 ) was still implemented in accordance with the old logic. And split the image token at the combination of image_token and text_token to avoid the problem of 'the Image features and image tokens do not match'.
Since these two PR were developed in parallel which led to logical conflicts after the PR were merged.

High-Level Design

Demonstrate the high-level design if this PR is complex.

Specific Changes

• Delete the patch for _merge_with_image_features which to assign the image token to the corresponding SP rank.
• Adjust the processing related to position_ids in _ulysses_flash_attn_forward.

API

Demonstrate how the API changes if any.

Usage Example

Provide usage example(s) for easier usage.

# Add code snippet or script demonstrating how to use this 

Test

Additional Info.

• Issue Number: Fixes issue # or discussion # if any.
• Training: [Note which backend this PR will affect: FSDP, Megatron, both, or none]
• Inference: [Note which backend this PR will affect: vLLM, SGLang, both, or none]

Checklist Before Submitting

• Read the Contribute Guide.
• Apply pre-commit checks.
• Add [BREAKING] to the PR title if it breaks any API.
• Update the documentation about your changes in the docs.
• New CI unit test(s) are added to cover the code path.
• Rely on existing unit tests on CI that covers the code path.

[eric-haibin-lin]

thanks! would u mind adding a unit test in tests/models/test_transformers_ulysses.py that reproduce this error?

[hiyouga]

LGTM. Now we should split not the image features but the input ids only

[ShareLer]

thanks! would u mind adding a unit test in tests/models/test_transformers_ulysses.py that reproduce this error?

It might not be possible to add the unit test of KimiVL because the unit tests in tests/models/test_transformers_ulysses.py rely on importing the model's config from transformers (ie: 'KimiVLConfig') to create a mock model by from_config, while in transformers the model and config of kimiVL are not defined.

It is necessary to add ulysses-related unit tests for VLM. Perhaps I can first try to add a unit test related to ulysses for QwenVL.

[hiyouga]

@ShareLer The unittest for qwen2vl ulysses already exists

verl/.github/workflows/e2e_ppo_trainer.yml

Lines 200 to 208 in 85fef90

	- name: Running Geo3k VLM PPO E2E training tests on 8 L20 GPUs with rmpad using function rm
	run: |
	ray stop --force
	TRAIN_FILES=$HOME/data/geo3k/train.parquet VAL_FILES=$HOME/data/geo3k/test.parquet \
	MAX_PROMPT_LEN=1536 MAX_RESPONSE_LEN=1536 \
	MODEL_ID=Qwen/Qwen2-VL-2B-Instruct \
	ADV_ESTIMATOR=gae RM_PAD=True USE_KL=True ENABLE_CHUNKED_PREFILL=False \
	SP_SIZE=2 \
	bash tests/e2e/ppo_trainer/run_function_reward.sh

I think current state is sufficient