[Model][VLM] Support Bee-8B Model

[uyzhang]

Purpose

Support Bee-8B-RL and Bee-8B-SFT

Example Serving Command

vllm serve \
    Open-Bee/Bee-8B-RL \
    --served-model-name bee-8b-rl \
    --tensor-parallel-size 8 \
    --gpu-memory-utilization 0.8 \
    --host 0.0.0.0 \
    --port 8000 \
    --trust-remote-code

Example Offline Inference

import os
from transformers import AutoProcessor
from vllm import LLM, SamplingParams
from PIL import Image
import requests
from io import BytesIO


def load_image(image_path):
    """Load image from URL or local path"""
    if image_path.startswith(('http://', 'https://')):
        response = requests.get(image_path, timeout=10)
        response.raise_for_status()
        image = Image.open(BytesIO(response.content))
    else:
        image = Image.open(image_path)

    # Convert RGBA to RGB if needed
    if image.mode == "RGBA":
        background = Image.new('RGB', image.size, (255, 255, 255))
        background.paste(image, mask=image.split()[-1])
        image = background

    return image.convert("RGB")


def main():

    model_path = "Open-Bee/Bee-8B-RL"

    llm = LLM(
        model=model_path,
        limit_mm_per_prompt={"image": 5},
        trust_remote_code=True,
        tensor_parallel_size=1,
        gpu_memory_utilization=0.8,
    )

    sampling_params = SamplingParams(
        temperature=0.8,
        max_tokens=16384,
    )

    image_url = "http://images.cocodataset.org/val2017/000000039769.jpg"
    image = load_image(image_url)
    text = "Describe this image."

    messages = [
        {
            "role":
            "user",
            "content": [
                {
                    "type": "image",
                    "image": image
                },
                {
                    "type": "text",
                    "text": text
                },
            ],
        },
    ]

    processor = AutoProcessor.from_pretrained(model_path,
                                              trust_remote_code=True)
    prompt = processor.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    mm_data = {"image": image}
    llm_inputs = {
        "prompt": prompt,
        "multi_modal_data": mm_data,
    }

    outputs = llm.generate([llm_inputs], sampling_params=sampling_params)
    generated_text = outputs[0].outputs[0].text

    print(generated_text)


if __name__ == '__main__':
    main()

[mergify]

Documentation preview: https://vllm--27012.org.readthedocs.build/en/27012/

[gemini-code-assist]

Code Review

This pull request adds support for the Bee-8B model. The implementation correctly integrates the model into vLLM by inheriting from existing LLaVA-like model classes and providing a model-specific multimodal projector and processing logic. The changes also include documentation updates, examples, and tests. I've found one critical compatibility issue in the implementation that needs to be addressed.

[ywang96]

Thanks for your contribution! LGTM