[Model] Add PP-Chart2Table Model Support

docs/source/en/_toctree.yml

@@ -1284,6 +1284,8 @@
         title: PP-OCRv5_server_det
       - local: model_doc/pp_ocrv5_server_rec
         title: PP-OCRv5_server_rec
+      - local: model_doc/pp_chart2table
+        title: PPChart2Table
       - local: model_doc/pp_lcnet
         title: PPLCNet
       - local: model_doc/pp_lcnet_v3

docs/source/en/model_doc/pp_chart2table.md

@@ -0,0 +1,204 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+*This model was released on 2025-05-20 and added to Hugging Face Transformers on 2026-03-18.*
+
+# PP-Chart2Table
+
+<div class="flex flex-wrap space-x-1">
+<img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-DE3412?style=flat&logo=pytorch&logoColor=white">
+</div>
+
+## Overview
+
+**PP-Chart2Table** is a SOTA multimodal model developed by the PaddlePaddle team, specializing in chart parsing for both Chinese and English. Its high performance is driven by a novel "Shuffled Chart Data Retrieval" training task, which, combined with a refined token masking strategy, significantly improves its efficiency in converting charts to data tables. The model is further strengthened by an advanced data synthesis pipeline that uses high-quality seed data, RAG, and LLMs persona design to create a richer, more diverse training set. To address the challenge of large-scale unlabeled, out-of-distribution (OOD) data, the team implemented a two-stage distillation process, ensuring robust adaptability and generalization on real-world data.
+
+## Model Architecture 
+PP-Chart2Table adopts a multimodal fusion architecture that combines a vision tower for chart feature extraction and a language model for table structure generation, enabling end-to-end chart-to-table conversion.
+
+
+## Usage
+
+### Single input inference
+
+The example below demonstrates how to classify image with PP-Chart2Table using [`Pipeline`] or the [`AutoModel`].
+
+<hfoptions id="usage">
+<hfoption id="Pipeline">
+
+```py
+from transformers import pipeline
+
+pipe = pipeline("image-text-to-text", model="PaddlePaddle/PP-Chart2Table_safetensors")
+
+# PPChart2TableProcessor uses hardcoded "Chart to table" instruction internally via chat template
+conversation = [
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image",
+                "url": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png",
+            },
+        ],
+    },
+]
+result = pipe(text=conversation)
+print(result[0]["generated_text"])
+
+```
+
+</hfoption>
+
+<hfoption id="AutoModel">
+
+```py
+import requests
+from PIL import Image
+from transformers import AutoModelForImageTextToText, AutoProcessor
+
+model_path = "PaddlePaddle/PP-Chart2Table_safetensors"
+model = AutoModelForImageTextToText.from_pretrained(
+    model_path, 
+    device_map="auto",
+)
+processor = AutoProcessor.from_pretrained(model_path)
+
+# PPChart2TableProcessor uses hardcoded "Chart to table" instruction internally via chat template
+conversation = [
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image",
+                "url": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png",
+            },
+        ],
+    },
+]
+
+inputs = processor.apply_chat_template(
+    conversation,
+    tokenize=True, 
+    add_generation_prompt=True, 
+    truncation=True,
+    return_dict=True,
+    return_tensors="pt",
+).to(model.device)
+
+generated_ids = model.generate(**inputs, do_sample=False, max_new_tokens=256)
+generated_ids_trimmed = [out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
+result = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)
+print(result)
+
+```
+
+</hfoption>
+</hfoptions>
+
+### Batched inference
+
+Here is how you can do it with PP-Chart2Table using [`Pipeline`] or the [`AutoModel`]:
+
+<hfoptions id="usage">
+<hfoption id="Pipeline">
+
+```py
+from transformers import pipeline
+
+pipe = pipeline("image-text-to-text", model="PaddlePaddle/PP-Chart2Table_safetensors")
+
+# PPChart2TableProcessor uses hardcoded "Chart to table" instruction internally via chat template
+conversation = [
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image",
+                "url": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png",
+            },
+        ],
+    },
+]
+result = pipe(text=[conversation, conversation])
+print(result[0][0]["generated_text"])
+
+```
+
+</hfoption>
+
+<hfoption id="AutoModel">
+
+```py
+import requests
+from PIL import Image
+from transformers import AutoModelForImageTextToText, AutoProcessor
+
+model_path = "PaddlePaddle/PP-Chart2Table_safetensors"
+model = AutoModelForImageTextToText.from_pretrained(
+    model_path, 
+    device_map="auto",
+)
+processor = AutoProcessor.from_pretrained(model_path)
+
+# PPChart2TableProcessor uses hardcoded "Chart to table" instruction internally via chat template
+conversation = [
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image",
+                "url": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/chart_parsing_02.png",
+            },
+        ],
+    },
+]
+
+batch_conversation = [conversation, conversation]
+inputs = processor.apply_chat_template(
+    batch_conversation,
+    tokenize=True, 
+    add_generation_prompt=True, 
+    truncation=True,
+    return_dict=True,
+    return_tensors="pt",
+).to(model.device)
+
+generated_ids = model.generate(**inputs, do_sample=False, max_new_tokens=256)
+generated_ids_trimmed = [out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
+result = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)
+print(result)
+
+```
+
+</hfoption>
+</hfoptions>
+
+
+## PPChart2TableConfig
+
+[[autodoc]] PPChart2TableConfig
+
+## PPChart2TableImageProcessor
+
+[[autodoc]] PPChart2TableImageProcessor
+
+## PPChart2TableImageProcessorPil
+
+[[autodoc]] PPChart2TableImageProcessorPil
+
+## PPChart2TableProcessor
+
+[[autodoc]] PPChart2TableProcessor

src/transformers/conversion_mapping.py

@@ -80,6 +80,7 @@
     "mllama": "llava",
     "qwen2_5_vl": "qwen2_vl",
     "sam3_tracker_video": "sam3_tracker",
+    "pp_chart2table": "got_ocr2",
 }
 
 

src/transformers/models/__init__.py

@@ -321,6 +321,7 @@
     from .plbart import *
     from .poolformer import *
     from .pop2piano import *
+    from .pp_chart2table import *
     from .pp_doclayout_v2 import *
     from .pp_doclayout_v3 import *
     from .pp_lcnet import *

src/transformers/models/auto/configuration_auto.py

@@ -359,6 +359,7 @@
         ("plbart", "PLBartConfig"),
         ("poolformer", "PoolFormerConfig"),
         ("pop2piano", "Pop2PianoConfig"),
+        ("pp_chart2table", "PPChart2TableConfig"),
         ("pp_doclayout_v2", "PPDocLayoutV2Config"),
         ("pp_doclayout_v3", "PPDocLayoutV3Config"),
         ("pp_lcnet", "PPLCNetConfig"),
@@ -879,6 +880,7 @@
         ("plbart", "PLBart"),
         ("poolformer", "PoolFormer"),
         ("pop2piano", "Pop2Piano"),
+        ("pp_chart2table", "PPChart2Table"),
         ("pp_doclayout_v2", "PPDocLayoutV2"),
         ("pp_doclayout_v3", "PPDocLayoutV3"),
         ("pp_lcnet", "PPLCNet"),

src/transformers/models/auto/image_processing_auto.py

@@ -194,6 +194,10 @@
             ("pixio", {"torchvision": "BitImageProcessor", "pil": "BitImageProcessorPil"}),
             ("pixtral", {"torchvision": "PixtralImageProcessor", "pil": "PixtralImageProcessorPil"}),
             ("poolformer", {"torchvision": "PoolFormerImageProcessor", "pil": "PoolFormerImageProcessorPil"}),
+            (
+                "pp_chart2table",
+                {"torchvision": "PPChart2TableImageProcessor", "pil": "PPChart2TableImageProcessorPil"},
+            ),
             ("pp_doclayout_v2", {"torchvision": "PPDocLayoutV2ImageProcessor"}),
             ("pp_doclayout_v3", {"torchvision": "PPDocLayoutV3ImageProcessor"}),
             ("pp_lcnet", {"torchvision": "PPLCNetImageProcessor"}),

src/transformers/models/auto/modeling_auto.py

@@ -995,6 +995,7 @@ class _BaseModelWithGenerate(PreTrainedModel, GenerationMixin):
         ("pi0", "PI0ForConditionalGeneration"),
         ("pix2struct", "Pix2StructForConditionalGeneration"),
         ("pixtral", "LlavaForConditionalGeneration"),
+        ("pp_chart2table", "GotOcr2ForConditionalGeneration"),
         ("qwen2_5_vl", "Qwen2_5_VLForConditionalGeneration"),
         ("qwen2_vl", "Qwen2VLForConditionalGeneration"),
         ("qwen3_5", "Qwen3_5ForConditionalGeneration"),

src/transformers/models/auto/processing_auto.py

@@ -137,6 +137,7 @@
             ("pix2struct", "Pix2StructProcessor"),
             ("pixtral", "PixtralProcessor"),
             ("pop2piano", "Pop2PianoProcessor"),
+            ("pp_chart2table", "PPChart2TableProcessor"),
             ("qwen2_5_omni", "Qwen2_5OmniProcessor"),
             ("qwen2_5_vl", "Qwen2_5_VLProcessor"),
             ("qwen2_audio", "Qwen2AudioProcessor"),

src/transformers/models/pp_chart2table/__init__.py

@@ -0,0 +1,29 @@
+# Copyright 2026 The HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import TYPE_CHECKING
+
+from ...utils import _LazyModule
+from ...utils.import_utils import define_import_structure
+
+
+if TYPE_CHECKING:
+    from .configuration_pp_chart2table import *
+    from .image_processing_pp_chart2table_fast import *
+    from .processing_pp_chart2table import *
+else:
+    import sys
+
+    _file = globals()["__file__"]
+    sys.modules[__name__] = _LazyModule(__name__, _file, define_import_structure(_file), module_spec=__spec__)