[Model] Serve DeepSeek-OCR-2 with its official 768px local-crop geometry

[chaijiacheng888]

[Model] Serve DeepSeek-OCR-2 with its official 768px local-crop geometry

Motivation

DeepSeek-OCR-2 is served with the 640px local-crop geometry inherited from the DeepSeek-OCR (v1)
processor. Its own geometry is 768px: a 768px crop yields (768 // 16 // 4) ** 2 == 144 visual
tokens, the length of the tuned query_768 table, while 640px yields 100, which matches no tuned
table and falls back to F.interpolate query embeddings. The model's own paper states the page
budget this produces — V-token 1120 for OCR-2 against 1156 for OCR-1, i.e. 6 x 144 + 256 and
9 x 100 + 256.

#17897 ported the DeepEncoder V2 classes and query tables, but the shared processor kept OCR-1's
crop size and two thresholds compared against OCR-1's 640 as a literal, so OCR-2 ran off-design in
three places, all inside tokenize_with_images:

1. the crop step passed the IMAGE_SIZE constant, so tiles stayed 640px no matter what
   image_size said — this is what actually keeps the 768px crop from happening;
2. if img_w <= 640 and img_h <= 640: — the small-image early-out, so 641–768px pages were tiled
   at all;
3. if self.image_size <= 640 and not cropping: — the global-view guard, so >2-image requests
   skipped the resize the official processor still performs.

All three are silent. Restoring the official geometry halves text Edit: 0.0985 → 0.0533 on Ascend
and 0.0892 → 0.0519 on RTX 4090, with every other metric improving too (tables below).

Modifications

Geometry has a single source of truth, selected per checkpoint from the model config. Neither
checkpoint carries the crop size — both ship identical processor configs whose
candidate_resolutions=[[1024, 1024]] is the global base.

• python/sglang/srt/configs/deepseek_ocr.py: add OCR2_IMAGE_SIZE = 768 and
  is_ocr2_config() / local_crop_size(); pass the selected size into the crop step and the
  small-image early-out; reduce the global-view guard to if not cropping:. OCR-1 is byte-identical.
• python/sglang/srt/multimodal/processors/deepseek_ocr.py: add apply_ocr_geometry(), which
  patches ocr2_mode and image_size onto the HF processor — Transformers builds it before SGLang
  sees it, so this cannot be a constructor argument. The _is_ocr2 predicate moves down to
  configs/ and is shared by both layers.
• python/sglang/srt/models/deepseek_ocr.py: import that predicate instead of defining it.
• test/registered/unit/multimodal/test_deepseek_ocr_geometry.py (new, CPU): pins the 768/640
  selection, the patching, and that dynamic_preprocess follows the image_size it is given.

Accuracy Tests

OmniDocBench, page level, quick_match. The blocks are scored version-matched: the v1.5 block on
the official v1.5 dataset with the v1.5 scorer, the v1.6 block on the v1.6 dataset with the v1.6
scorer. The datasets are not interchangeable, and v1.6 replaced the CDM implementation, so a CDM
number only means something within one block.

v1.5, 1355 pages, Ascend 910B (1x, bfloat16):

	Overall ↑	text Edit ↓	TEDS ↑	TEDS-S ↑	CDM ↑	Read Order ↓
640 (pre-PR)	87.91	0.1043	85.56	90.13	88.60	0.0916
768	91.17	0.0516	87.98	92.27	90.70	0.0587
Δ	+3.26	−0.0527	+2.42	+2.14	+2.10	−0.0329
OmniDocBench_v1.5	89.17	0.049	85.60	90.06	86.85	0.060
Deepseek_OCR2 PDF	91.09	0.048	87.75	92.06	90.31	0.057

v1.6, 1651 pages:

	Overall ↑	text Edit ↓	TEDS ↑	TEDS-S ↑	CDM ↑	Read Order ↓
RTX 4090 — 640	89.76	0.0892	84.26	88.61	93.94	0.1619
RTX 4090 — 768	91.75	0.0519	86.39	89.78	94.06	0.1430
Δ	+2.00	−0.0373	+2.14	+1.17	+0.12	−0.0189
Ascend 910B — 640	89.10	0.0985	84.03	88.00	93.11	0.1715
Ascend 910B — 768	91.95	0.0533	86.70	90.17	94.48	0.1444
Δ	+2.85	−0.0452	+2.67	+2.17	+1.37	−0.0271
OmniDocBench_v1.6	90.25	0.050	83.89	87.75	91.84	0.144

Both platforms agree: the two 768 rows are 0.20 apart and the two 640 rows 0.66.

# server
python -m sglang.launch_server \
    --model-path deepseek-ai/DeepSeek-OCR-2 --enable-multimodal \
    --context-length 8192 --allow-auto-truncate --enable-custom-logit-processor

# score (v1.6 block; the v1.5 block is the same with the v1_5 dataset and scorer)
docker run --rm -v /path/to/data:/data -v /path/to/out:/workspace/result \
    ghcr.io/zeng-weijun/omnidocbench-eval:repro-ubuntu2204 --config /data/cfg_v16.yaml

# cfg_v16.yaml
end2end_eval:
  metrics:
    text_block: {metric: [Edit_dist]}
    display_formula: {metric: [Edit_dist, CDM], cdm_workers: 13}
    table: {metric: [TEDS, Edit_dist], teds_workers: 13}
    reading_order: {metric: [Edit_dist]}
  dataset:
    dataset_name: end2end_dataset
    ground_truth: {data_path: /data/OmniDocBench_v16_clean.json}
    prediction:   {data_path: /data/pred_md}
    match_method: quick_match
    match_workers: 13
    quick_match_truncated_timeout_sec: 300
    match_timeout_sec: 420
    timeout_fallback_max_chunk_span: 10
    timeout_fallback_order_penalty: 0.10

The ground truth is the released v1.6 dataset with 37 dangling relation_type="truncated"
reading-order edges dropped; the official scorer raises on the first one. Only the reading-order
metric on those 12 pages is affected.

Speed Tests and Profiling

Preprocessing-only change. A 768px crop carries 144 visual tokens instead of 100, while 641–768px
pages drop from 657 tokens to 257 (no local crops), so the cost depends on the page mix.

End-to-end, all 1651 pages through the RTX 4090 server at 12 concurrent requests: ~30 minutes
either way, 0.9 pages/s, indistinguishable.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Notes

• The server must run with --context-length 8192 + --allow-auto-truncate: a 768px page expands
  to ~1100 image tokens and long pages can emit up to 8192, so the output budget is clamped to the
  remaining context rather than the request being rejected.

---

CI States

Latest PR Test (Base): ❌ Run #35303604206
Latest PR Test (Extra): ❌ Run #35303604121
Latest PR Test (AMD ROCm 10): ❌ Run #35303604233

[sglang-npu-bot]

/tag-and-rerun-ci

[cx22757]

/rerun-failed-ci

[chaijiacheng888]

/rerun-failed-ci

[chaijiacheng888]

/rerun-failed-ci

[chaijiacheng888]

/rerun-failed-ci

[chaijiacheng888]

/rerun-failed-ci

[chaijiacheng888]

/rerun-failed-ci

[sglang-npu-bot]

only failed test/registered/utils/test_numa_utils.py, and it repaired by 40055