Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions examples/online_serving/minicpmo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,42 @@ generation records `clock=media`; artifacts generated with
`--pace as-fast-as-possible` record `clock=invalid` and evaluation rejects
them unless `--allow-invalid-clock` is explicit.

### Judge context overflow with sampled frames

`--judge-frame-overflow error` is the default for image-frame requests: a
rejected request fails without reducing its frames. To opt into reduced-frame recovery, add
`--judge-frame-overflow reduce` to `evaluate`, normally together with
`--judge-video-mode frame-sample`.

The policy applies to both temporal image-frame calls and frame-sample content
calls. It retries only explicit context-length errors returned as HTTP 400 or
500. Each retry uses approximately half as many frames, sampled uniformly from
the original timeline. Multiple retained frames preserve the endpoints; a
single retained frame uses the original middle frame. Ordinary HTTP failures,
rate limits, transport timeouts, and an overflow with only one frame remaining
still propagate. The existing `video_url` context-overflow fallback to sampled
frames is preserved. If that fallback's frame request also overflows, it only
reduces frames with `--judge-frame-overflow reduce`; `error` propagates the
frame-request failure. Temporal judging always uses image frames and follows
the same explicit policy.

This is failure recovery, **not proactive token budgeting**. It does not resize
images, estimate the judge's tokenizer, guarantee that one image fits, or recover
an oversized request that only times out instead of returning a recognized
context-length error.

Each score records `judge_frame_overflow`. Content and temporal frame results
record `frame_count_initial`, `frame_count`, and `frame_budget_retries` (the number
of recovery retries, not a measured token budget). The RTD summary reports
`judge_frame_overflow_policies` and `frame_reduction_samples`.

Reduced frames can omit short events or useful OCR evidence and therefore change
the evaluation workload. Use a separate `--score-root` for each policy/model
configuration, or explicitly use `--overwrite`; otherwise existing score files
are reused. Do not compare mixed-policy summaries as though they used identical
inputs. No claim is made that reduced-frame scores equal the original protocol's
scores.

## Related examples

- [Offline MiniCPM-o inference](../../offline_inference/minicpmo/)
Expand Down
Loading
Loading