Skip to content

docs(offload): clarify LMCache MP lookup timeout scope - #2442

Open
kvnloo wants to merge 1 commit into
ROCm:mainfrom
kvnloo:docs/lmcache-mp-timeout-scope-main
Open

kvnloo wants to merge 1 commit into
ROCm:mainfrom
kvnloo:docs/lmcache-mp-timeout-scope-main

Conversation

@kvnloo

@kvnloo kvnloo commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Summary

The earlier docs follow-up (#2407) was stacked on the native-MP feature branch and closed when that branch merged. Its remaining timeout clarification is not present on current main.

This one-file follow-up documents the control flow that still exists on main:

  • lmcache.mp.mq_timeout belongs to LMCache adapter message-queue operations;
  • lmcache.mp.lookup_timeout starts after lookup submission returns and bounds ATOM's polling loop;
  • lmcache.mp.lookup_poll_interval controls sleeps between polls.

Therefore lookup_timeout is not a hard wall-clock bound around a blocking adapter submission or status call.

No runtime/default/topology behavior changes.

Evidence

The original deterministic CPU characterization is here:
https://github.com/kvnloo/ATOM/actions/runs/36160612338

That run exercised the real lookup facade with a fake adapter/clock and completed 65 MP tests. It demonstrated that submission time is outside the outer lookup polling deadline and that a completed status call is accepted before the next deadline check.

I re-read current main at 922b3519; _MPLookupClient.lookup() still submits first, sets the deadline afterward, then checks adapter status before checking the deadline.

No live server/GPU timeout behavior is claimed.

AI-assisted source review and drafting.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

  • ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

  • ✅ Run after the PR is approved and Pre Checkin passes
  • ✅ Run immediately when an approval review is submitted
  • ✅ Can be requested before approval with labels
Label Tests
ci:full Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom Run native ATOM model accuracy tests
ci:vllm Run ATOM vLLM OOT model accuracy tests
ci:sglang Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 2442 --add-label <label>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants