Skip to content

docs(offload): clarify LMCache MP lookup timeout scope - #2407

Closed
kvnloo wants to merge 3 commits into
ROCm:feat/dsv4-lmcache-mpfrom
kvnloo:docs/lmcache-mp-recipe-clarifications
Closed

kvnloo wants to merge 3 commits into
ROCm:feat/dsv4-lmcache-mpfrom
kvnloo:docs/lmcache-mp-recipe-clarifications

Conversation

@kvnloo

@kvnloo kvnloo commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Docs-only follow-up stacked on #2250's current head 94c07412.

The #2250 owner already integrated the separately reported --eviction-policy LRU server-recipe correction. This PR therefore contains only one remaining clarification: document the timeout scopes already implemented by native MP.

  • lmcache.mp.mq_timeout is passed to the LMCache scheduler/worker adapters for message-queue operations.
  • lmcache.mp.lookup_timeout starts after lookup submission returns and bounds ATOM's polling loop while the tier keeps returning no answer.
  • lmcache.mp.lookup_poll_interval controls the sleep between polls.

So lookup_timeout is not a hard wall-clock bound on a blocking adapter submission or status call.

No topology, lifetime, scheduler, runtime-default, server recipe, or LMCache dependency change.

Evidence

CPU characterization run exercised the real ATOM lookup facade with a deterministic adapter/clock alongside the existing MP tests:

  • 65 MP tests passed, no failures/skips.
  • A submission consuming 8 fake seconds followed by a 3-second polling budget does not time out until the post-submission polling deadline.
  • A status call consuming 8 fake seconds and then returning a hit is accepted before the outer polling deadline is checked again.

I refreshed the source against #2250 head 94c07412; the lookup control flow remains unchanged for this contract.

The branch is merged forward with owner changes rather than force-pushed. Net diff against #2250 remains one README file, +8/−0.

AI-assisted source review and drafting; no server/GPU timeout behavior claimed.

@kvnloo kvnloo changed the title docs(offload): correct LMCache MP server and timeout recipe docs(offload): clarify LMCache MP lookup timeout scope Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants