Skip to content

[Doc] Conductor docs refresh - #4824

Merged
ykwd merged 2 commits into
kvcache-ai:mainfrom
Chase-Rong:doc/conductor-docs-refresh
Oct 23, 2026
Merged

ykwd merged 2 commits into
kvcache-ai:mainfrom
Chase-Rong:doc/conductor-docs-refresh

Conversation

@Chase-Rong

@Chase-Rong Chase-Rong commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Description

This PR rewrites both pages from the current implementation and fills smaller
gaps in the related pages. Documentation only, no code changes.

design/conductor/conductor-architecture-design.md (rewritten)

  • Components: EventManager, ConductorService (shared by HTTP and RPC),
    ZMQClient, KVEventHandler, PrefixCacheTable, and the C++/Python/Rust
    clients.
  • Data model: context key (tenant_id, model_name, lora_name, block_size), one
    hash profile per context, the 200000-entry cap per context, and the
    device / cpu_local / cpu_share / disk tiers with the medium-to-tier
    mapping for vLLM, SGLang, and Mooncake events.
  • Hash profiles (vllm_v1, sglang, sglang_bigram) and how block coverage
    differs between them.
  • Event flow, clear scope per publisher type, gap detection, replay over
    DEALER, stale sources, and reconnect.
  • Query flow with cumulative tiers and clamping to the queried token count.
  • Config keys (http_server_port, rpc_server_port, kvevent_instance),
    environment variables, the WITH_CONDUCTOR CMake build, and the current
    directory layout.

api-reference/http/conductor-indexer.md (rewritten)

  • Encoding is MessagePack only, with Content-Type: application/msgpack.
    Requests are validated strictly, and errors use the
    {error, reason, field, index} format.
  • /register documents the required hash_profile and the optional
    cache_group. The additionalsalt field, which does not exist, is removed.
  • /query documents the actual response shape:
    {"instances": {id: {longest_matched, dp, rank_matches, device, cpu_local, cpu_share, disk}}}. The old doc showed a response keyed by tenant with
    GPU/CPU/DISK fields.
  • Documents GET /global_view and GET /services. Removes /query_by_hash,
    the lora_id / block_hash aliases, and the XXH3 hashing section, none of
    which are implemented.
  • Replaces the Store master publisher section. The master does not parse
    object keys into seq_hashes, and it does emit model_name /
    block_size / lora_name / dp_rank from its flags. The section now
    explains how Conductor derives block identity from object_key under each
    hash profile, and the steps to index a master stream. The
    (mooncake-store-master-publisher)= anchor is kept because the Store
    deployment guide links to it.

Module

  • Transfer Engine (mooncake-transfer-engine)
  • Mooncake Store (mooncake-store)
  • Mooncake Conductor (mooncake-conductor)
  • Reshard (mooncake-reshard)
  • Mooncake EP (mooncake-ep)
  • Mooncake PG (mooncake-pg)
  • Integration (mooncake-integration)
  • P2P Store (mooncake-p2p-store)
  • Python Wheel (mooncake-wheel)
  • Common (mooncake-common)
  • Mooncake RL (mooncake-rl)
  • CI/CD
  • Docs
  • Other

Type of Change

  • Bug fix
  • New feature
  • Refactor
  • Breaking change
  • Documentation update
  • Performance improvement
  • Other

How Has This Been Tested?

Built the docs locally and checked the rendered conductor pages, the mermaid
diagram, and the mooncake-store-master-publisher anchor. The build produced
no new warnings; the only warnings were intersphinx inventory fetch failures
caused by the local network. Ran pre-commit on the changed files.

Test commands:

cd docs && python -m sphinx -b html source build/html
pre-commit run --files $(git diff --name-only origin/main HEAD)

Test results:

  • Unit tests pass
  • Integration tests pass (if applicable)
  • Manual testing done (describe below)

Every documented field name, default, route, and error code was checked against
mooncake-conductor/src (event_manager.cpp, config.cpp, main.cpp,
event_handler.cpp, zmq_client.cpp, hash_strategy.cpp) and
mooncake-store/src/kv_event/kv_event_publisher.cpp.

Checklist

  • I have performed a self-review of my own code
  • I have formatted my code using ./scripts/code_format.sh
  • I have run pre-commit on the files changed in this PR and all hooks pass
  • I have updated the documentation (if applicable)
  • I have added tests to prove my changes are effective
  • For changes >500 LOC: I have filed an RFC issue

AI Assistance Disclosure

  • No AI tools were used
  • AI tools were used (specify below)

charles875518 and others added 2 commits October 21, 2026 16:16
The architecture page still described the removed Go implementation, and
the HTTP indexer reference predated the Conductor series: it documented a
JSON wire format, GPU/CPU/DISK tiers, /query_by_hash, and master-side
seq_hash parsing, none of which match the code.

- Rewrite the architecture page around the C++ components, the
  device/cpu_local/cpu_share/disk tiers, hash profiles, gap recovery,
  configuration, and the CMake build.
- Rewrite the HTTP reference for the msgpack routes (/register,
  /unregister, /query, /global_view, /services), their validation and
  error format, and how Conductor indexes the Store master stream from
  object keys.
- Document the session_id field on Store stored events.
- Fix the Python API link text and note the default RPC port.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added documentation Improvements or additions to documentation run-ci labels Oct 21, 2026
@ykwd
ykwd merged commit ebaf3bc into kvcache-ai:main Oct 23, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation run-ci

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants