[Aionw]

Description

Implement the DataProto catalog increment of Phase 2 from #3534 by moving its tracked Python source and directly owned tests into the unified root Python project.

• move the authoritative mooncake.dataproto_catalog implementation to python/mooncake;
• move the catalog tests to python/tests/integration/dataproto;
• remove both legacy tracked locations in the same change;
• keep structured-object imports lazy and preserve the existing mooncake.dataproto_catalog public import;
• update root/scikit-build packaging checks, transitional legacy wheel staging and installation inputs, CODEOWNERS, and labeler paths;
• add source-tree and installed-wheel coverage for the canonical location and optional-dependency boundary.

No documentation changes are needed because this source-layout migration does not change the public API or command behavior.

Overlap

• [Wheel] Add DataProto catalog lifecycle management #3374 landed on main on 2026-08-25, after this PR was opened, and added DataProto catalog lifecycle behavior to the legacy source and test. The 2026-09-28 merge of main (d9152a4) carries [Wheel] Add DataProto catalog lifecycle management #3374's implementation to python/mooncake/dataproto_catalog.py through rename detection and merges its tests into python/tests/integration/dataproto/test_dataproto_catalog.py. This PR adds no catalog feature work of its own.
• [Wheel] Support DataProto matrix partial reads #3567 (landed 2026-09-23) changed the structured-object functionality consumed lazily by the catalog. This PR does not modify structured_object_store.py, partial reads, or any of [Wheel] Support DataProto matrix partial reads #3567's feature work; structured_object_store.py is migrated separately in [Wheel] Migrate structured object storage to unified package #3633.
• [Misc] Apply ruff-format to Python sources and clear ruff check debt #4008 (open) applies ruff format to the legacy dataproto_catalog.py and its test. This PR formats the same three statements in the moved files, so whichever lands second drops those hunks.
• Since this PR was opened, [Store] Migrate lightweight Python utilities to unified package #3586, [Store] Migrate async Python client to unified package #3624, [Wheel] Migrate Python CLI modules to unified package #3626, [Wheel] Migrate SSD administration to unified Python package #3630, [Wheel] Migrate HTTP metadata service to unified package #3632, and [EP] Migrate Python layer to unified package #3635 have landed on main; this branch includes them through its main syncs, most recently the 2026-09-28 merge of main (d9152a4). [Reshard] Migrate to unified Python package #3585 (Reshard) is still open and remains out of scope.
• [RFC]: Refactor the mooncake-wheel package layout #3425 is the older, narrower package-layout RFC already discussed by [RFC]: Unify Mooncake's Python package, build, and release architecture #3534; this PR follows the Phase 2 layout defined in [RFC]: Unify Mooncake's Python package, build, and release architecture #3534.

Module

• Transfer Engine (mooncake-transfer-engine)
• Mooncake Store (mooncake-store)
• Reshard (mooncake-reshard)
• Mooncake EP (mooncake-ep)
• Mooncake PG (mooncake-pg)
• Integration (mooncake-integration)
• P2P Store (mooncake-p2p-store)
• Python Wheel (mooncake-wheel)
• Common (mooncake-common)
• Mooncake RL (mooncake-rl)
• CI/CD
• Docs
• Other

Type of Change

• Bug fix
• New feature
• Refactor
• Breaking change
• Documentation update
• Performance improvement
• Other

How Has This Been Tested?

Original submission (before merging main on 2026-09-28; superseded by "After merging main" below). Since #3586 moved buffer_pool.py, python/ must come before mooncake-wheel/ on PYTHONPATH:

Test commands:

env PYTHONPATH="$PWD/mooncake-wheel:$PWD/python" PYTHONNOUSERSITE=1 \
  /tmp/mooncake-dataproto.inUI5D/venv/bin/python -m pytest -q \
  python/tests/integration/dataproto/test_dataproto_catalog.py
python -m pytest -q python/tests/packaging/test_project.py
bash -n scripts/build_wheel.sh scripts/test_installation.sh
env CMAKE_BUILD_PARALLEL_LEVEL=128 python -m build --wheel \
  --outdir /tmp/mooncake-dataproto.inUI5D/dist
python -m twine check \
  /tmp/mooncake-dataproto.inUI5D/dist/mooncake_transfer_engine-0.3.12.post1-cp310-cp310-linux_x86_64.whl
env MOONCAKE_TEST_WHEEL=/tmp/mooncake-dataproto.inUI5D/dist/mooncake_transfer_engine-0.3.12.post1-cp310-cp310-linux_x86_64.whl \
  python -m pytest -q python/tests/packaging/test_installed_wheel.py
pre-commit run --files $(git diff --name-only --diff-filter=ACMR origin/main...HEAD)

Test results:

• Unit tests pass: 4 DataProto catalog tests.
• Packaging tests pass: 5 source/project contract tests and 1 isolated installed-wheel test.
• Root PEP 517 wheel build passes with 128-way CMake parallelism.
• twine check passes; the wheel manifest contains exactly one mooncake/dataproto_catalog.py.
• The wheel installs with --no-deps into an isolated environment outside the repository and imports the catalog without loading mooncake.structured_object_store.
• PR-scoped pre-commit passes all applicable hooks.

The legacy end-to-end scripts/test_installation.sh was not run because it requires separately prebuilt legacy release artifacts and system-package installation. Its updated inputs pass shell syntax validation; the supported root PEP 517 build and isolated installed-wheel path were exercised end to end.

After merging main (d9152a4, 2026-09-28):

PYTHONPATH=python:mooncake-wheel python -m pytest -q python/tests/integration/dataproto \
  python/tests/packaging mooncake-wheel/tests/test_structured_object_store.py
pre-commit run --files $(git diff --name-only --diff-filter=ACMR origin/main...HEAD)

• 166 passed, 38 skipped; the skips need Torch (26), the native mooncake.store (8), a built _fast_copy (2), or built wheels (2 installed-wheel smoke tests).
• The merge's conflicts in .github/CODEOWNERS, .github/labeler.yml, scripts/build_wheel.sh, scripts/test_installation.sh, python/tests/packaging/test_installed_wheel.py, python/tests/packaging/test_project.py were resolved by keeping both sides.
• python/tests/integration/dataproto/test_dataproto_catalog.py needed a manual merge: it now holds all of [Wheel] Add DataProto catalog lifecycle management #3374's tests plus this PR's source-location tests.
• Other merge adjustments: the source-import test puts python/ before mooncake-wheel/ on PYTHONPATH, because [Store] Migrate lightweight Python utilities to unified package #3586 moved buffer_pool.py to python/mooncake and the legacy mooncake/__init__.py imports it; dataproto_catalog.py is removed from the legacy-module list in test_scikit_build_consumes_unified_python_sources; the Tests label entry for python/tests/integration/dataproto/**/* is dropped because main's python/tests/**/* glob covers it; and ruff format is applied to three statements inherited from [Wheel] Add DataProto catalog lifecycle management #3374 in the moved files, because CI's changed-lines check treats moved files as fully changed. Apart from that formatting, the moved module is identical to main's legacy copy.
• CI's changed-lines Python check (ruff check plus scripts/ci/check_changed_python_format.py, ruff 0.6.9) passes.
• PR-scoped pre-commit passes all applicable hooks.
• The root PEP 517 wheel build, twine check, and installed-wheel smoke runs from the original submission were not repeated after the merge. CI does not run them either (it builds only the legacy scripts/build_wheel.sh wheel and runs scripts/test_installation.sh), so they still need a rerun.

Checklist

• I have performed a self-review of my own code
• I have formatted my code using ./scripts/code_format.sh (no C/C++ changes; applicable formatting passed through pre-commit)
• I have run pre-commit on the files changed in this PR and all hooks pass
• I have updated the documentation (not applicable: no public behavior or documented path changed)
• I have added tests to prove my changes are effective
• For changes >500 LOC: I have filed RFC issue [RFC]: Unify Mooncake's Python package, build, and release architecture #3534 (effective diff after merging main: 91 additions, 21 deletions)

AI Assistance Disclosure

• No AI tools were used
• AI tools were used (specify below)

OpenAI Codex performed the scoped source-layout migration, packaging and ownership updates, overlap audit, validation, diff review, and PR preparation. The human submitter remains responsible for reviewing every changed line and defending the change end to end.

Claude Code (Anthropic) merged main into this branch on 2026-09-28, resolved the conflicts, re-ran the focused tests listed under "After merging main", and updated this description.