[ascend-direct-dev]

Description

WIP. Adds an explicit hugetlb=false opt-in on create_shared_segment / SharedSegment::Create.

The default mmap path is unchanged: unnamed memfd plus MADV_HUGEPAGE (THP), no HugeTLB pool. hugetlb=true uses MFD_HUGETLB and fails closed if vm.nr_hugepages cannot back the span. Mixing THP and HugeTLB blobs in the same group is rejected via the fingerprint.

This is the shared-segment follow-up to #3952. It does not change TENT / Ascend Direct.

Module

• Transfer Engine (mooncake-transfer-engine)
• Mooncake Store (mooncake-store)
• Mooncake Conductor (mooncake-conductor)
• Reshard (mooncake-reshard)
• Mooncake EP (mooncake-ep)
• Mooncake PG (mooncake-pg)
• Integration (mooncake-integration)
• P2P Store (mooncake-p2p-store)
• Python Wheel (mooncake-wheel)
• Common (mooncake-common)
• Mooncake RL (mooncake-rl)
• CI/CD
• Docs
• Other

Type of Change

• Bug fix
• New feature
• Refactor
• Breaking change
• Documentation update
• Performance improvement
• Other

How Has This Been Tested?

Test commands:

# C++ shared-segment unit tests (HugeTLB cases skip if the pool is empty)
./build/mooncake-transfer-engine/tests/shared_segment_test

# Python engine API
python -m unittest mooncake-wheel/tests/test_shared_segment.py

Test results:

• Unit tests pass
• Integration tests pass (if applicable)
• Manual testing done (describe below)

Locally, with a HugeTLB pool reserved, SharedSegmentHugeTlbTest.* mapped 2MiB ht pages, two ranks/processes shared the same pages, and HostRegister still produced a device VA. Default hugetlb=false did not consume HugePages_Free.

Checklist

• I have performed a self-review of my own code
• I have formatted my code using ./scripts/code_format.sh
• I have run pre-commit on the files changed in this PR and all hooks pass
• I have updated the documentation (if applicable)
• I have added tests to prove my changes are effective
• For changes >500 LOC: I have filed an RFC issue

AI Assistance Disclosure

• No AI tools were used
• AI tools were used (specify below)

Cursor Grok 4.6 helped draft the HugeTLB memfd path, Python/create_shared_segment plumbing, and unit tests. Every changed line was reviewed by the submitter.

Made with Cursor