Repository navigation
[Store] Support 512MB hugepage size for client buffer allocation - #3789
Conversation
|
|
||
| // 512MiB hugepages (PMD size on arm64 kernels with 64K base pages) are not | ||
| // defined by older glibc/kernel headers; provide fallbacks. | ||
| #ifndef MAP_HUGE_512M |
There was a problem hiding this comment.
Linux UAPI names these constants MAP_HUGE_512MB and MFD_HUGE_512MB. Since the guards check the non-standard *_512M names, they always define private aliases even when the system headers provide the constants. Please use the standard names directly, or guard and define those exact names if old-header compatibility is required.
github-actions
Bot
added
documentation
|
The single failing job, Maybe re-run the failed job? It looks like a flaky PG elastic test on the runner. Thanks! |
512MiB is the PMD hugepage size on arm64 kernels running with 64K base pages, where 2MB pages give poor TLB coverage and 1GB pages are unavailable. - MC_STORE_HUGEPAGE_SIZE now accepts 512MB in addition to 2MB/1GB (get_hugepage_size_from_env, allocate_buffer_numa_segments). - Provide MAP_HUGE_512M / MFD_HUGE_512M fallbacks for older glibc/kernel headers that do not define them. - MmapArena honors the configured hugepage size: align the pool to it, pass the size-specific MAP_HUGE_* flag to mmap(), and strip the size bits on the no-hugepage retry. - Document 512MB in the store deployment guide.
The Linux UAPI spells these constants with a trailing 'B' (MAP_HUGE_512MB in asm-generic/mman-common.h, MFD_HUGE_512MB in linux/memfd.h). Guard and define the standard names so the fallbacks only kick in on headers that genuinely lack them.
de74a9a to
6996f5c
Compare
|
CI has passed. Can somebody merge it? |
Description
Add support for 512MB hugepages in Mooncake Store's client buffer allocation paths. 512MiB is the PMD hugepage size on arm64 kernels running with 64K base pages, where 2MB pages give poor TLB coverage and 1GB pages are unavailable (Like GB200,GB300).
MC_STORE_HUGEPAGE_SIZEnow accepts512MBin addition to2MB/1GB(get_hugepage_size_from_env,allocate_buffer_numa_segments).MAP_HUGE_512M/MFD_HUGE_512Mfallbacks for older glibc/kernel headers that do not define them.MmapArenanow honors the configured hugepage size: it aligns the pool to the configured page size, passes the size-specificMAP_HUGE_*flag tommap(), and strips the size bits on the no-hugepage retry.512MBin the store deployment guide.Module
mooncake-transfer-engine)mooncake-store)mooncake-reshard)mooncake-ep)mooncake-pg)mooncake-integration)mooncake-p2p-store)mooncake-wheel)mooncake-common)mooncake-rl)Type of Change
How Has This Been Tested?
Validated on an arm64 Linux host (64K base pages) with 512MB HugeTLB pages reserved: client buffers are allocated with
MAP_HUGE_512Mand serve reads/writes correctly. Unit tests for the new size parsing are added in this PR and run in CI.Test commands:
# On an arm64 host with 512MB hugepages reserved: MC_STORE_USE_HUGEPAGE=1 MC_STORE_HUGEPAGE_SIZE=512MB ./build/mooncake-store/tests/mooncake_store_testsTest results:
Manual testing: ran mooncake-store client with
MC_STORE_USE_HUGEPAGE=1 MC_STORE_HUGEPAGE_SIZE=512MBon an arm64 host with 512MB HugeTLB pages; verified mappings use 512MB pages via/proc/self/smaps(KernelPageSize/Hugetlb) and that put/get traffic works.Checklist
./scripts/code_format.shAI Assistance Disclosure
Kimi Code CLI helped prepare this PR branch (rebasing the change onto latest
main, adding theHugepageSizeEnvTestunit tests, running pre-commit). The human submitter has reviewed every changed line and is responsible for the change.