Skip to content

Revert "fix: size the GLM-5.2 MI355X eval HiCache pool absolutely (#2983)" #3002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
functionstackx merged 1 commit into from
Sep 11, 2026
+5 −13
Merged

Revert "fix: size the GLM-5.2 MI355X eval HiCache pool absolutely (#2983)" #3002

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 5 additions & 13 deletions benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,26 +91,17 @@ if agentic_kv_offload_enabled; then
# env-var override for maximum throughput on nodes with >4 TB DRAM.
HICACHE_RATIO="${HICACHE_RATIO:-1.5}"
fi
# GSM8K never fills the agentic host pool; ratio 1.5 OOMs the TP4 DRAM share.
if [ "${EVAL_ONLY:-false}" = "true" ]; then
HICACHE_EVAL_SIZE_GB="${HICACHE_EVAL_SIZE_GB:-16}"
HICACHE_POOL_ARGS=(--hicache-size "$HICACHE_EVAL_SIZE_GB")
HICACHE_POOL_DESC="size=${HICACHE_EVAL_SIZE_GB} GB/rank (eval-only)"
else
HICACHE_POOL_ARGS=(--hicache-ratio "$HICACHE_RATIO" --hicache-size 0)
HICACHE_POOL_DESC="ratio=$HICACHE_RATIO"
fi
# write_through_selective skips DRAM writes for non-reusable KV blocks,
# reducing host-bus traffic without affecting the cache hit rate.
HICACHE_WRITE_POLICY="${HICACHE_WRITE_POLICY:-write_through_selective}"
HICACHE_IO_BACKEND="${HICACHE_IO_BACKEND:-direct}"
HICACHE_MEM_LAYOUT="${HICACHE_MEM_LAYOUT:-page_first_direct}"
case "$KV_OFFLOAD_BACKEND" in
hicache)
echo "HiCache (GPU+host DRAM only): $HICACHE_POOL_DESC, write_policy=$HICACHE_WRITE_POLICY, io_backend=$HICACHE_IO_BACKEND, mem_layout=$HICACHE_MEM_LAYOUT"
echo "HiCache (GPU+host DRAM only): ratio=$HICACHE_RATIO, write_policy=$HICACHE_WRITE_POLICY, io_backend=$HICACHE_IO_BACKEND, mem_layout=$HICACHE_MEM_LAYOUT"
CACHE_ARGS=(
--enable-hierarchical-cache
"${HICACHE_POOL_ARGS[@]}"
--hicache-ratio "$HICACHE_RATIO"
--hicache-write-policy "$HICACHE_WRITE_POLICY"
--hicache-io-backend "$HICACHE_IO_BACKEND"
--hicache-mem-layout "$HICACHE_MEM_LAYOUT"
Comment on lines 91 to 107

@claude claude Bot Sep 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 This revert removes the EVAL_ONLY branch that capped the HiCache pool at a small fixed size and reinstates the ratio=1.5 sizing for eval-only GSM8K runs, but no new perf-changelog.yaml entry was added for config-key glm5.2-fp4-mi355x-sglang-agentic-mtp, violating AGENTS.md's non-negotiable rule that every recipe modification requires an append-only changelog entry. Fix: append a perf-changelog.yaml entry documenting this revert and its rationale (EAGLE bug fix upstream, not a DRAM-sharing issue) for this config key.

Extended reasoning...

AGENTS.md line 31 states every change that can affect benchmark performance and every recipe modification requires a new perf-changelog.yaml entry, appended at the tail. Grepping perf-changelog.yaml for glm5.2-fp4-mi355x-sglang-agentic-mtp shows only earlier entries (PR #2570, #2562) and no new entry tied to this diff/PR. This is a recipe modification (HiCache pool sizing behavior for EVAL_ONLY changes), so the changelog invariant is violated regardless of whether the revert itself is correct.

Verification: normal (repository non-negotiable invariant). The diff modifies a benchmark recipe and changes HiCache pool sizing behavior for EVAL_ONLY runs — it removes the EVAL_ONLY branch (deleted lines setting --hicache-size "$HICACHE_EVAL_SIZE_GB") and reinstates --hicache-ratio "$HICACHE_RATIO" (default 1.5) for both backends in benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh.…

Expand Down Expand Up @@ -141,10 +132,11 @@ EOF
MOONCAKE_MASTER_PID=$!
sleep 2
kill -0 "$MOONCAKE_MASTER_PID"
echo "HiCache+Mooncake: $HICACHE_POOL_DESC, l3_per_rank=${L3_PER_RANK_GB} GB, dram_budget=${TOTAL_CPU_DRAM_GB} GB"
echo "HiCache+Mooncake: ratio=$HICACHE_RATIO, l3_per_rank=${L3_PER_RANK_GB} GB, dram_budget=${TOTAL_CPU_DRAM_GB} GB"
CACHE_ARGS=(
--enable-hierarchical-cache
"${HICACHE_POOL_ARGS[@]}"
--hicache-ratio "$HICACHE_RATIO"
--hicache-size 0
--hicache-write-policy "$HICACHE_WRITE_POLICY"
--hicache-io-backend "$HICACHE_IO_BACKEND"
--hicache-mem-layout "$HICACHE_MEM_LAYOUT"
Expand Down