Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/skills-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ skills/perf-tp-dp-comm-overlap/SKILL
skills/perf-megatron-fsdp/SKILL
skills/perf-sequence-packing/SKILL
skills/perf-hierarchical-context-parallel/SKILL
skills/perf-expert-parallel-overlap/SKILL
skills/perf-moe-comm-overlap/SKILL
skills/perf-activation-recompute/SKILL
skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL
skills/nemo-mbridge-perf-moe-comm-overlap/SKILL
skills/nemo-mbridge-perf-activation-recompute/SKILL
skills/perf-memory-tuning/SKILL
skills/perf-moe-dispatcher-selection/SKILL
skills/perf-moe-hardware-configs/SKILL
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
---
name: perf-activation-recompute
name: nemo-mbridge-perf-activation-recompute
description: Validate and use selective and full activation recompute in Megatron Bridge to reduce GPU memory usage at the cost of extra compute.
license: Apache-2.0
when_to_use: Reducing GPU memory via activation recompute, or investigating a commit that changed recompute settings and caused OOM or a regression; 'recompute_granularity', 'recompute_num_layers', 'recompute_modules', 'recompute_method', 'selective recompute', 'full recompute', 'activation memory OOM'.
---

# Activation Recompute

Stable docs: @docs/training/activation-recomputation.md
Card: @skills/perf-activation-recompute/card.yaml
Card: @skills/nemo-mbridge-perf-activation-recompute/card.yaml

## What It Is

Expand All @@ -28,7 +29,7 @@ how many layers via `recompute_num_layers`.

1. **Set `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` first** — most
borderline OOMs are caused by memory fragmentation, not capacity. This
fixes it at zero cost. See @skills/perf-memory-tuning/SKILL.md.
fixes it at zero cost. See @skills/mbridge-perf-memory-tuning/SKILL.md.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: This references @skills/mbridge-perf-memory-tuning/SKILL.md, but the actual directory is still skills/perf-memory-tuning/. This name was not renamed in this PR (and mbridge-perf-memory-tuning doesn't match the nemo-mbridge-* prefix pattern used elsewhere in this PR either). This is a broken cross-reference.

Suggested change
fixes it at zero cost. See @skills/mbridge-perf-memory-tuning/SKILL.md.
fixes it at zero cost. See @skills/perf-memory-tuning/SKILL.md.

2. Start with `recompute_granularity=selective`, `recompute_modules=[core_attn]`
(often already the default in recipes).
3. Add `layernorm` to recompute modules — nearly free compute-wise but saves
Expand Down Expand Up @@ -128,7 +129,7 @@ Key takeaways:
(hidden=28672) is expensive to recompute
- Combining `mlp` + `core_attn` is slightly worse than `mlp` alone
- For this workload, the actual OOM fix was `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`
(memory fragmentation, not capacity). See @skills/perf-memory-tuning/SKILL.md.
(memory fragmentation, not capacity). See @skills/mbridge-perf-memory-tuning/SKILL.md.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same broken reference — mbridge-perf-memory-tuning doesn't exist.

Suggested change
(memory fragmentation, not capacity). See @skills/mbridge-perf-memory-tuning/SKILL.md.
(memory fragmentation, not capacity). See @skills/perf-memory-tuning/SKILL.md.


## Code Anchors

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ recommended_path:
first_try: "recompute_granularity=selective, recompute_modules=[core_attn]"
if_still_oom: "add layernorm (cheap) or mlp (expensive but saves ~3 GB)"
last_resort: "recompute_granularity=full, recompute_method=uniform"
alternative: "see skills/perf-memory-tuning/ for VPP tuning and other memory strategies"
alternative: "see skills/mbridge-perf-memory-tuning/ for VPP tuning and other memory strategies"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue — skills/mbridge-perf-memory-tuning/ doesn't exist. Should remain skills/perf-memory-tuning/ until that skill is actually renamed.

Suggested change
alternative: "see skills/mbridge-perf-memory-tuning/ for VPP tuning and other memory strategies"
alternative: "see skills/perf-memory-tuning/ for VPP tuning and other memory strategies"

expected_metric_change:
- metric: peak_memory
direction: down
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[]
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
---
name: perf-expert-parallel-overlap
name: nemo-mbridge-perf-expert-parallel-overlap
description: Validate and use MoE expert-parallel communication overlap in Megatron-Bridge, including overlap_moe_expert_parallel_comm, delay_wgrad_compute, and flex dispatcher backends such as DeepEP and HybridEP.
license: Apache-2.0
when_to_use: Enabling EP overlap to hide dispatch/combine latency, or tracing a throughput regression to an EP overlap config change; 'overlap_moe_expert_parallel_comm', 'delay_wgrad_compute', 'flex dispatcher', 'DeepEP overlap', 'HybridEP overlap'.
---

# MoE Expert-Parallel Overlap Skill

Stable docs: @docs/training/communication-overlap.md
Card: @skills/perf-expert-parallel-overlap/card.yaml
Card: @skills/nemo-mbridge-perf-expert-parallel-overlap/card.yaml

## References

- Stable docs: @docs/training/communication-overlap.md
- Structured metadata: @skills/perf-expert-parallel-overlap/card.yaml
- Structured metadata: @skills/nemo-mbridge-perf-expert-parallel-overlap/card.yaml

## What It Is

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[]
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
name: perf-moe-comm-overlap
name: nemo-mbridge-perf-moe-comm-overlap
description: MoE expert-parallel communication overlap in Megatron Bridge. Covers dispatch/combine overlap, flex dispatcher backends, and expert wgrad scheduling.
license: Apache-2.0
when_to_use: Tuning MoE communication overlap, or tracing a MoE throughput regression to a comm-overlap config change; 'overlap_moe_expert_parallel_comm', 'MoE dispatch overlap', 'flex dispatcher', 'DeepEP overlap', 'expert wgrad scheduling'.
---

Expand All @@ -9,7 +10,7 @@ when_to_use: Tuning MoE communication overlap, or tracing a MoE throughput regre
For the higher-level overview, see:

- @docs/training/communication-overlap.md
- @skills/perf-moe-comm-overlap/card.yaml
- @skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml

## Quick Decision

Expand Down
1 change: 1 addition & 0 deletions skills/nemo-mbridge-perf-moe-comm-overlap/evals/evals.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[]
Loading