Small clarification comment on implementation

1dfb8b5

Select commit

Loading

Failed to load commit list.

Uh oh!

There was an error while loading. Please reload this page.

API Compatibility Check on: push 4

Pre-flight check

OPTIONAL: Check API Backward Compatibility

OPTIONAL: API Backward Compatibility Check Summary

Auto-assign Milestone to PR on: push

assign-milestone

Installation Test on: push

pre-flight / pre-flight

Pip - Python3.12 - AMD64/Linux - NGC PyTorch

UV - Python3.12 - AMD64/Linux - NGC PyTorch

Install test summary

CICD Megatron-LM on: push 51

is-not-external-contributor

pre-flight / pre-flight

linting

cicd-wait-in-queue

cicd-container-build

cicd-parse-unit-tests

tests/unit_tests/test_model_configs.py - latest

tests/unit_tests/test_fp8_param.py - latest

tests/unit_tests/pipeline_parallel/**/*.py - latest

tests/unit_tests/models/**/*.py - latest

tests/unit_tests/data/**/*.py - latest

tests/unit_tests/dist_checkpointing/test_optimizer.py - latest

tests/unit_tests/dist_checkpointing/**/*.py - latest

tests/unit_tests/dist_checkpointing/models/**/*.py - latest

tests/unit_tests/dist_checkpointing/models/test_moe_experts.py - latest

tests/unit_tests/transformer/**/*.py - latest

tests/unit_tests/transformer/moe/**/*.py - latest

tests/unit_tests/distributed/fsdp/**/*.py - latest

tests/unit_tests/**/*.py - latest

cicd-parse-integration-tests

gpt/gpt3_mcore_te_tp1_pp4_vp1_dist_optimizer_overlap_grad_reduce_param_gather_overlap_optimizer - latest

gpt/gpt3_mcore_te_tp2_pp1_gdn - latest

gpt/gpt3_mcore_te_tp4_pp1_resume_torch_dist_dist_optimizer_overlap_grad_reduce_param_gather - latest

moe/gpt3_moe_mcore_te_tp4_ep2_etp2_pp2_resume_torch_dist_dist_optimizer - latest

Nemo_CICD_Test

Coverage_Fake

Coverage (unit-test)

cleanup-taint-node

Copyright check on: push

pre-flight / pre-flight

copyright-check / main

copyright-check-summary

Codeowners Approval Workflow on: push

pre-flight / pre-flight

codeowners-approval

multi-approval-bot-summary

Build, test, and publish a PyPi wheel (to testpypi). on: push

pre-flight / pre-flight

build-test-publish-wheels / build-and-test-wheels (megatron-core, arm64, quay.io/pypa/manylinux_2_28_aarch64)

build-test-publish-wheels / build-and-test-wheels (megatron-core, amd64, quay.io/pypa/manylinux_2_28_x86_64)

build-test-publish-wheels / build-and-test-wheels (megatron-fsdp, quay.io/pypa/manylinux_2_28_x86_64, amd64)

build-test-publish-wheels / publish-wheels

build-test-publish-wheel-summary

Select a check to view from the sidebar