cp: build: Bump modelopt and TE (2304) into r0.3.0

[ko3n1g]

beep boop [🤖]: Hi @ko3n1g 👋,

we've cherry picked #2304 into  for you! 🚀

Please review and approve this cherry pick by your convenience!

Summary by CodeRabbit

• Chores
  • Updated transformer-engine reference to a new revision.
  • Updated nvidia-modelopt and nvidia-resiliency-ext dependencies to new versions.

[ko3n1g]

/ok to test 519adec

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

📝 Walkthrough

Walkthrough

Updated dependency versions and configurations in pyproject.toml. Modified transformer-engine git revision, added nvidia-modelopt and nvidia-resiliency-ext as public dependencies with versioning, and reorganized corresponding source entries under tool.uv.sources. Applied minor formatting adjustments.

Changes

Cohort / File(s)	Summary
Dependency Configuration pyproject.toml	Updated transformer-engine VCS revision; added nvidia-modelopt~=0.41.0 and nvidia-resiliency-ext as public dependencies in override-dependencies; reorganized source entries for nvidia-modelopt and nvidia-resiliency-ext under [tool.uv.sources]; added trailing comma after cryptography dependency.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

• Megatron-Bridge#2304: Both PRs modify the nvidia-modelopt entry in pyproject.toml with version/source updates.
• Megatron-Bridge#2059: Both PRs update the transformer-engine VCS dependency reference to a new revision.

Suggested labels

r0.3.0

Suggested reviewers

• chtruong814

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Results For Major Changes	⚠️ Warning	PR updates critical ML framework dependencies affecting convergence and performance with inadequate documentation and a flagged validation issue regarding transformer-engine commit hash.	Provide transformer-engine commit hash verification, test results for convergence regression, performance benchmarks, detailed changelog, and resolve the existing commit hash validation issue before approval.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately reflects the main change: bumping modelopt and TE dependencies as part of a cherry-pick operation into the r0.3.0 branch.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch cherry-pick-2304-r0.3.0

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@pyproject.toml`:
- Line 114: The transformer-engine git reference uses an incorrect ancient
commit hash (d9b7fc5770a88af06e2e9c2bd97b550614c3a69f) that is not within the
declared version constraint (transformer-engine >=2.10.0a0,<2.13.0); update the
git+ URL to point at a commit hash (or a release tag) that corresponds to a
v2.10.x–v2.12.x release, e.g. locate a commit that is an ancestor of a
v2.10+/v2.12 tag in the TransformerEngine repo and replace the existing hash in
the "transformer-engine @
git+https://github.com/NVIDIA/TransformerEngine.git@..." entry (or switch to the
appropriate annotated tag) so the VCS ref matches the declared version range.

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check if the commit exists in the TE repo and find related tags
git ls-remote https://github.com/NVIDIA/TransformerEngine.git | grep "d9b7fc5770a88af06e2e9c2bd97b550614c3a69f"

# Also list tags that might give us version info
echo "=== Recent TE tags ==="
git ls-remote --tags https://github.com/NVIDIA/TransformerEngine.git | tail -30

Repository: NVIDIA-NeMo/Megatron-Bridge

Length of output: 1789

---

🏁 Script executed:

#!/bin/bash
# Try to fetch the specific commit to verify if it exists
echo "Attempting to fetch the specific commit..."
git fetch --dry-run https://github.com/NVIDIA/TransformerEngine.git d9b7fc5770a88af06e2e9c2bd97b550614c3a69f 2>&1 | head -20

# Also check if maybe it's a typo - search for similar commits
echo -e "\n=== Checking HEAD of main/master ==="
git ls-remote https://github.com/NVIDIA/TransformerEngine.git | grep -E "HEAD|main|master"

# Check all recent commits (not just tags)
echo -e "\n=== Checking for commits in main branch ==="
git ls-remote --heads https://github.com/NVIDIA/TransformerEngine.git main | head -5

Repository: NVIDIA-NeMo/Megatron-Bridge

Length of output: 483

---

🏁 Script executed:

#!/bin/bash
# Fetch repo info and check commit details
mkdir -p /tmp/te_check
cd /tmp/te_check

# Clone a shallow copy focusing on the relevant commit and tags
git clone --depth=500 https://github.com/NVIDIA/TransformerEngine.git 2>&1 | tail -5

cd TransformerEngine

# Check what version/tags are associated with this commit
echo "=== Checking commit info ==="
git log --oneline d9b7fc5770a88af06e2e9c2bd97b550614c3a69f -1

echo -e "\n=== Tags reachable from this commit ==="
git tag --contains d9b7fc5770a88af06e2e9c2bd97b550614c3a69f 2>/dev/null | head -20

echo -e "\n=== Distance from tags v2.10, v2.11, v2.12 ==="
git log --oneline d9b7fc5770a88af06e2e9c2bd97b550614c3a69f..v2.11 | wc -l
echo "Commits between target commit and v2.11:"
git log --oneline d9b7fc5770a88af06e2e9c2bd97b550614c3a69f..v2.11 | head -5

echo -e "\n=== Check if commit is between v2.10 and v2.13 ==="
git merge-base --is-ancestor d9b7fc5770a88af06e2e9c2bd97b550614c3a69f v2.13 && echo "Commit is ancestor of v2.13 ✓" || echo "Commit is NOT ancestor of v2.13 ✗"

Repository: NVIDIA-NeMo/Megatron-Bridge

Length of output: 1019

---

🏁 Script executed:

#!/bin/bash
mkdir -p /tmp/te_check2
cd /tmp/te_check2

# Clone with full history for commit analysis
echo "Cloning full repo..."
git clone https://github.com/NVIDIA/TransformerEngine.git 2>&1 | tail -3

cd TransformerEngine

# Now check the commit
echo "=== Commit details ==="
git log --oneline d9b7fc5770a88af06e2e9c2bd97b550614c3a69f -1

echo -e "\n=== Tags containing this commit ==="
git tag --contains d9b7fc5770a88af06e2e9c2bd97b550614c3a69f 2>/dev/null

echo -e "\n=== Check version range >=2.10.0a0,<2.13.0 ==="
git merge-base --is-ancestor v2.10.0 d9b7fc5770a88af06e2e9c2bd97b550614c3a69f && echo "v2.10.0 is ancestor (commit is after v2.10)" || echo "v2.10.0 is NOT ancestor"
git merge-base --is-ancestor d9b7fc5770a88af06e2e9c2bd97b550614c3a69f v2.13.0 && echo "Commit is ancestor of v2.13.0 (commit is before v2.13)" || echo "Commit is NOT ancestor of v2.13.0"

echo -e "\n=== Commit timestamp and author ==="
git log -1 --format="%ai %an: %s" d9b7fc5770a88af06e2e9c2bd97b550614c3a69f

Repository: NVIDIA-NeMo/Megatron-Bridge

Length of output: 638

---

🏁 Script executed:

#!/bin/bash
cd /tmp/te_check2/TransformerEngine

echo "=== All v2.x tags ==="
git tag -l 'v2.*' | sort -V

echo -e "\n=== Check commit ancestry against actual tags ==="
git merge-base --is-ancestor v2.10 d9b7fc5770a88af06e2e9c2bd97b550614c3a69f && echo "✓ v2.10 is ancestor of commit" || echo "✗ v2.10 is NOT ancestor"
git merge-base --is-ancestor d9b7fc5770a88af06e2e9c2bd97b550614c3a69f v2.11 && echo "✓ Commit is ancestor of v2.11" || echo "✗ Commit is NOT ancestor of v2.11"

echo -e "\n=== Check if v2.12 or v2.13 exist ==="
git tag -l 'v2.1[2-3]'

echo -e "\n=== Latest v2.x tag ==="
git tag -l 'v2.*' | sort -V | tail -1

echo -e "\n=== Detailed version info ==="
git describe --tags d9b7fc5770a88af06e2e9c2bd97b550614c3a69f 2>&1

Repository: NVIDIA-NeMo/Megatron-Bridge

Length of output: 391

---

Fix the TransformerEngine commit hash—it does not fall within the specified version constraint.

Line 84 constrains transformer-engine to >=2.10.0a0,<2.13.0, but the commit hash on line 114 (d9b7fc5770a88af06e2e9c2bd97b550614c3a69f) is from early repository history (v0.1 era, per git describe), not from the v2.10+ range. This commit is not an ancestor of any v2.x tag and will cause dependency resolution failures. Verify the correct commit hash for the intended version and update line 114.

🤖 Prompt for AI Agents

In `@pyproject.toml` at line 114, The transformer-engine git reference uses an
incorrect ancient commit hash (d9b7fc5770a88af06e2e9c2bd97b550614c3a69f) that is
not within the declared version constraint (transformer-engine
>=2.10.0a0,<2.13.0); update the git+ URL to point at a commit hash (or a release
tag) that corresponds to a v2.10.x–v2.12.x release, e.g. locate a commit that is
an ancestor of a v2.10+/v2.12 tag in the TransformerEngine repo and replace the
existing hash in the "transformer-engine @
git+https://github.com/NVIDIA/TransformerEngine.git@..." entry (or switch to the
appropriate annotated tag) so the VCS ref matches the declared version range.