Skip to content

feat: Dynamo K8s Integration - #2990

Open
jthomson04 wants to merge 9 commits into
NVIDIA-NeMo:mainfrom
jthomson04:dynamo-k8s-integration-nomx
Open

feat: Dynamo K8s Integration#2990
jthomson04 wants to merge 9 commits into
NVIDIA-NeMo:mainfrom
jthomson04:dynamo-k8s-integration-nomx

Conversation

@jthomson04

@jthomson04 jthomson04 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Issues

List issues that this PR closes (syntax):

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

@copy-pr-bot

copy-pr-bot Bot commented Jun 29, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the Documentation Improvements or additions to documentation label Jun 29, 2026
@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown

❌ Submodule Fast-Forward Check Failed

Check based on commit: 4db7a3a (PR #2990 from dynamo-k8s-integration-nomx)

❌ Submodules that need attention:

Gym: ❌ Commits have DIVERGED from a common ancestor
TARGET (main branch): https://github.com/NVIDIA-NeMo/Gym/commits/610a08ab5fe9f8f5fb5fff36b170429ea67f0f92/
CURRENT (PR #2990 from dynamo-k8s-integration-nomx): https://github.com/NVIDIA-NeMo/Gym/commits/cbe8edc3f31665d4d1f34550b6fbb1d541fc39ae/

Please ensure all submodule commits are fast-forwards of the main branch before merging.

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown

❌ Submodule Fast-Forward Check Failed

Check based on commit: cf022cb (PR #2990 from dynamo-k8s-integration-nomx)

❌ Submodules that need attention:

Gym: ❌ Commits have DIVERGED from a common ancestor
TARGET (main branch): https://github.com/NVIDIA-NeMo/Gym/commits/610a08ab5fe9f8f5fb5fff36b170429ea67f0f92/
CURRENT (PR #2990 from dynamo-k8s-integration-nomx): https://github.com/NVIDIA-NeMo/Gym/commits/cbe8edc3f31665d4d1f34550b6fbb1d541fc39ae/

Please ensure all submodule commits are fast-forwards of the main branch before merging.

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown

✅ Submodule Fast-Forward Check Results

Check based on commit: 869a0bc (PR #2990 from dynamo-k8s-integration-nomx)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown

✅ Submodule Fast-Forward Check Results

Check based on commit: 957db15 (PR #2990 from dynamo-k8s-integration-nomx)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

@jthomson04
jthomson04 marked this pull request as ready for review June 30, 2026 00:11
@jthomson04
jthomson04 requested review from a team as code owners June 30, 2026 00:11
@jthomson04 jthomson04 mentioned this pull request Jun 30, 2026
Open
@jthomson04
jthomson04 force-pushed the dynamo-k8s-integration-nomx branch from 957db15 to 68786f9 Compare July 1, 2026 23:37
@jthomson04

jthomson04 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 68786f9

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

✅ Submodule Fast-Forward Check Results

Check based on commit: 68786f9 (PR #2990 from dynamo-k8s-integration-nomx)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

@jthomson04 jthomson04 added the CI:L1 Run doctests, unit tests, and functional tests label Jul 1, 2026
@jthomson04

jthomson04 commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 11d1a04

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

✅ Submodule Fast-Forward Check Results

Check based on commit: 11d1a04 (PR #2990 from dynamo-k8s-integration-nomx)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

@terrykong
terrykong requested a review from yuki-97 July 6, 2026 17:35
terrykong
terrykong reviewed Jul 6, 2026
View reviewed changes

@terrykong terrykong left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @jthomson04 . will review. tagging @yuki-97 to review after me just to avoid duplicate discussion points

@jthomson04

jthomson04 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 66f050b

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

✅ Submodule Fast-Forward Check Results

Check based on commit: 66f050b (PR #2990 from dynamo-k8s-integration-nomx)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

jthomson04 added 3 commits July 7, 2026 18:20
@jthomson04
feat(dynamo): add NCCL-backed generation integration
692fc0a 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@jthomson04
feat(nemo-gym): preserve native Dynamo token metadata
f36a9ae 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@jthomson04
feat(nrl-k8s): manage DynamoGraphDeployments
58c20d2 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
jthomson04 added 3 commits July 7, 2026 18:20
@jthomson04
docs(dynamo): add Kubernetes examples and integration guide
3dd9284 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@jthomson04
chore: satisfy pre-commit lint checks
606c973 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@jthomson04
test: improve Dynamo generation coverage
cf2325a 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@jthomson04
jthomson04 force-pushed the dynamo-k8s-integration-nomx branch from 66f050b to cf2325a Compare July 7, 2026 18:24
@jthomson04

jthomson04 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test cf2325a

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

✅ Submodule Fast-Forward Check Results

Check based on commit: cf2325a (PR #2990 from dynamo-k8s-integration-nomx)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

@jthomson04
refactor(nrl-k8s): simplify Dynamo Kubernetes lifecycle
6a15c3c 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

✅ Submodule Fast-Forward Check Results

Check based on commit: 6a15c3c (PR #2990 from dynamo-k8s-integration-nomx)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

@jthomson04
fix: address Dynamo Kubernetes review feedback
ff4889e 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@jthomson04

jthomson04 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test ff4889e

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

✅ Submodule Fast-Forward Check Results

Check based on commit: ff4889e (PR #2990 from dynamo-k8s-integration-nomx)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

@jthomson04
fix(dynamo): validate Kubernetes context before namespace
e1ab52a 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@jthomson04 jthomson04 mentioned this pull request Jul 29, 2026
Merged
@jthomson04

jthomson04 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Note: This PR has shifted to #3391 in order to get slurm support in first.

@jthomson04 jthomson04 mentioned this pull request Aug 13, 2026
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@terrykong terrykong terrykong left review comments
@yuki-97 yuki-97 Awaiting requested review from yuki-97

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

CI:L1 Run doctests, unit tests, and functional tests Documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jthomson04 @terrykong