Skip to content

SafeUnpickler class for safe pickle usage - #4319

Merged
dimapihtar merged 26 commits into
NVIDIA:mainfrom
dimapihtar:pickle_safe_list
Apr 24, 2026
Merged

SafeUnpickler class for safe pickle usage#4319
dimapihtar merged 26 commits into
NVIDIA:mainfrom
dimapihtar:pickle_safe_list

Conversation

@dimapihtar

@dimapihtar dimapihtar commented Apr 15, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

Introduces SafeUnpickler class which has defined set of "safe" classes to load. Prevents loading vulnerable data and makes pickle ussage safer.

⚠️ For major changes (either in lines of code or in its impact), please make sure to first share a design doc with the team. If you're unsure what's the best way to do so, contact the @mcore-oncall.

Contribution process

Pre-checks

  • I have added relevant unit tests
  • I have added relevant functional tests
  • I have added proper typing to my code Typing guidelines
  • I have added relevant documentation
  • I have run the autoformatter.sh on my PR

Code review

Feel free to message or comment the @mcore-oncall to help accelerate your merge into main. The less complex your PR is, the faster it will be approved and merged!

All PRs start as draft. If you open a non-draft PR, it will be automatically converted to draft.

Step 1: Mark PR as "Ready for Review"

  1. When your PR is ready, click Ready for Review.
  2. An oncall reviewer is auto-assigned and expert reviewers are notified based on your changes.
    • Some PRs may jump straight to step 2. This is determined by .github/CODEOWNERS.

⚠️ Only mark as ready once merge-conflicts are resolved and the CI is passing.
Final Review might get declined if these requirements are not fulfilled.

Step 2: Final Review

For PRs that change megatron/core, once all expert reviewers have approved, the Final Review label is applied automatically and final reviewers are assigned.

For PRs outside megatron/core, this step is skipped.

Step 3: Approved

Once all required reviewers have approved, the Approved label is applied automatically.

Merge

Any member of mcore-engineers will be able to merge your PR.

For MRs into `dev` branch The proposed review process for `dev` branch is under active discussion.

MRs are mergable after one approval by either eharper@nvidia.com or zijiey@nvidia.com.

@dimapihtar
add safe list for pickle
beb3523 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar
move SafeUnpickler to safe_globals.py
dadd1f9 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar
remove extra line
21f4082 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Apr 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@dimapihtar dimapihtar added complexity: low Expert Review [deprecated] Apply this label to indicate that your PR is ready for expert review. Run functional tests labels Apr 15, 2026
@dimapihtar
dimapihtar marked this pull request as ready for review April 15, 2026 15:10
@dimapihtar
dimapihtar requested review from a team as code owners April 15, 2026 15:10
@dimapihtar

dimapihtar commented Apr 15, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 21f4082

@svcnvidia-nemo-ci
svcnvidia-nemo-ci requested a review from a team April 15, 2026 15:10
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the Final Review PR is in the "final review" stage label Apr 15, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added this to the Core 0.16 milestone Apr 15, 2026
@dimapihtar
fix code style
e69c5a2 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar

dimapihtar commented Apr 15, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test e69c5a2

@dimapihtar
fix imports
4c117a9 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar

dimapihtar commented Apr 15, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 4c117a9

@dimapihtar
fix import
41653df 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar

dimapihtar commented Apr 15, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 41653df

@dimapihtar
avoid circular import error
b2243b3 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar

dimapihtar commented Apr 15, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test b2243b3

@dimapihtar
fix code style
e433662 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar

dimapihtar commented Apr 15, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test e433662

@svcnvidia-nemo-ci svcnvidia-nemo-ci removed the Final Review PR is in the "final review" stage label Apr 22, 2026
@yashaswikarnati yashaswikarnati mentioned this pull request Apr 22, 2026
Closed
2 tasks
@dimapihtar
add safe implementation of
907574e 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar
get back _load_from_bytes to safe list
165fb93 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar
revert changes
d0ae5d2 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the Final Review PR is in the "final review" stage label Apr 22, 2026
@dimapihtar

dimapihtar commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test d0ae5d2

@dimapihtar
fix code style
c66962d 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@dimapihtar

dimapihtar commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test c66962d

@dimapihtar
dimapihtar removed the request for review from a team April 22, 2026 21:05
@dimapihtar

dimapihtar commented Apr 23, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 17ac2a0

@svcnvidia-nemo-ci svcnvidia-nemo-ci added Approved All necessary approvals have been made and removed Final Review PR is in the "final review" stage labels Apr 23, 2026
@dimapihtar
dimapihtar added this pull request to the merge queue Apr 24, 2026
@svcnvidia-nemo-ci

svcnvidia-nemo-ci commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started!

You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/24885940184

Merged via the queue into NVIDIA:main with commit d4cacef Apr 24, 2026
186 of 190 checks passed
@dimapihtar
dimapihtar deleted the pickle_safe_list branch April 24, 2026 12:04
@chtruong814 chtruong814 added the core_r0.17.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge. label Apr 30, 2026
yangbofun pushed a commit to xlm-research/Megatron-LM that referenced this pull request May 22, 2026
@dimapihtar
SafeUnpickler class for safe pickle usage (NVIDIA#4319)
fe275fd 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
yhgalaxy pushed a commit to yhgalaxy/Megatron-LM that referenced this pull request Jun 17, 2026
@dimapihtar @yhgalaxy
SafeUnpickler class for safe pickle usage (NVIDIA#4319)
4600d15 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
Signed-off-by: yhgalaxy <yhgalaxy@outlook.com>
jon-barker pushed a commit to jon-barker/Megatron-LM that referenced this pull request Jul 10, 2026
@dimapihtar Jon Barker
SafeUnpickler class for safe pickle usage (NVIDIA#4319)
5662d4d 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
Signed-off-by: Jon Barker <jbarker@aws-cmh-slurm-1-vscode-02.cm.cluster>
terminator123 pushed a commit to 021ai/Megatron-LM that referenced this pull request Aug 3, 2026
@dimapihtar @terminator123
SafeUnpickler class for safe pickle usage (NVIDIA#4319)
e372452 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@ko3n1g ko3n1g mentioned this pull request Aug 21, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@jaredcasper jaredcasper jaredcasper approved these changes
@ericharper ericharper ericharper approved these changes
@Phlip79 Phlip79 Phlip79 approved these changes

Assignees

No one assigned

Labels

Approved All necessary approvals have been made complexity: low core_r0.17.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge. Expert Review [deprecated] Apply this label to indicate that your PR is ready for expert review. Run functional tests

Projects

None yet

Milestone

Core 0.16

Development

Successfully merging this pull request may close these issues.

6 participants

@dimapihtar @ericharper @svcnvidia-nemo-ci @jaredcasper @Phlip79 @chtruong814