Skip to content

SWE-bench: don't pass external environment variables into Apptainer containers#1116

Merged
gwarmstrong merged 1 commit intomainfrom
ludwig-n/swe-bench-clean-env
Dec 16, 2025
Merged

SWE-bench: don't pass external environment variables into Apptainer containers#1116
gwarmstrong merged 1 commit intomainfrom
ludwig-n/swe-bench-clean-env

Conversation

@ludwig-n
Copy link
Collaborator

@ludwig-n ludwig-n commented Dec 16, 2025

With this PR, environment variables from the Nemo-Skills container are no longer passed through to the Apptainer containers. This was a security risk, as this gave the agent access to potentially sensitive info.

If an environment variable does need to be passed into the Apptainer containers, one can use the APPTAINERENV_ prefix. For example, setting an environment variable in the cluster config as APPTAINERENV_KEY=value will make it available inside of Apptainer as KEY=value. See the Apptainer docs for more info.

The Slurm SWE-bench tests pass.

Summary by CodeRabbit

  • Bug Fixes
    • Improved container environment isolation during evaluation execution by ensuring a clean environment is maintained, preventing potential environment contamination issues.

✏️ Tip: You can customize this high-level summary in your review settings.

Signed-off-by: Nikolai Ludwig <nliudvig@nvidia.com>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 16, 2025

📝 Walkthrough

Walkthrough

The --cleanenv flag was added to Apptainer container execution commands in two locations within _execute_container_command to provide environment isolation inside containers, working alongside existing --writable-tmpfs and --no-mount options.

Changes

Cohort / File(s) Change Summary
Container environment isolation
nemo_skills/inference/eval/swebench.py
Added --cleanenv flag to Apptainer exec commands in two locations within _execute_container_command function

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check ✅ Passed The title accurately describes the main change: adding the --cleanenv flag to prevent external environment variables from being passed into Apptainer containers, matching the core security improvement in the changeset.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch ludwig-n/swe-bench-clean-env

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ludwig-n ludwig-n changed the title SWE-bench: don't pass external env vars into Apptainer container SWE-bench: don't pass external environment variables into Apptainer containers Dec 16, 2025
@gwarmstrong gwarmstrong merged commit e09953e into main Dec 16, 2025
5 checks passed
@gwarmstrong gwarmstrong deleted the ludwig-n/swe-bench-clean-env branch December 16, 2025 16:49
wasiahmad pushed a commit that referenced this pull request Dec 19, 2025
…ontainers (#1116)

Signed-off-by: Nikolai Ludwig <nliudvig@nvidia.com>
wasiahmad pushed a commit that referenced this pull request Dec 19, 2025
…ontainers (#1116)

Signed-off-by: Nikolai Ludwig <nliudvig@nvidia.com>

Signed-off-by: wasiahmad <wasiahmad@ucla.edu>
hsiehjackson pushed a commit that referenced this pull request Jan 13, 2026
…ontainers (#1116)

Signed-off-by: Nikolai Ludwig <nliudvig@nvidia.com>
Signed-off-by: Cheng-Ping Hsieh <chsieh@nvidia.com>
wasiahmad pushed a commit that referenced this pull request Feb 4, 2026
…ontainers (#1116)

Signed-off-by: Nikolai Ludwig <nliudvig@nvidia.com>
dgtm777 pushed a commit that referenced this pull request Mar 18, 2026
…ontainers (#1116)

Signed-off-by: Nikolai Ludwig <nliudvig@nvidia.com>
dgtm777 pushed a commit that referenced this pull request Mar 18, 2026
…ontainers (#1116)

Signed-off-by: Nikolai Ludwig <nliudvig@nvidia.com>
Signed-off-by: dgitman <dgitman@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants