Skip to content

fix(security): prevent path traversal in TerminalBench 2 tar extraction - #5928

Closed
Junass1 wants to merge 2 commits into
NousResearch:mainfrom
Junass1:fix/terminalbench2-tar-traversal-protection
Closed

fix(security): prevent path traversal in TerminalBench 2 tar extraction#5928
Junass1 wants to merge 2 commits into
NousResearch:mainfrom
Junass1:fix/terminalbench2-tar-traversal-protection

Conversation

@Junass1

@Junass1 Junass1 commented Apr 7, 2026

Copy link
Copy Markdown
Contributor

This PR closes a quiet but dangerous gap in TB2 archive handling

Some bugs scream. This one didn’t.

TerminalBench 2 evaluation accepted base64-encoded tar.gz archives from dataset task payloads and extracted them into temporary directories. Until now, that extraction path trusted archive metadata too much.

That meant a malicious archive could attempt to:

  • escape the intended extraction root with traversal paths
  • introduce unsafe non-file members such as symlinks
  • write unexpected content outside the task’s temp workspace during eval setup

What changed

The extraction flow was hardened to treat archive contents as hostile input.

This PR:

  • rejects absolute paths
  • rejects Windows drive-prefixed paths
  • rejects .. traversal segments
  • rejects non-regular-file archive members
  • performs bounded manual extraction instead of trusting extractall()

In short: task archives can still populate the eval workspace, but they can no longer redefine where that workspace ends.

Why this matters

TB2 tasks are effectively untrusted content at the archive boundary.

Even though this happens in an evaluation pipeline, archive traversal here is still high impact:

  • it can contaminate the local machine or runner state
  • it can interfere with neighboring tasks
  • it weakens confidence in benchmark isolation
  • it creates an avoidable file-write primitive in a code path that should be deterministic

Proof we locked it down

Added focused security tests covering:

  • valid nested file extraction
  • traversal rejection
  • symlink member rejection

How to test

python -m pytest tests/environments/benchmarks/test_terminalbench2_env_security.py -q

teknium1 added a commit that referenced this pull request Apr 7, 2026
… tar traversal, credential leakage

Salvaged from PRs #5800 (memosr), #5806 (memosr), #5915 (Ruzzgar), #5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
teknium1 added a commit that referenced this pull request Apr 8, 2026
… tar traversal, credential leakage (#5944)

Salvaged from PRs #5800 (memosr), #5806 (memosr), #5915 (Ruzzgar), #5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
@teknium1

teknium1 commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

Merged via #5944 as part of a consolidated security hardening PR. Your tar traversal fix and security tests were applied as-is — thorough CVE-2007-4559 mitigation. Thanks @Awsh1!

@teknium1 teknium1 closed this Apr 8, 2026
saxster pushed a commit to saxster/hermes-agent that referenced this pull request Apr 8, 2026
… tar traversal, credential leakage (NousResearch#5944)

Salvaged from PRs NousResearch#5800 (memosr), NousResearch#5806 (memosr), NousResearch#5915 (Ruzzgar), NousResearch#5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
Tommyeds pushed a commit to Tommyeds/hermes-agent that referenced this pull request Apr 12, 2026
… tar traversal, credential leakage (NousResearch#5944)

Salvaged from PRs NousResearch#5800 (memosr), NousResearch#5806 (memosr), NousResearch#5915 (Ruzzgar), NousResearch#5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
… tar traversal, credential leakage (NousResearch#5944)

Salvaged from PRs NousResearch#5800 (memosr), NousResearch#5806 (memosr), NousResearch#5915 (Ruzzgar), NousResearch#5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 28, 2026
… tar traversal, credential leakage

Salvaged from PRs NousResearch#5800 (memosr), NousResearch#5806 (memosr), NousResearch#5915 (Ruzzgar), NousResearch#5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
… tar traversal, credential leakage (NousResearch#5944)

Salvaged from PRs NousResearch#5800 (memosr), NousResearch#5806 (memosr), NousResearch#5915 (Ruzzgar), NousResearch#5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
… tar traversal, credential leakage (NousResearch#5944)

Salvaged from PRs NousResearch#5800 (memosr), NousResearch#5806 (memosr), NousResearch#5915 (Ruzzgar), NousResearch#5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
… tar traversal, credential leakage (NousResearch#5944)

Salvaged from PRs NousResearch#5800 (memosr), NousResearch#5806 (memosr), NousResearch#5915 (Ruzzgar), NousResearch#5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
jh1nresh pushed a commit to jh1nresh/hermes-agent that referenced this pull request Aug 26, 2026
… tar traversal, credential leakage

Salvaged from PRs NousResearch#5800 (memosr), NousResearch#5806 (memosr), NousResearch#5915 (Ruzzgar), NousResearch#5928 (Awsh1).

Changes:
- Use hmac.compare_digest for API key comparison (timing attack prevention)
- Apply provider env var blocklist to Docker containers (credential leakage)
- Replace tar.extractall() with safe extraction in TerminalBench2 (CVE-2007-4559)
- Add SSRF protection via is_safe_url to ALL platform adapters:
  base.py (cache_image_from_url, cache_audio_from_url),
  discord, slack, telegram, matrix, mattermost, feishu, wecom
  (Signal and WhatsApp protected via base.py helpers)
- Update tests: mock is_safe_url in Mattermost download tests
- Add security tests for tar extraction (traversal, symlinks, safe files)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants