Skip to content

fix(cache): preserve macOS entries during ancestor churn - #1800

Merged
mldangelo-oai merged 4 commits into
mainfrom
fix/cache-darwin-ancestor-monitor
Aug 24, 2026
Merged

fix(cache): preserve macOS entries during ancestor churn#1800
mldangelo-oai merged 4 commits into
mainfrom
fix/cache-darwin-ancestor-monitor

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a Darwin kqueue vnode monitor for scanned files and ancestor directories so unrelated sibling or temporary-file writes no longer invalidate safe cache identities.
  • Continue failing closed when a source file or ancestor directory is replaced, restored, or has watched vnode attributes changed during a scan.
  • Normalize monitored paths and make descriptor ownership, setup failure, interruption cleanup, and repeated close deterministic.
  • Cover macOS sibling churn, source-file replacement, ancestor-directory replacement, attribute-event registration, lifecycle failure paths, platform monitor selection, and compressed-wrapper cache behavior.

Why

Linux uses inotify and Windows holds path locks while caching, but macOS previously had no ancestor monitor and compared every ancestor directory modification timestamp instead. A normal compressed-model scan creates temporary files under a shared ancestor, changing that ancestor and causing every cache store to be rejected. The existing compressed-wrapper regression reproduced deterministically with total_entries == 0 instead of 2.

The Darwin monitor watches vnode deletion, rename, revocation, and attribute events. Ordinary directory writes remain harmless, while replacing and restoring either the scanned file or one of its ancestors, or changing watched access metadata, still invalidates the cache identity.

Validation

  • Full tests/cache: 241 passed, 6 platform skips before the final attribute-event follow-up; the focused follow-up cache run passed.
  • Repository-wide Ruff format and lint: clean.
  • Repository-wide mypy: 479 source files clean.
  • Full fast suite on the exact current head: 23,276 passed, 806 skipped.
  • Independent exact-head reviews found no actionable issues.
  • Native kqueue cases remain platform-gated on the Linux devbox; deterministic monitor-construction, ownership, event-registration, failure, and interruption paths are exercised with mocks.

@mldangelo-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: fdd9bfc8db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Workflow run and artifacts

Performance Benchmarks

Compared 13 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 13 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 3.331s -> 3.339s (+0.2%).

Workload Benchmark Target Size Files Baseline Current Change Status
warm-cache-rescan tests/benchmarks/test_scan_benchmarks.py::test_scan_warm_cached_repository_rescan release-candidate 547.3 KiB 32 118.82ms 113.91ms -4.1% stable
nested-payload-review tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_base64] nested_base64 98 B 1 200.8us 205.7us +2.4% stable
nested-payload-review tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_hex] nested_hex 130 B 1 216.2us 212.0us -1.9% stable
nested-payload-review tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_raw] nested_raw 78 B 1 189.0us 186.0us -1.6% stable
padded-multi-stream-upload tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_padded_multi_stream_upload multi_stream_padded 4.1 KiB 1 237.0us 239.7us +1.1% stable
rejected-basic-auth-candidates tests/benchmarks/test_scan_benchmarks.py::test_rejected_basic_auth_candidates_scan_linearly - 371.1 KiB 1 1.859s 1.877s +1.0% stable
chunked-upload-stream tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_upload_stream chunked_stream 278.2 KiB 1 91.21ms 91.97ms +0.8% stable
single-checkpoint-preflight tests/benchmarks/test_scan_benchmarks.py::test_scan_single_checkpoint_before_load single_checkpoint.pkl 183.0 KiB 1 84.79ms 84.12ms -0.8% stable
duplicate-heavy-registry tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_registry_snapshot registry-snapshot 915.2 KiB 13 462.00ms 459.13ms -0.6% stable
direct-malicious-upload tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_direct_malicious_upload malicious_reduce 52 B 1 151.1us 151.9us +0.6% stable
mixed-model-repository tests/benchmarks/test_scan_benchmarks.py::test_scan_release_candidate_repository release-candidate 547.3 KiB 32 507.39ms 504.87ms -0.5% stable
suspicious-pickle-intake tests/benchmarks/test_scan_benchmarks.py::test_scan_suspicious_pickle_intake suspicious-intake 183.8 KiB 4 117.60ms 117.19ms -0.3% stable
clean-training-checkpoint tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_clean_training_checkpoint safe_large 278.2 KiB 1 89.47ms 89.29ms -0.2% stable

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves ScanResultsCache correctness on macOS by introducing a Darwin-specific ancestor/path monitor that detects vnode replacement (rename/delete/revoke) without invalidating cache identities due to unrelated directory writes (e.g., temporary-file churn during scans).

Changes:

  • Add a macOS _DarwinPathMonitor based on kqueue vnode events and wire it into ancestor-identity monitoring/verification.
  • Add Darwin-focused regression tests covering unrelated sibling churn and restored file/directory replacement, plus a unit test for Darwin monitor selection via monkeypatching.
  • Document the behavior change in the root CHANGELOG.md.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
modelaudit/cache/scan_results_cache.py Adds _DarwinPathMonitor and integrates it into AncestorIdentity monitoring on macOS.
tests/cache/test_cache_correctness.py Adds macOS regressions for cache identity behavior under sibling churn and ancestor/file replacement, and verifies Darwin monitor selection.
CHANGELOG.md Notes the macOS cache-identity behavior change under [Unreleased].

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread modelaudit/cache/scan_results_cache.py Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: fdd9bfc8db

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread modelaudit/cache/scan_results_cache.py Fixed
@mldangelo-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-24T14:11:06.562756Z b4ab23b Manual request
🔒 Security Review Completed 2026-08-24T14:14:37.144987Z b4ab23b Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@mldangelo-oai

Copy link
Copy Markdown
Contributor Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb9b355668

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread modelaudit/cache/scan_results_cache.py Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: fb9b355668

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@mldangelo-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@mldangelo-oai

Copy link
Copy Markdown
Contributor Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: b4ab23b756

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: b4ab23b756

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@mldangelo-oai
mldangelo-oai merged commit 2c3512b into main Aug 24, 2026
49 of 51 checks passed
@mldangelo-oai
mldangelo-oai deleted the fix/cache-darwin-ancestor-monitor branch August 24, 2026 15:37
@github-actions github-actions Bot mentioned this pull request Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants