fix(cache): ignore macOS file access-time events by mldangelo-oai · Pull Request #1821 · promptfoo/modelaudit

mldangelo-oai · 2026-08-25T22:17:55Z

Summary

Ignore file-only NOTE_ATTRIB events in the macOS cache identity monitor because reading a model to hash it updates st_atime without changing its content, mode, st_mtime, or st_ctime.
Keep delete, rename, and revoke monitoring on the model, preserve NOTE_ATTRIB monitoring on every ancestor, and continue rejecting restored model-file permissions through the existing file change token.
Add a failing-first platform-independent kqueue regression plus native macOS coverage for restored file permissions; retain the existing alias, ancestor-permission, and replacement regressions.

Why

The Darwin vnode monitor introduced by #1800 subscribes to NOTE_ATTRIB on the model itself. On macOS, the scanner's own hash read advances the access time and emits that event, so every identity-capture retry reports File kept changing while capturing cache identity and cache entries cannot be published or reused.

Verification

Failing-first regression: test_darwin_path_monitor_ignores_file_access_time_attribute_changes fails on current main and passes after the fix.
Native macOS cache suite: 249 passed, 6 skipped.
Combined cache, cache CLI, cache optimization, CLI cache directory, regular hashing, pickle scanner, and PyTorch ZIP scanner suites: 1,704 passed, 6 skipped, 9 deselected.
End-to-end benign pickle scan: both scans succeed and the second scan records a cache hit.
Repository-wide Ruff lint and formatting pass; changelog Prettier check passes.
The borrowed local environment has mypy 2.1.0 while the repository locks 1.20.0; its two union-attr errors are pre-existing in unchanged July code, and changed-file mypy passes when that known baseline-only error code is disabled. Current-main locked CI Type Check is green.
A broader suite also reproduces the existing macOS-only test_scan_file_invalidates_cache_when_shard_sibling_changes failure on both the branch and untouched baseline because descriptor-bound shard pinning is unavailable; it is unrelated to this change.

chatgpt-codex-connector · 2026-08-25T22:18:10Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-08-29T10:31:52.802210Z	`54d6e9e`	Manual request
🔒 Security Review	✅ Completed 2026-08-29T10:32:42.581738Z	`54d6e9e`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

mldangelo-oai · 2026-08-25T22:18:23Z

@codex review

chatgpt-codex-connector · 2026-08-25T22:20:09Z

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 2a081ac0b9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

github-actions · 2026-08-25T22:20:38Z

Workflow run and artifacts

Performance Benchmarks

Compared 13 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 13 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 4.367s -> 4.366s (-0.0%).

Workload	Benchmark	Target	Size	Files	Baseline	Current	Change	Status
`nested-payload-review`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_raw]`	`nested_raw`	78 B	1	279.9us	291.2us	+4.0%	stable
`padded-multi-stream-upload`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_padded_multi_stream_upload`	`multi_stream_padded`	4.1 KiB	1	332.5us	338.3us	+1.7%	stable
`direct-malicious-upload`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_direct_malicious_upload`	`malicious_reduce`	52 B	1	218.8us	221.5us	+1.2%	stable
`duplicate-heavy-registry`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_registry_snapshot`	`registry-snapshot`	915.2 KiB	13	576.97ms	570.47ms	-1.1%	stable
`suspicious-pickle-intake`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_suspicious_pickle_intake`	`suspicious-intake`	183.8 KiB	4	148.27ms	146.86ms	-0.9%	stable
`nested-payload-review`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_base64]`	`nested_base64`	98 B	1	290.5us	293.2us	+0.9%	stable
`nested-payload-review`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_hex]`	`nested_hex`	130 B	1	306.4us	309.1us	+0.9%	stable
`mixed-model-repository`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_release_candidate_repository`	`release-candidate`	547.3 KiB	32	641.46ms	637.54ms	-0.6%	stable
`warm-cache-rescan`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_warm_cached_repository_rescan`	`release-candidate`	547.3 KiB	32	151.17ms	151.90ms	+0.5%	stable
`rejected-basic-auth-candidates`	`tests/benchmarks/test_scan_benchmarks.py::test_rejected_basic_auth_candidates_scan_linearly`	`-`	371.1 KiB	1	2.522s	2.532s	+0.4%	stable
`single-checkpoint-preflight`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_single_checkpoint_before_load`	`single_checkpoint.pkl`	183.0 KiB	1	104.51ms	104.68ms	+0.2%	stable
`clean-training-checkpoint`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_clean_training_checkpoint`	`safe_large`	278.2 KiB	1	109.20ms	109.29ms	+0.1%	stable
`chunked-upload-stream`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_upload_stream`	`chunked_stream`	278.2 KiB	1	112.38ms	112.44ms	+0.1%	stable

chatgpt-codex-connector · 2026-08-25T22:20:57Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: 2a081ac0b9

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…ss-time-monitor

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b0c35b219e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

mldangelo-oai · 2026-08-29T08:18:33Z

@codex review

mldangelo-oai · 2026-08-29T08:18:34Z

@codex security review

chatgpt-codex-connector · 2026-08-29T08:19:39Z

Codex Review: Didn't find any major issues. 👍

Reviewed commit: b0c35b219e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-29T08:20:36Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: b0c35b219e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector · 2026-08-29T08:20:37Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: b0c35b219e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d94a3da2e7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

mldangelo-oai · 2026-08-29T09:46:31Z

@codex review

mldangelo-oai · 2026-08-29T09:46:32Z

@codex security review

chatgpt-codex-connector · 2026-08-29T09:48:21Z

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: d94a3da2e7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-29T09:49:11Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: d94a3da2e7

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector · 2026-08-29T09:49:18Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: d94a3da2e7

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

mldangelo-oai · 2026-08-29T10:30:06Z

@codex review

mldangelo-oai · 2026-08-29T10:30:06Z

@codex security review

chatgpt-codex-connector · 2026-08-29T10:31:51Z

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 54d6e9e74e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-29T10:32:41Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: 54d6e9e74e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector · 2026-08-29T10:32:43Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: 54d6e9e74e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

fix(cache): ignore macOS file access-time events

2a081ac

mldangelo-oai added codex codex-automation labels Aug 25, 2026

This was referenced Aug 26, 2026

fix(ci): route hash timing test to performance lane #1824

Merged

fix(network): reject side-effectful model-card image examples #1825

Merged

Merge remote-tracking branch 'origin/main' into fix/cache-darwin-acce…

b0c35b2

…ss-time-monitor

chatgpt-codex-connector Bot reviewed Aug 29, 2026

View reviewed changes

Comment thread modelaudit/cache/scan_results_cache.py Outdated

fix: preserve Darwin cache attribute monitoring

d94a3da

chatgpt-codex-connector Bot reviewed Aug 29, 2026

View reviewed changes

Comment thread modelaudit/cache/scan_results_cache.py Outdated

fix: safely settle Darwin cache events

54d6e9e

mldangelo-oai merged commit b5341b5 into main Aug 29, 2026
48 of 50 checks passed

mldangelo-oai deleted the fix/cache-darwin-access-time-monitor branch August 29, 2026 12:12

github-actions Bot mentioned this pull request Aug 29, 2026

chore: release main #1785

Open

Uh oh!

Conversation

mldangelo-oai commented Aug 25, 2026

Summary

Why

Verification

Uh oh!

chatgpt-codex-connector Bot commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

mldangelo-oai commented Aug 25, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 25, 2026

Uh oh!

github-actions Bot commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Benchmarks

Uh oh!

chatgpt-codex-connector Bot commented Aug 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

mldangelo-oai commented Aug 29, 2026

Uh oh!

mldangelo-oai commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

mldangelo-oai commented Aug 29, 2026

Uh oh!

mldangelo-oai commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 29, 2026

Uh oh!

mldangelo-oai commented Aug 29, 2026

Uh oh!

mldangelo-oai commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 29, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

chatgpt-codex-connector Bot commented Aug 25, 2026 •

edited

Loading

github-actions Bot commented Aug 25, 2026 •

edited

Loading