test(skills): deflake config-cache invalidation on coarse-mtime filesystems - #65037
Closed
vadimcomanescu wants to merge 1 commit into
Closed
test(skills): deflake config-cache invalidation on coarse-mtime filesystems#65037vadimcomanescu wants to merge 1 commit into
vadimcomanescu wants to merge 1 commit into
Conversation
…ystems The raw config cache keys on (path, st_mtime_ns, st_size). In test_skill_config_raw_cache_invalidates_on_config_edit both configs are the same byte length, so mtime is the only field that can invalidate the entry. The test rewrote the file and called os.utime(config_path, None), stamping the current time — on filesystems with coarse (1s) mtime granularity the second write can share the first write's mtime when both land in the same second, so the stale entry stays cached and the final assertion flakes. Bump the mtime a full second past its current value instead, which survives second-granularity truncation and makes the invalidation deterministic on every filesystem. No production change.
tonydwb
reviewed
Jul 15, 2026
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved
Looks Good
- Test(skills): deflake config-cache invalidation on coarse-mtime filesystems
- 10 additions, 1 deletion — test improvement
- No issues detected
Reviewed by Hermes Agent
Contributor
|
Thanks @vadimcomanescu — closing: the sole target test (skill_config_raw_cache_invalidates…) was removed in the suite-wide test prune (PR #74383). The coarse-mtime deflake approach was sound — if the class resurfaces in a surviving test, this design is the right one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Deflakes
tests/agent/test_skill_utils.py::test_skill_config_raw_cache_invalidates_on_config_edit, which fails intermittently on CI runners whose filesystem has coarse (1-second) mtime granularity.The raw config cache in
agent/skill_utils.pykeys entries on(path, st_mtime_ns, st_size)(introduced in #46149). The test writesconfig.yamlwithdisabled: [old-skill], reads it (populating the cache), rewrites it withdisabled: [new-skill], then asserts the next read reflects the edit. Both configs are the same byte length (32 bytes), sost_sizeis identical andst_mtime_nsis the only field that can invalidate the entry.The test forces a mtime change with
os.utime(config_path, None)— added in #46149 alongside the cache precisely to make the edit detectable. ButNonestamps the current time: on a filesystem with 1s mtime granularity, when both writes land in the same second the second write shares the first write's mtime, the cache key doesn't change, the stale entry stays cached, and the final assertion flakes. Runners with nanosecond mtime resolution never hit this, which is why it passes almost everywhere and only flakes under coarse-granularity/loaded runners.The fix keeps that intent but makes it deterministic: stamp an mtime a full second past the file's current value, which survives second-granularity truncation. It's a test-only change — the cache implementation is unchanged and correct for real usage, where config edits are always more than one mtime tick apart.
Related Issue
No tracked issue — surfaced as an intermittent CI failure. Searched open and merged PRs/issues for
skill_config_raw_cache/get_disabled_skill_names cacheand found no existing report or fix. Cache and test both originate in #46149.Type of Change
Changes Made
tests/agent/test_skill_utils.py: replaceos.utime(config_path, None)with an explicit+1smtime bump (os.utime(config_path, ns=(bumped_ns, bumped_ns))) so the cache-invalidation assertion is deterministic under coarse mtime granularity. Comment added explaining why.How to Test
scripts/run_tests.sh tests/agent/test_skill_utils.py -q— 24 passed.mtime change ⇒ invalidation) is exactly what the test still verifies.Checklist
Code
test(skills): ...)Documentation & Housekeeping
cli-config.yaml.example— N/A (no config keys)CONTRIBUTING.md/AGENTS.md— N/A (no architecture/workflow change)🤖 Generated with Claude Code