fix(utils): atomic writers and their callers refuse to resurrect a deleted named profile home

[kokhlo]

Summary

hermes profile delete removes the tree and writes a tombstone under profiles/.deleted/<name>, but a background writer that still carries the dead profile as its Hermes home could bring profiles/<name>/ back: the atomic writers created the target's parent with a bare mkdir(parents=True), and so did many callers right before invoking them. Observed as profiles/<name>/cache/reasoning_caps.json reappearing ~0.3 s after a successful delete (the reasoning-caps-warm daemon thread finishing its catalog fetch) — the same resurrection class #97128 closed for logging and state.

Fixes #112592.

Complementary to #112594 (opened before my claim comment): that PR guards _atomic_write + _write_json_cache; this one additionally covers the caller sites that mkdir the parent themselves right before a writer — the report calls out that "even a guarded writer would find the home already re-created" when the caller has already brought the home back. Surfaces here do not overlap beyond the two shared sites (rebased on the same main). Maintainer's call on the split; happy to rebase or drop either half.

What changed

• utils._atomic_write, atomic_roundtrip_yaml_update, atomic_roundtrip_yaml_save: parent-dir creation now goes through mkdir_under_hermes_home, the fix(profiles): deleted named profiles stay deleted — tombstones block logging resurrection (#89438, salvage #90141) #97128 guard — a tombstoned (or missing) named profile home raises FileNotFoundError instead of being materialized. Non-profile paths are unaffected (named_profile_home returns None -> plain mkdir).
• Caller sites that mkdir the parent themselves right before a writer, so the home is never created in the first place:
  • hermes_cli/models.py:_write_json_cache (model/pricing/reasoning-caps caches)
  • agent/models_dev.py:_save_etag (models.dev ETag sidecar)
  • gateway/lifecycle_ledger.py:_write_sentinel + _append_exit_diag (gateway lifecycle logs)
  • agent/secret_sources/_cache.py:atomic_write_json (secret cache entries)
  • tools/mcp_oauth.py (refresh fence, token/client JSON, CIMD marker, snapshot restore)
  • tools/memory_tool_store.py (load, file lock, add/replace/remove writes)

All touched write paths were already best-effort wrapped (debug-log, _quietly, or try/except OSError), so a background writer hitting a deleted profile now degrades to a debug log instead of resurrecting the directory.

Tests

New tests/utils/test_atomic_writers_deleted_profile.py (11 tests):

• each writer on a tombstoned named profile home raises FileNotFoundError / leaves nothing on disk — including the exact late reasoning-caps save from the report (via set_hermes_home_override, platform-neutral), the models cache, lifecycle sentinel, OAuth tokens, memory store and the roundtrip YAML update
• positive controls: a custom home whose path merely contains a profiles segment (/srv/profiles/buildcache), the default .hermes home, and a plain tmp path all still create parents and write — mirroring test_unrelated_profiles_dir_still_mkdirs
• verified RED->GREEN: reverting only the _atomic_write + _write_json_cache mkdir swaps fails exactly the four resurrection tests; the full fix passes 11/11

Adjacent suites stay green: test_deleted_profile_tombstone.py, test_reasoning_caps_disk_cache.py, test_utils_atomic_roundtrip_yaml_save.py (35 passed), test_mcp_oauth.py + test_memory_tool.py + test_multiplex_lifecycle.py (123 passed; 2 pre-existing macOS callback-port failures reproduced on clean main via git stash).

[kvnloo]

Deleting a profile should mean it's gone. Before this, a background daemon — the reasoning-caps warmer in the report — could re-create profiles/<name>/ about 0.3 s after a successful delete. The delete looks unreliable, and the zombie directory leaves stale state behind for a future profile of the same name. This closes the hole for the atomic writers and, importantly, for the caller sites that were mkdir'ing the parent ahead of them — the report's point that a guarded writer alone wouldn't help when the caller has already brought the home back. Approving.

The mechanism checks out. mkdir_under_hermes_home (hermes_constants.py:288) runs assert_named_profile_home_live before the mkdir, raising FileNotFoundError for tombstoned or missing named profile homes. named_profile_home correctly returns None for the default home even when the path merely contains a profiles segment, and the tests pin the positive controls (/srv/profiles/buildcache, default .hermes, plain tmp). I verified the legitimate recreate flow is safe: create_profile (hermes_cli/profiles.py:899) calls clear_named_profile_deleted and builds in a staging sibling published by rename — it never goes through the guarded writers, so refusing to materialize can't block a real re-creation. The best-effort claim holds too: _quietly (agent/models_dev.py:175) catches Exception and FileNotFoundError is an OSError, so the models.dev paths degrade to a debug log; atomic_write_json's docstring already documents "Raises OSError on failure; callers decide whether that is best-effort," so the new raise lands inside the existing contract.

Non-blocking: load_on_disk_store() (tools/memory_tool.py:48) documents "never raises," but store.load_from_disk() sits outside the try block and can now raise FileNotFoundError for a deleted named profile home. The CLI /memory path (hermes_cli/cli_commands_mixin.py:1861) calls it with no handler. Either move the load inside the try or correct the docstring — the contract as written is now false for exactly the case this PR introduces.

Non-blocking: the function-level from hermes_constants import mkdir_under_hermes_home is unnecessary in agent/models_dev.py, agent/secret_sources/_cache.py, and tools/mcp_oauth.py — all three already import from hermes_constants at module top level (openrouter_variant_base, secure_parent_dir). Extending the existing top-level import is exactly as safe as the lazy one.

Non-blocking: mkdir_under_hermes_home is check-then-mkdir, so a profile delete landing between the liveness assert and the mkdir still resurrects. The window shrank from "every background write" to "a write racing a concurrent delete" — inherent without a lock, just naming the residual.

One design thread, offered not required: this is the third PR chasing the same class (#97128 for logging/state, #112594 for the writers, this one for the caller mkdirs), and a grep still finds ~40 bare parent.mkdir(parents=True, ...) call sites. Per-site discipline will keep leaking; a lint rule flagging bare parent-mkdir on home-derived paths — or making the guarded mkdir the default import — would end the class instead of the instances.

[teknium1]

Salvaged into #112653 with your commit cherry-picked (authorship preserved). That PR lands #111927 (rename identity migration) together with the #112592 atomic-writer cluster (#112594 first-in, #112601 caller sites, #112596 sweep) on current main; overlapping same-idiom hunks were resolved to the first-landed version. Merge is gated on the maintainer; this PR will be closed with credit + SHA once that lands.

[teknium1]

Landed on main in #112653 (9085ef967c) with your commits cherry-picked and authorship intact — thank you, @kokhlo. Closing here since the merge went through the consolidated carrier.