Skip to content

fix(hooks): make transcript ingest wing configurable - #2070

Open
grtninja wants to merge 9 commits into
MemPalace:developfrom
grtninja:fix/2057-hook-transcript-wing
Open

fix(hooks): make transcript ingest wing configurable#2070
grtninja wants to merge 9 commits into
MemPalace:developfrom
grtninja:fix/2057-hook-transcript-wing

Conversation

@grtninja

Copy link
Copy Markdown
Contributor

Summary

Closes #2057.

Hook-driven transcript mining is currently hardcoded to --wing sessions, even though the diary checkpoint path already derives a stable project wing from the transcript JSONL cwd/path. This adds a backward-compatible routing policy:

{
  "hooks": {
    "transcript_wing": "sessions"
  }
}
  • sessions remains the default and preserves current behavior.
  • project derives the stable wing through the existing _wing_from_transcript_path() logic.
  • MEMPALACE_HOOK_TRANSCRIPT_WING overrides file config.
  • invalid values fail safe to sessions rather than silently relocating memories.

Implementation

  • Add MempalaceConfig.hook_transcript_wing with sessions|project validation.
  • Resolve the transcript wing once for both direct and daemon hook write routes.
  • Keep ambiguous project derivation on sessions rather than creating a surprise wing_sessions migration.
  • Include the selected wing in daemon mine dedupe keys so the same source can be intentionally routed to a different wing without sharing an in-flight slot.
  • Preserve the existing direct-spawn per-target guard, whose command hash already includes --wing.

Test-first verification

The repair branch was created from current upstream develop and used a temporary owner-controlled test-first workflow. The workflow:

  1. added the regression tests and observed the pre-fix failures;
  2. applied the minimal config/routing change;
  3. ran the focused transcript-wing tests;
  4. ran tests/test_hooks_cli.py and tests/test_hook_write_routing.py;
  5. ran Ruff, compileall, and git diff --check;
  6. removed its temporary repair script/workflow before committing.

The submitted branch contains only:

  • mempalace/config.py
  • mempalace/hooks_cli.py
  • tests/test_hooks_cli.py

Risk

Low and opt-in. Existing installations remain on the historical sessions wing unless they explicitly select project. No storage schema, existing drawer, or default hook behavior is changed.

@fatkobra fatkobra left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: hooks.transcript_wing=project derives one wing from the triggering transcript and then mines path.parent using that single wing.

That assumption is valid for Claude Code’s per-project transcript directory, but not for Codex’s date-bucketed sessions/YYYY/MM/DD/ layout. One Codex date directory can contain rollout files from several projects.

A hook triggered by project A can therefore submit the whole date directory with project A’s wing, causing sibling transcripts from projects B or C to be filed or re-filed into A’s wing. The daemon route has the same problem because it also submits source=path.parent.

Please do one of the following:

  1. scope this directory-level policy to transcript layouts where the parent directory is project-specific;
  2. mine only the triggering transcript; or
  3. partition files by each transcript’s independently derived project wing before mining/submission.

Please add a regression with two Codex rollout files in the same date directory but with different cwd/project values, and verify that neither file is routed through the other file’s wing.

grtninja commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-08T22:00:38Z

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable
  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / aa89bd82272f55381206c83b6f306e79351824eb (develop)
  • State: OPEN / ready / mergeable=false; changed files=3
  • Checks/workflows observed at this head: Docker=action_required, Tests=action_required
  • Review state: 1 reviews / 0 review threads / 0 current unresolved
  • Contributor guidance recheck: AGENTS.md, CONTRIBUTING.md, SECURITY.md
  • Exact-surface overlap: none detected among this open-PR union
  • Owner next bounded action: Owner: resolve the current merge conflict against the exact base before requesting further review.
  • Bot/review policy: no duplicate bot trigger and no human maintainer nudge; re-read this exact head/diff before any review reply.

This is a public-safe coordination receipt only. No push, merge, publication, credential, runtime, model, GPU, auth, or protected-reasoning mutation was performed or authorized by this pass.

grtninja commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-09T04:01:16Z

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable
  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / aa89bd82272f55381206c83b6f306e79351824eb (develop)
  • State: OPEN / ready / mergeable=false; changed files=3
  • Checks/workflows observed at this head: Docker=action_required, Tests=action_required
  • Review state: 1 reviews / 0 review threads / 0 current unresolved
  • Contributor guidance recheck: AGENTS.md, CONTRIBUTING.md, SECURITY.md
  • Exact-surface overlap: none detected among this open-PR union
  • Owner next bounded action: Owner: resolve the current merge conflict, then address the CHANGES_REQUESTED transcript-wing scope/partition finding and add the requested two-rollout regression before re-review.
  • Bot/review policy: no duplicate bot trigger and no human maintainer nudge; re-read this exact head/diff before any review reply.

This is a public-safe coordination receipt only. No push, merge, publication, credential, runtime, model, GPU, auth, or protected-reasoning mutation was performed or authorized by this pass.

grtninja commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-09T10:00:48Z

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable
  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / aa89bd82272f55381206c83b6f306e79351824eb (develop)
  • State: OPEN / ready / mergeable=false; changed files=3
  • Checks/workflows observed at this head: Docker=action_required, Tests=action_required
  • Review state: 1 reviews / 0 review threads / 0 current unresolved
  • Current unresolved paths: none
  • Contributor guidance recheck: AGENTS.md, CONTRIBUTING.md, SECURITY.md
  • Exact-surface overlap: none detected among this open-PR union
  • Owner next bounded action: Owner: resolve the current merge conflict against develop, address the CHANGES_REQUESTED transcript-wing scope/partition finding, add the requested two-rollout regression, then rerun focused tests.
  • Bot/review policy: no duplicate bot trigger and no human maintainer-review nudge; re-read current head/diff before any reply.
  • Coordination boundary: public-safe packet only; no push, merge, publish, protected model/reasoning mutation, or process restart in this pass.

grtninja commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-09T16:01:56Z

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable
  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / aa89bd82272f55381206c83b6f306e79351824eb (develop)
  • State: OPEN / ready / mergeable=false; observed merge-state=DIRTY; changed files=3
  • Checks/workflows at this head: Docker=action_required, Tests=action_required
  • Review state: 1 reviews (latest CHANGES_REQUESTED by fatkobra) / 0 review threads / 0 current unresolved
  • Current unresolved paths: none
  • Contributor guidance recheck: AGENTS.md, CONTRIBUTING.md, SECURITY.md
  • Exact-surface overlap: none detected among this open-PR union
  • Owner next bounded action: Owner: resolve the develop conflict, address the current CHANGES_REQUESTED transcript-wing scope/partition issue, add the two-rollout regression, then rerun focused tests.
  • Bot/review policy: no duplicate bot trigger and no human maintainer-review nudge; re-read current head/diff before any reply.
  • Coordination boundary: public-safe packet only; no push, merge, publish, protected model/reasoning mutation, or process restart in this pass.

grtninja commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-09T22:00:57Z

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable
  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / aa89bd82272f55381206c83b6f306e79351824eb (develop)
  • State: OPEN / ready / mergeable=false; observed merge-state=DIRTY; changed files=3
  • Checks/workflows at this head: Docker=action_required, Tests=action_required
  • Review state: 1 reviews (latest CHANGES_REQUESTED by fatkobra) / 0 review threads / 0 current unresolved
  • Current unresolved paths: none
  • Contributor guidance recheck: AGENTS.md, CONTRIBUTING.md, SECURITY.md
  • Exact-surface overlap: none detected among this open-PR union
  • Owner next bounded action: Owner: resolve the develop conflict, address current CHANGES_REQUESTED transcript-wing scope/partition, add the two-rollout regression, then rerun focused tests.
  • Bot/review policy: no duplicate bot trigger and no human maintainer-review nudge; re-read current head/diff before any reply.
  • Coordination boundary: public-safe packet only; no push, merge, publish, protected model/reasoning mutation, or process restart in this pass.

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-10T04:00:32Z

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable
  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / aa89bd82272f55381206c83b6f306e79351824eb (develop)
  • State: OPEN / ready / mergeable=false; observed merge-state=DIRTY; changed files=3
  • Checks/workflows at this head: Docker=action_required, Tests=action_required
  • Review state: 1 reviews / 0 review threads / 0 current unresolved
  • Current unresolved paths: none
  • Contributor guidance recheck: AGENTS.md, CONTRIBUTING.md, SECURITY.md
  • Exact-surface overlap: none detected among this open-PR union
  • Owner next bounded action: Owner: resolve the develop conflict, address current CHANGES_REQUESTED transcript-wing scope/partition, add the two-rollout regression, then rerun focused tests.
  • Bot/review policy: no duplicate bot trigger and no human maintainer-review nudge; re-read current head/diff before any reply.
  • Coordination boundary: public-safe packet only; no push, merge, publish, protected model/reasoning mutation, or process restart in this pass.

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-10T10:00:38Z

  • Fresh 10:01Z refresh: authoritative 34-PR union unchanged versus 04:00Z; no head, state, merge-state, review/thread, check, or changed-file deltas.

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable

  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / aa89bd82272f55381206c83b6f306e79351824eb (develop)

  • State: OPEN / ready / mergeable=false; observed merge-state=DIRTY; changed files=3

  • Checks/workflows at this head: Docker=action_required, Tests=action_required

  • Review state: 1 reviews / 0 review threads / 0 current unresolved

  • Current unresolved paths: none

  • Contributor guidance recheck: AGENTS.md, CONTRIBUTING.md, SECURITY.md

  • Exact-surface overlap: none detected among this open-PR union

  • Owner next bounded action: Owner: resolve the develop conflict, address current CHANGES_REQUESTED transcript-wing scope/partition, add the two-rollout regression, then rerun focused tests.

  • Bot/review policy: no duplicate bot trigger and no human maintainer-review nudge; re-read current head/diff before any reply.

  • Coordination boundary: public-safe packet only; no push, merge, publish, protected model/reasoning mutation, or process restart in this pass.

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-10 16:01:44Z

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-10 22:01:41Z

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-11 04:00:16Z

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-11 10:01:08Z

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable
  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / base aa89bd82272f55381206c83b6f306e79351824eb (develop)
  • State: OPEN; mergeability unknown (REST mergeable=unknown)
  • Fresh evidence: author/involves union refreshed at 2026-08-11T10:01:08Z; 34/34 PR info, reviews, threads, issue comments, checks, and file lists returned successfully. Direct REST refresh is 34/34; structural comparison vs 04:00Z has no head/base/state/draft/changed-file movement; three MemPalace mergeability fields are transient unknown and remain recheck-only. The 28 REST comment-count increments are the expected 04:00 coordination packets; no substantive post-04 issue-comment delta was observed.
  • Reviews/threads: 1 review submissions; 0 inline review threads; 0 currently unresolved.
  • Checks: Docker:action_required, Tests:action_required.
  • Changed surface: 3 file(s). Contributor guidance, exact-surface overlap, and current-head review state were rechecked.
  • Owner next action: Owner: re-read the current head and diff, address any current review findings, and run the narrowest relevant validation.
  • Priority blockers: none observed on the current review-thread snapshot.
  • Coordination guard: no duplicate bot trigger, no human maintainer nudge, no push/merge/publish, and no protected model/reasoning mutation. G07 Main Arbiter remains unchanged; this packet preserves the recipient's current model/reasoning.

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-11 22:00:42Z

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable
  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / base aa89bd82272f55381206c83b6f306e79351824eb (develop)
  • State: OPEN; mergeability dirty (REST mergeable=false)
  • Fresh evidence: author/involves union refreshed at 2026-08-11T22:00:42Z; current open union remains 32 PRs across 22 repositories (30 authored plus 2 owned Dependabot). 32/32 PR-info, reviews, threads, issue comments, checks, and file lists returned successfully. Direct REST rechecked all 34 historical records: 32 remain open, with no current head/base/state/draft/merge-state/mergeability/changed-file movement versus 16:00Z. The 26 one-comment count increments are expected from the 16:00 packets; no substantive post-16:00 issue-comment delta was observed.
  • Reviews/threads: 1 review submissions; 0 inline review threads; 0 currently unresolved.
  • Checks: Docker:action_required, Tests:action_required.
  • Changed surface: 3 file(s). Contributor guidance, exact-surface overlap, base freshness, and current-head review state were rechecked.
  • Owner next action: Owner: refresh the base/check gate and resolve the dirty merge blocker before requesting review or merge.
  • Priority blockers: none observed on current review-thread snapshot.
  • Coordination guard: no duplicate bot trigger, no human maintainer nudge, no push/merge/publish, and no protected model/reasoning mutation. G07 Main Arbiter remains unchanged; this packet preserves the recipient's current model/reasoning.

Copy link
Copy Markdown
Contributor Author

Current-head coordination packet — 2026-08-12 04:01:48Z

  • PR: fix(hooks): make transcript ingest wing configurable #2070 — fix(hooks): make transcript ingest wing configurable
  • Exact head/base: 8bb47598324fbae8d27644fd31bd6ea211534974 / base aa89bd82272f55381206c83b6f306e79351824eb (develop)
  • State: OPEN; mergeability dirty (REST mergeable=false)
  • Fresh evidence: author/involves union refreshed at 2026-08-12T04:01:48Z; current open union remains 32 PRs across 22 repositories (30 authored plus 2 owned Dependabot). 32/32 PR-info, reviews, threads, issue comments, checks, and file lists returned successfully. Direct REST rechecked all 34 historical records: 32 remain open, with no current head/base/state/draft/merge-state/mergeability/changed-file movement versus 22:00Z. The 26 one-comment count increments are expected from the 22:00 packets; no substantive post-22:00 issue-comment delta was observed.
  • Reviews/threads: 1 review submissions; 0 inline review threads; 0 currently unresolved.
  • Checks: Docker:action_required, Tests:action_required.
  • Changed surface: 3 file(s). Contributor guidance, exact-surface overlap, base freshness, and current-head review state were rechecked.
  • Owner next action: Owner: refresh the base/check gate and resolve the dirty merge blocker before requesting review or merge.
  • Priority blockers: none observed on current review-thread snapshot.
  • Coordination guard: no duplicate bot trigger, no human maintainer nudge, no push/merge/publish, and no protected model/reasoning mutation. G07 Main Arbiter remains unchanged; this packet preserves the recipient's current model/reasoning.

@ATKabli

ATKabli commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Independent confirmation from a small production fleet (2 nodes, shared pgvector
palace): the "sessions" hardcode this PR targets is a real problem, not a
theoretical one. We hit it running MemPalace hooks on two machines against one
shared palace — every node's live transcript ingest converged into the same
sessions wing regardless of which agent/machine produced it, because the wing is
hardcoded at both _ingest_transcript call sites (the daemon submit payload and the
_spawn_mine argv). This PR is the only one of the three open hooks-wing PRs that
actually touches both of those call sites, which matches what we needed to fix.

Where our patch differs: we made the wing an arbitrary operator-named value
(MEMPALACE_HOOKS_WING env, falling back to a hooks_wing config key, falling back
to the historical sessions default) rather than a sessions/project policy
toggle. For a multi-node fleet where the same repos/transcript layouts get checked
out on more than one machine, project-derived wings would still collide across
nodes — we needed each node to declare its own stable wing name (snake,
meiling), not have one derived from the transcript path. Might be worth a third
policy value here (transcript_wing: "<name>") alongside sessions/project, or a
parallel env var read at the same two call sites.

One more data point in favor of getting this right at ingest rather than fixing it
up later: we had 1,772 drawers land in the wrong wing before we shipped the fix, and
re-winging them was not a metadata edit — drawer ids embed the wing prefix but the
content hash used to derive the id excludes wing, so moving a drawer to the correct
wing means rewriting its id. That's the same shape of problem the reviewer flagged
on #2071's migrate-wings --rename (tunnel endpoints going stale after a wing
rename) — once ids/references bake in a wrong wing, fixing it after the fact is
lossy. Getting the wing right at the two _ingest_transcript call sites, before
anything is written, is the cheap fix; anything downstream of that is a migration
with edge cases.

Also flagging: the Codex date-bucket sibling-project concern raised above is real
and would bite an operator running Codex hooks against a shared palace the same way
the sessions hardcode bit us — worth resolving before this merges regardless of
which config model wins.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hook transcript ingest: derive a stable per-project wing instead of hardcoded "sessions"

3 participants