feat(age): close drawer-sync gap with unique index + delete-through

[jphein]

Summary

Closes the drawer-sync gap between the relational mempalace_drawers table and the AGE graph. Two paired hygiene fixes plus a small follow-up cleanup.

1. Drawer.id unique index (_ensure_drawer_unique_index)

Runs on graph bootstrap, installs a unique B-tree on properties->>'id'. AGE MERGE (d:Drawer {id: X}) has a read-then-create gap that isn't atomic — two concurrent writers both miss and CREATE, producing duplicate Drawer nodes. The index makes the second CREATE fail at the Postgres layer. 844 dupes accumulated on familiar.jphe.in before manual cleanup + index install.

The installer is defensive:

• skips silently when the Drawer label hasn't been created yet (no writethrough fired)
• skips if the table still has dupes (cleanup hasn't run yet)

2. Delete-through hook (make_age_deletethrough)

Mirrors the writethrough on the delete path. PostgresCollection.delete() now resolves doomed ids before the SQL DELETE, then invokes kg.delete_drawers() so the AGE Drawer nodes (and their incident MENTIONS edges) go away with the source rows. Without this, every drawer delete leaked an orphan Drawer node into the graph; 254 orphans accumulated on familiar before manual cleanup.

3. Ruff 0.15.14 format + missing logger (chore commit)

Forward-compat with the v3.3.6 ruff pin (#222):

• f-string quote normalization in knowledge_graph_age.py + test_age_kg_units.py
• collapse short logger.warning to single line
• adds missing import logging + module logger in knowledge_graph_age.py — fixes a latent F821 on the new logger.debug call in the defensive index-installer branch. This bug pre-exists in the a9e15b8 commit; would have surfaced under any lint run.

Test plan

• All Python parses
• ruff check + ruff format --check pass on mempalace/ and tests/test_age_kg_units.py
• CI green
• 7 new tests pass (make_age_deletethrough / make_null_deletethrough / make_deletethrough_from_env)

Note on the missing 3rd commit

The original task listed three local commits: ea7f733 (docs), a9e15b8 (code), 15fed7e (changelog). ea7f733 was a state-refresh that's already been further superseded on origin/main (drawer count claims, backfill status, daemon-host references all overtaken by 5+ newer commits). Cherry-picking it would have meant reverting newer doc updates. The current values (365K+ drawers, 3234 tests, daemon on familiar) are preserved in this PR via the changelog-cherry-pick conflict resolution.

PR ordering

Best reviewed after #222 (upstream v3.3.6 sync) merges so the ruff 0.15.14 pin matches the format in this PR. Should rebase cleanly.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses synchronization gaps between the relational mempalace_drawers table and the Apache AGE graph. By introducing a unique index on drawer IDs and implementing a delete-through hook, the system now ensures data consistency and prevents the accumulation of duplicate or orphaned nodes in the graph.

Highlights

• Unique Index: Added a unique B-tree index on Drawer.id to prevent duplicate nodes in the AGE graph during concurrent write operations.
• Delete-through Hook: Implemented a delete-through hook to ensure that deleting rows from the relational mempalace_drawers table automatically removes corresponding nodes and edges from the AGE graph.
• Maintenance: Updated code formatting to match Ruff 0.15.14 standards and resolved a missing logger import in knowledge_graph_age.py.

Ignored Files

• Ignored by pattern: FORK_CHANGELOG.md (1)
  • FORK_CHANGELOG.md

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a delete-through hook to propagate drawer deletions from the relational Postgres database to the Apache AGE graph, preventing orphan Drawer nodes. It also adds a unique index on Drawer.properties->>'id' during graph bootstrap to close a concurrent write race. The review feedback highlights two critical issues: first, a potential data drift bug in PostgresCollection.delete where the hook could be notified of deletions that never occurred if both ids and where filters are used; second, a performance bottleneck in _ensure_drawer_unique_index which performs an expensive sequential scan on every startup instead of checking if the index already exists first.

[gemini-code-assist]

If both ids and where are provided, the DELETE statement will only remove the subset of ids that also match the where filter. However, deleted_ids is currently set to the entire ids list, causing the delete-through hook to be notified of deletions that never actually occurred in the relational database. This leads to data drift between the relational table and the AGE graph.

Restricting the fast-path assignment to cases where where is absent ensures we query the database to resolve the exact matching IDs when a filter is present.

Suggested change

	deleted_ids = list(ids) if ids else None
	deleted_ids = list(ids) if (ids and not where) else None

[gemini-code-assist]

The _ensure_drawer_unique_index method runs on every graph bootstrap (which occurs during every initialization of KnowledgeGraphAGE). Performing a sequential scan and COUNT(DISTINCT) on the Drawer table on every startup is extremely expensive and will severely degrade startup performance as the graph grows.

Checking if the drawer_id_unique index already exists in pg_indexes first allows us to return early and completely avoid this expensive query on subsequent startups.

Suggested change

	cur.execute(
	f"SELECT COUNT(*) - COUNT(DISTINCT (properties::text)::jsonb->>'id') "
	f'FROM "{graph}"."Drawer"'
	)
	dup_excess = cur.fetchone()[0]
	cur.execute(
	"SELECT 1 FROM pg_indexes WHERE schemaname = %s AND indexname = 'drawer_id_unique'",
	(graph,),
	)
	if cur.fetchone() is not None:
	return
	cur.execute(
	f"SELECT COUNT(*) - COUNT(DISTINCT (properties::text)::jsonb->>'id') "
	f'FROM "{graph}"."Drawer"'
	)
	dup_excess = cur.fetchone()[0]