Skip to content

feat(identity-resolution): make attribute policy and account assignment readable from the warehouse - #2313

Closed
aleksdotbar wants to merge 2 commits into
feat/person-attribute-registryfrom
feat/person-attribute-snapshots
Closed

feat(identity-resolution): make attribute policy and account assignment readable from the warehouse#2313
aleksdotbar wants to merge 2 commits into
feat/person-attribute-registryfrom
feat/person-attribute-snapshots

Conversation

@aleksdotbar

Copy link
Copy Markdown
Contributor

Stacked on #2285 (person-attribute registry) — based on
feat/person-attribute-registry, so the diff shows only this change once
that merges.

Problem

The query path cannot see attribute policy or account-to-person assignment.
Policy is administrative state edited elsewhere; assignment is only implicit
in the persons journal. Without both readable from the analytical store, a
grouped or comparative request either consults a service it should not
depend on, or proceeds without knowing what it is allowed to do and who the
evidence belongs to.

Change

Two projections, reached two different ways.

  • identity.person_attribute_policy_snapshot — a published snapshot.
    Policy crosses a store boundary, so it is copied: full snapshot, count
    verification, atomic swap. Keys stay the raw warehouse strings, so the
    relation joins the claim relations byte-equal.
  • identity.person_account_assignments_current — a view. The persons
    journal is already in ClickHouse and is replaced wholesale by an atomic
    swap, so nothing needs copying: the view is consistent with its source by
    construction, and its revision is the journal's own publication stamp.
    Latest-wins is keyed by account rather than by person — the two disagree
    exactly when an account is rebound, which is the case this relation exists
    to resolve.

Supporting changes:

  • Publishing mechanics (staging, count verification, watermark guard, atomic
    swap, staging GC) move into a writer shared with the persons-log publisher
    instead of being copied alongside it.
  • publish-policy subcommand and hourly CronJob, journalled like the other
    runs. An unchanged policy set short-circuits, but only after confirming the
    published snapshot still matches what was journalled — a checksum alone
    would skip forever against a warehouse that had been wiped or re-pointed.
    Skips are journalled too, so the journal still answers whether the schedule
    fired.
  • Journal GETs under /v1/person-attributes-policy-publish, behind the
    existing admin gate.
  • The raw-identifier-to-UUID mapping becomes the insight_uuid_from_raw
    macro, replacing eight inline copies. It is the recipe for joining
    raw-keyed relations to identity ones, and writing it by hand is how such a
    join silently returns nothing.
  • The stand seeder emits account-binding rows; without them the new view is
    empty on every seeded environment.

Fix carried along

CREATE DATABASE IF NOT EXISTS cannot run on a client pinned to that
database — ClickHouse resolves the request's database first and rejects the
statement. The persons-log publisher's fresh-environment path had therefore
never worked; it survived only because the init migration creates the
database. The bootstrap DDL now goes through a database that always exists.

Out of scope

The attribute catalog and the grouping and comparison APIs; mirroring the
account-person map (the projection contract deliberately allows swapping its
producer later).

Validation

  • 105 unit tests; clippy pedantic clean; helm render-contract 29 green;
    dbt parse clean; OpenAPI drift gate passing on the regenerated spec.
  • Against MariaDB 11.4 and ClickHouse 25.7.5: first run publishes; an
    unchanged set skips; a warehouse wiped behind the journal re-publishes
    rather than skipping; a policy edit republishes at the new revision; every
    run is journalled, skips included.
  • Against seeded journal data: a rebound account resolves to the later
    person, email observations stay out of the projection, and the policy
    snapshot joins the assignment view through the documented recipe.

Notes for the consuming layer

  • Policy joins the claim relations raw-to-raw; the assignment view joins them
    by hashing the claims side with insight_uuid_from_raw, account ids raw.
  • An absent policy row must be treated restrictively — the decision to allow
    publishing an empty snapshot depends on it.
  • Policy and assignment are pinned independently and refreshed by different
    jobs, so they are two diagnostics values, not one.

Closes #2312

The query path reads both from the analytical store, so it consults no
administrative service per request.

- Policy crosses a store boundary, so it is published: a full snapshot with
  count verification and an atomic swap. Mechanics are now shared with the
  persons-log publisher rather than copied.
- Assignment crosses nothing — the persons journal is already in ClickHouse
  and swapped wholesale — so it is a view. Zero copy, and consistent with
  its source by construction rather than by schedule.
- Latest-wins is keyed by account, not by person: the two disagree exactly
  when an account is rebound, which is what the relation exists to answer.
- Unchanged policy short-circuits, but only after confirming the published
  snapshot still matches what was journalled; a checksum alone would skip
  forever against a warehouse that had been wiped.
- CREATE DATABASE cannot run on a client pinned to that database, so the
  bootstrap DDL now goes through one that always exists. The publisher's
  fresh-environment path had never worked.
- The raw-to-UUID identifier mapping becomes a named macro, since it is the
  recipe for joining raw-keyed relations to identity ones.

Refs #2028

Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0b4b126d-2d34-473f-9fef-975dba5b56b1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

An admin who edits attribute policy could not make it take effect without
cluster credentials — the publish ran only on its schedule. The seed's HTTP
trigger was removed with its in-process queue and 503-when-full path, but that
reasoning was the queue's, not the trigger's, and it had been carried forward
three times without being re-derived.

The handler decides admission synchronously — gate, tenant, lock, journal row —
then hands an admitted run to a detached task. Back-pressure is the advisory
lock, so a concurrent run is a 409 rather than a queue that can fill; nothing
buffered comes back. The caller is recorded as the run's author, and the
journal names which path started it.

Both paths resolve the journal tenant the same way, and a caller outside that
tenant is refused: one snapshot keeps one journal, or the short-circuit stops
firing and callers cannot read the runs they started.

Comments in these files now follow the workspace rules: no module headers, no
doc comments, tagged lines only where code cannot carry the fact. Endpoint
paths are gone from handler docs — they live in the route table.

Promotes identity-resolution to a generated schema target, which the drift gate
has demanded since its document started describing the service rather than the
retired one. The internal routes keep hand-written models; nothing publishes
them.

Closes #2312

Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant