Skip to content

fix(identity-resolution): correct config environment keys - #2029

Merged
aleksdotbar merged 8 commits into
mainfrom
fix/compose-identity-resolution-env
Jul 31, 2026
Merged

fix(identity-resolution): correct config environment keys#2029
aleksdotbar merged 8 commits into
mainfrom
fix/compose-identity-resolution-env

Conversation

@aleksdotbar

@aleksdotbar aleksdotbar commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace hyphenated identity-resolution gear segments with shell-safe underscores
  • align Compose, Helm, GitOps secret generation, ingestion setup, and documentation
  • restore MariaDB and ClickHouse environment overrides for identity-resolution

Validation

  • docker compose --env-file .env.compose.example -f docker-compose.yml config --quiet
  • bash -n deploy/gitops/scripts/compose-app-secrets.sh
  • rendered the umbrella Helm chart with identity-resolution enabled
  • verified no dashed identity-resolution overrides remain

Summary by CodeRabbit

  • Bug Fixes
    • Standardized identity-resolution environment-variable keys to use the underscored identity_resolution namespace across deployments, secrets, database settings, and tenant configuration.
  • Documentation
    • Updated local-run instructions, Helm guidance, chart comments, and configuration examples to reflect the corrected environment-variable format.
  • Tests
    • Enhanced identity-resolution end-to-end tooling to support implementation-specific seed triggers and Rust CLI seed execution.

Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
(cherry picked from commit de64d7566f8878bf1d081706c31bfcf696d0f6f4)
@aleksdotbar
aleksdotbar requested a review from a team as a code owner July 29, 2026 19:41
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Regenerate the connectors-ddl snapshot

This PR changes src/ingestion/**. If your change affects any
bronze / silver / gold schema, regenerate the committed DDL snapshot
and include it in this PR.

Prerequisites (details: src/ingestion/scripts/bootstrap-db/README.md):

  • docker + a fresh throwaway ClickHouse 25.7.5 (README "Local ClickHouse for testing")
  • .env from .env.bootstrap.example pointing at it; use the host LAN IP,
    reachable from both the host and connector containers
    (host.docker.internal does not resolve on the macOS host itself)
  • python3.12 or python3.11 on PATH (pinned dbt venv)
  • HubSpot + Salesforce credentials in .env — their discover calls the
    live APIs; without them, apply ../connectors-ddl/{hubspot,salesforce}.sql
    (relative to bootstrap-db/) to seed their bronze, then run the dbt step
cd src/ingestion/scripts/bootstrap-db
set -a; source pins.env; source .env; set +a
./bootstrap-db.sh connectors-config.yaml   # fresh ClickHouse 25.7.5
./dump-ddl.sh                              # writes scripts/connectors-ddl/*.sql

Commit the resulting scripts/connectors-ddl/*.sql diff. If nothing
changed, no snapshot update is needed. (Regeneration is manual for now.)

@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Identity-resolution environment-variable overrides now use identity_resolution instead of identity-resolution across deployment configuration, Rust runtime wiring, E2E helpers, Helm jobs, tests, and documentation. The E2E harness also adds Rust seed CLI capability detection and execution.

Changes

Identity-resolution configuration and E2E wiring

Layer / File(s) Summary
Environment-key deployment wiring
charts/insight/..., deploy/gitops/scripts/compose-app-secrets.sh, docker-compose.yml, src/backend/services/identity-resolution/helm/templates/*, src/backend/services/identity-resolution/helm/values.yaml, src/backend/services/identity-resolution/seed/*
Deployment secrets, Compose services, and Helm jobs use APP__gears__identity_resolution__config__* keys.
E2E identity runtime support
src/ingestion/tests/e2e/lib/identity.py
The E2E identity process exposes seed capabilities, runs the Rust seed CLI, and uses underscored Rust configuration overrides.
Configuration contracts and validation
src/backend/services/identity-resolution/README.md, src/backend/services/identity-resolution/config/insight.yaml, src/backend/services/identity-resolution/src/{config.rs,gear.rs}, src/backend/services/identity-resolution/helm/templates/deployment.yaml, src/backend/services/identity-resolution/helm/tests/test_seed_cronjob_contract.py
Documentation, comments, and Helm contract assertions use the underscored environment-key convention.

Estimated code review effort: 2 (Simple) | ~15 minutes

Possibly related PRs

Suggested reviewers: mitasovr

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: correcting identity-resolution configuration environment keys.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/compose-identity-resolution-env

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@charts/insight/templates/secrets.yaml`:
- Around line 317-338: Refresh documentation at
charts/insight/templates/secrets.yaml lines 317-338 to distinguish hyphenated
YAML gear names from underscored environment-variable keys; update _rust_env()
in src/ingestion/tests/e2e/lib/identity.py lines 303-309 so its docstring no
longer claims every gear segment remains hyphenated; and revise
src/backend/services/identity-resolution/README.md line 42 to remove the
obsolete requirement to use env instead of export.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a6cfe1e6-7822-4873-aa0f-cf4d25fe6194

📥 Commits

Reviewing files that changed from the base of the PR and between b1ecd78 and 39fa58f.

📒 Files selected for processing (11)
  • charts/insight/templates/secrets.yaml
  • charts/insight/values.yaml
  • deploy/gitops/scripts/compose-app-secrets.sh
  • docker-compose.yml
  • src/backend/services/identity-resolution/README.md
  • src/backend/services/identity-resolution/config/insight.yaml
  • src/backend/services/identity-resolution/helm/templates/deployment.yaml
  • src/backend/services/identity-resolution/helm/values.yaml
  • src/backend/services/identity-resolution/src/config.rs
  • src/backend/services/identity-resolution/src/gear.rs
  • src/ingestion/tests/e2e/lib/identity.py

Comment thread charts/insight/templates/secrets.yaml
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
cyberantonz
cyberantonz previously approved these changes Jul 30, 2026
@cyberantonz
cyberantonz self-requested a review July 30, 2026 03:07
@cyberantonz
cyberantonz dismissed their stale review July 30, 2026 03:07

Nope. I changed my mind.

@aleksdotbar
aleksdotbar enabled auto-merge July 30, 2026 08:07

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docker-compose.yml (1)

121-124: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Provide an upgrade path for the new ClickHouse init scripts.

clickhouse-data persists the database directory, so upgrading existing Compose installs will not pick up these new 01-presentation-role.sql / 02-presentation-user.sql scripts under /docker-entrypoint-initdb.d. Analytics requires the presentation user, so add an idempotent upgrade migration or document a safe reset/migration step; otherwise fresh-install validation can pass while existing environments fail to connect read-only.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docker-compose.yml` around lines 121 - 124, Add an upgrade path for existing
persistent ClickHouse installations alongside the new `01-presentation-role.sql`
and `02-presentation-user.sql` mounts: provide an idempotent migration that
creates the read-only presentation role and user when absent, or document the
required safe migration/reset procedure. Ensure existing `clickhouse-data`
volumes receive the `presentation` user without affecting fresh initialization.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@docker-compose.yml`:
- Around line 121-124: Add an upgrade path for existing persistent ClickHouse
installations alongside the new `01-presentation-role.sql` and
`02-presentation-user.sql` mounts: provide an idempotent migration that creates
the read-only presentation role and user when absent, or document the required
safe migration/reset procedure. Ensure existing `clickhouse-data` volumes
receive the `presentation` user without affecting fresh initialization.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 43f5a4fc-bf4e-4af3-8215-3e294b8db442

📥 Commits

Reviewing files that changed from the base of the PR and between 39fa58f and f249533.

📒 Files selected for processing (6)
  • charts/insight/templates/secrets.yaml
  • charts/insight/values.yaml
  • deploy/gitops/scripts/compose-app-secrets.sh
  • docker-compose.yml
  • src/backend/services/identity-resolution/README.md
  • src/ingestion/tests/e2e/lib/identity.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/ingestion/tests/e2e/lib/identity.py
  • src/backend/services/identity-resolution/README.md

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/ingestion/tests/e2e/lib/identity.py (1)

305-310: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Use the underscored key for the CLI tenant override.

Line 308 still uses identity-resolution, so run_seed_cli(tenant=...) does not set tenant_default_id. Fresh-database seed runs will therefore fail tenant resolution instead of using the supplied tenant.

Proposed fix
-            env["APP__gears__identity-resolution__config__tenant_default_id"] = tenant
+            env["APP__gears__identity_resolution__config__tenant_default_id"] = tenant
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/ingestion/tests/e2e/lib/identity.py` around lines 305 - 310, Update the
tenant environment override in run_seed_cli to use the underscored configuration
key identity_resolution instead of identity-resolution, so the supplied tenant
reaches tenant_default_id during fresh-database seed runs. Preserve the existing
conditional override and extra_env merge behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/ingestion/tests/e2e/lib/identity.py`:
- Around line 305-310: Update the tenant environment override in run_seed_cli to
use the underscored configuration key identity_resolution instead of
identity-resolution, so the supplied tenant reaches tenant_default_id during
fresh-database seed runs. Preserve the existing conditional override and
extra_env merge behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 28178ce4-a2b8-44e0-b07b-c3c252e037c9

📥 Commits

Reviewing files that changed from the base of the PR and between f249533 and ce4d8c4.

📒 Files selected for processing (5)
  • charts/insight/templates/secrets.yaml
  • charts/insight/values.yaml
  • src/backend/services/identity-resolution/helm/values.yaml
  • src/backend/services/identity-resolution/src/gear.rs
  • src/ingestion/tests/e2e/lib/identity.py

@aleksdotbar
aleksdotbar added this pull request to the merge queue Jul 30, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Jul 30, 2026
@aleksdotbar
aleksdotbar enabled auto-merge July 30, 2026 21:28
Normalize the identity-resolution env-key segments arriving with the seed CronJob and seed CLI to underscores.

Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
Normalize the identity-resolution env-key segment on the incoming sync CronJob and its e2e rig to underscores.

Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/backend/services/identity-resolution/helm/tests/test_seed_cronjob_contract.py (1)

1-14: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Align the CronJob count assertion with the stated contract.

The module documentation says the suite must tolerate a third CronJob and select jobs by name. The test still requires exactly len(JOBS) rendered CronJobs. A future unrelated CronJob will fail this test.

Remove the total-count assertion, or restrict the count to identity-resolution CronJobs.

Proposed test adjustment
-def test_default_render_ships_exactly_the_two_documented_cronjobs(default_docs) -> None:
-    names = sorted(_cronjobs(default_docs))
-    assert len(names) == len(JOBS), names
+def test_default_render_ships_documented_cronjobs(default_docs) -> None:
     for job in JOBS:
         _cronjob(default_docs, job)

Also applies to: 169-174

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/backend/services/identity-resolution/helm/tests/test_seed_cronjob_contract.py`
around lines 1 - 14, Remove the exact total CronJob count assertion from the
test module, since the contract must tolerate unrelated CronJobs. Keep selecting
and validating the identity-resolution jobs by their names through the existing
JOBS-based logic, preserving all per-job assertions.
🧹 Nitpick comments (1)
src/backend/services/identity-resolution/src/gear.rs (1)

110-117: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Keep rustdoc only on shared-library APIs.

This code is in the identity-resolution service module. If the crate is a binary or service crate, remove the new /// block. Keep it only if Cargo.toml defines this module as part of a shared library.

As per coding guidelines: use /// documentation comments only on exported items in shared library crates; do not add them to binaries or services.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/backend/services/identity-resolution/src/gear.rs` around lines 110 - 117,
Remove the added rustdoc block describing the sync subcommand unless Cargo.toml
confirms this module belongs to a shared-library crate; for the
identity-resolution service binary, keep the subcommand implementation unchanged
and omit the `///` documentation comments.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In
`@src/backend/services/identity-resolution/helm/tests/test_seed_cronjob_contract.py`:
- Around line 1-14: Remove the exact total CronJob count assertion from the test
module, since the contract must tolerate unrelated CronJobs. Keep selecting and
validating the identity-resolution jobs by their names through the existing
JOBS-based logic, preserving all per-job assertions.

---

Nitpick comments:
In `@src/backend/services/identity-resolution/src/gear.rs`:
- Around line 110-117: Remove the added rustdoc block describing the sync
subcommand unless Cargo.toml confirms this module belongs to a shared-library
crate; for the identity-resolution service binary, keep the subcommand
implementation unchanged and omit the `///` documentation comments.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d9ebddff-818b-44d1-9352-62b106804885

📥 Commits

Reviewing files that changed from the base of the PR and between ce4d8c4 and a6feb81.

📒 Files selected for processing (11)
  • charts/insight/templates/secrets.yaml
  • charts/insight/values.yaml
  • deploy/gitops/scripts/compose-app-secrets.sh
  • docker-compose.yml
  • src/backend/services/identity-resolution/README.md
  • src/backend/services/identity-resolution/helm/templates/seed-cronjob.yaml
  • src/backend/services/identity-resolution/helm/templates/sync-cronjob.yaml
  • src/backend/services/identity-resolution/helm/tests/test_seed_cronjob_contract.py
  • src/backend/services/identity-resolution/helm/values.yaml
  • src/backend/services/identity-resolution/seed/smoke-orgchart-visibility.sh
  • src/backend/services/identity-resolution/src/gear.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/backend/services/identity-resolution/README.md
  • charts/insight/templates/secrets.yaml

@aleksdotbar
aleksdotbar requested a review from a team July 31, 2026 09:46
@aleksdotbar
aleksdotbar added this pull request to the merge queue Jul 31, 2026
Merged via the queue into main with commit 1c268ac Jul 31, 2026
52 of 55 checks passed
@aleksdotbar
aleksdotbar deleted the fix/compose-identity-resolution-env branch July 31, 2026 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants