Skip to content

feat(analytics): inject flat tenant filter in the compiler's shared WHERE (#1967) - #2086

Merged
cyberantonz merged 6 commits into
constructorfabric:mainfrom
cyberantonz:pres/1967-tenant-filter
Jul 31, 2026
Merged

feat(analytics): inject flat tenant filter in the compiler's shared WHERE (#1967)#2086
cyberantonz merged 6 commits into
constructorfabric:mainfrom
cyberantonz:pres/1967-tenant-filter

Conversation

@cyberantonz

@cyberantonz cyberantonz commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

What

Replace the single-tenant MVP no-op with a server-injected, leading tenant_id = <ctx> predicate in the metric_results compiler — the one place every structured contract read compiles through. A request scoped to tenant A can no longer read tenant B's rows.

  • Carry the resolved tenant on ValidatedMetricResultsRequest (from the SecurityContext already passed to validate_request); the value never comes from client SQL.
  • The predicate leads every contract read: all observation reads via metric_where / shared_observation_where (batch, timeseries, ranking, capped, breakdown, histogram) and both peer-cohort CTE reads. No read path bypasses it.

Two settled decisions

Out of scope (by design, documented)

  • The legacy per-metric query_ref path stays unfiltered: its arbitrary FROM shapes (subqueries, bare bronze tables) make a flat predicate unsafe. Its stale/misleading comments are corrected.
  • Saved-query raw SELECTs use the author-driven {tenant} param ([pres] Query named params (tenant + period) #1966).
  • Hierarchy/subtree scoping and the row-policy backstop remain deferred to the isolation benchmark.

Specs

Extended the presentation PRD + DESIGN: marked cpt-presentation-fr-tenant-filter, cpt-presentation-nfr-tenant-isolation, and cpt-presentation-component-metric-compiler implemented; recorded the actual contract column. Per-artifact cfs validate passes with 0 errors.

Tests

  • cargo test -p analytics — 485 pass. Added tenant_predicate_leads_and_binds_context_tenant_on_every_contract_read (asserts the predicate and its bound value lead every read, with balanced ?/param counts); updated 3 param-order tests.
  • cargo clippy -p analytics --all-targets clean; cargo fmt clean.
  • Unblocks the cross-tenant e2e isolation test (e2e: cross-tenant data-row isolation test (blocked on ClickHouse tenant filter) #1359).

Closes #1967
Part of #1803

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added tenant-aware filtering to analytics observation and peer-cohort queries.
    • Tenant context is captured during request validation and applied consistently across supported query types.
  • Bug Fixes

    • Improved tenant isolation across periods, timeseries, rankings, breakdowns, histograms, and peer queries.
    • Corrected tenant identifier usage and query parameter handling.
  • Documentation

    • Updated analytics specifications to reflect supported tenant filtering and isolation coverage.

@cyberantonz
cyberantonz requested a review from a team as a code owner July 31, 2026 02:32
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The metric-results validation path now carries a tenant UUID into the compiler. Observation and peer-cohort SQL reads apply tenant predicates with aligned bindings. Tests, fixtures, handler comments, and specifications reflect the tenant-aware behavior.

Changes

Tenant filtering

Layer / File(s) Summary
Validated tenant context
src/backend/services/analytics/src/domain/metric_results/validation.rs, src/backend/services/analytics/src/domain/metric_results/batch.rs, src/backend/services/analytics/src/domain/metric_results/builder.rs
ValidatedMetricResultsRequest stores tenant_id. Validation and test fixtures populate the field.
Compiler tenant predicates
src/backend/services/analytics/src/domain/metric_results/compiler.rs
Observation and peer-cohort reads add tenant_id predicates. Parameter builders and tests verify binding order and placeholder alignment.
Handler and specification alignment
src/backend/services/analytics/src/api/handlers.rs, docs/domain/presentation-layer/specs/*
Comments and specifications describe tenant_id filtering and exclude the legacy execute_metric_query path.
Ingestion fixture tenant values
src/ingestion/tests/e2e/metrics/templates/*
Reusable metric templates now use fixed non-zero tenant UUIDs.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

  • Issue 2087: Addresses tenant isolation in analytics SQL reads and overlaps with the structured-read and legacy-path distinction in this change.

Possibly related PRs

Suggested reviewers: ktursunov

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 52.94% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: injecting a flat tenant filter in the analytics compiler's shared WHERE clause.
Linked Issues check ✅ Passed The PR satisfies issue #1967 by replacing the compiler no-op with a SecurityContext-bound tenant_id predicate in the shared WHERE clause.
Out of Scope Changes check ✅ Passed The documentation, comments, fixtures, and tests directly support the tenant-filtering implementation and validation.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/backend/services/analytics/src/api/handlers.rs`:
- Around line 297-301: Protect execute_metric_query before tenant callers can
use the legacy query_ref path by restricting it to prevalidated tenant-safe
sources or enforcing tenant scope safely per query. In
docs/domain/presentation-layer/specs/DESIGN.md:55 and :67, limit shipment and
isolation claims to structured metric_results reads; in :290-298, document the
legacy path as outside the guarantee unless protected. In
docs/domain/presentation-layer/specs/PRD.md:210-212 and :288, keep contract-read
tenant isolation and its NFR incomplete until the legacy path is protected and
covered by isolation tests.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7ff4f844-dc2e-4c1a-8914-d325d40e213a

📥 Commits

Reviewing files that changed from the base of the PR and between db65ea1 and 79db011d80bcb5d5806d36b5d82a98e38c08e6fe.

📒 Files selected for processing (7)
  • docs/domain/presentation-layer/specs/DESIGN.md
  • docs/domain/presentation-layer/specs/PRD.md
  • src/backend/services/analytics/src/api/handlers.rs
  • src/backend/services/analytics/src/domain/metric_results/batch.rs
  • src/backend/services/analytics/src/domain/metric_results/builder.rs
  • src/backend/services/analytics/src/domain/metric_results/compiler.rs
  • src/backend/services/analytics/src/domain/metric_results/validation.rs

Comment thread src/backend/services/analytics/src/api/handlers.rs
cyberantonz and others added 2 commits July 31, 2026 11:38
…HERE (constructorfabric#1967)

Replace the single-tenant MVP no-op with a server-injected `tenant_id = <ctx>`
predicate in the metric_results compiler — the one place every structured
contract read compiles through. The predicate leads every observation read
(`metric_where` / `shared_observation_where`) and both peer-cohort CTE reads, so
a request scoped to tenant A cannot read tenant B's rows.

- Carry the resolved tenant on `ValidatedMetricResultsRequest` (from the
  SecurityContext already passed to `validate_request`); never from client SQL.
- Filter on `tenant_id`, the column the gold observation and cohort contract
  exposes (silver `insight_tenant_id` aliased to `tenant_id`); this surface is
  internally consistent, so the constructorfabric#1596 name drift does not affect it. Value is
  the raw tenant UUID — the representation the metric lineage stamps (no
  sipHash; that is identity-only, constructorfabric#1550).
- Leave the legacy per-metric `query_ref` path unfiltered by design: its
  arbitrary FROM shapes make a flat predicate unsafe; its stale/misleading
  comments are corrected.
- Extend the presentation PRD/DESIGN: mark the tenant-filter FR, NFR, and
  compiler component implemented; record the actual contract column name.

Closes constructorfabric#1967
Part of constructorfabric#1803

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
…ts path

Address review: the compiler filter (constructorfabric#1967) covers only the structured
metric_results reads, not the legacy execute_metric_query query_ref path, which
runs arbitrary FROM shapes and stays unscoped. Narrow the FR/NFR completion
claims accordingly — the compiler component is done, but the umbrella "every
contract read" FR and the isolation NFR remain open until the legacy path is
protected or restricted to tenant-safe sources. Document the legacy path as an
explicit boundary outside the guarantee.

Signed-off-by: Anton Zelenov <antonz@constructor.tech>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/domain/presentation-layer/specs/DESIGN.md (1)

281-281: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Align tenant-filter documentation with one authoritative contract.

Line 281 and Line 361 still say the tenant-row filter is deferred to #1967 and reference insight_tenant_id, while the Metric Compiler contract at Line 300 uses tenant_id for the injected tenant predicate. Update these statements consistently. If saved-query execution intentionally bypasses the metric-results compiler, document that exception explicitly.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/domain/presentation-layer/specs/DESIGN.md` at line 281, Update the
tenant-filter statements in DESIGN.md, including the referenced deferred-work
text, to use the authoritative tenant predicate field tenant_id instead of
insight_tenant_id and align the wording with the Metric Compiler contract. If
saved-query execution bypasses the metric-results compiler, explicitly document
that exception in the relevant section.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@docs/domain/presentation-layer/specs/DESIGN.md`:
- Line 281: Update the tenant-filter statements in DESIGN.md, including the
referenced deferred-work text, to use the authoritative tenant predicate field
tenant_id instead of insight_tenant_id and align the wording with the Metric
Compiler contract. If saved-query execution bypasses the metric-results
compiler, explicitly document that exception in the relevant section.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1f83d768-0edb-41ca-baad-48c6cc872184

📥 Commits

Reviewing files that changed from the base of the PR and between 79db011d80bcb5d5806d36b5d82a98e38c08e6fe and 79d6307a220a9d2d37359903c8636a2894e314d6.

📒 Files selected for processing (4)
  • docs/domain/presentation-layer/specs/DESIGN.md
  • docs/domain/presentation-layer/specs/PRD.md
  • src/backend/services/analytics/src/domain/metric_results/builder.rs
  • src/backend/services/analytics/src/domain/metric_results/validation.rs
🚧 Files skipped from review as they are similar to previous changes (3)
  • docs/domain/presentation-layer/specs/PRD.md
  • src/backend/services/analytics/src/domain/metric_results/builder.rs
  • src/backend/services/analytics/src/domain/metric_results/validation.rs

@cyberantonz
cyberantonz force-pushed the pres/1967-tenant-filter branch from 79d6307 to 6b63360 Compare July 31, 2026 03:39
@cyberantonz
cyberantonz enabled auto-merge July 31, 2026 03:41
The compiler now binds the request's SecurityContext tenant as a leading
tenant_id predicate on every gold read, so seeded bronze rows must carry
the tenant the harness's gateway JWT is scoped to. The nil-UUID stamp
matched nothing and every metric smoke test read zero rows.

Signed-off-by: Anton Zelenov <antonz@constructor.tech>
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Regenerate the connectors-ddl snapshot

This PR changes src/ingestion/**. If your change affects any
bronze / silver / gold schema, regenerate the committed DDL snapshot
and include it in this PR.

Prerequisites (details: src/ingestion/scripts/bootstrap-db/README.md):

  • docker + a fresh throwaway ClickHouse 25.7.5 (README "Local ClickHouse for testing")
  • .env from .env.bootstrap.example pointing at it; use the host LAN IP,
    reachable from both the host and connector containers
    (host.docker.internal does not resolve on the macOS host itself)
  • python3.12 or python3.11 on PATH (pinned dbt venv)
  • HubSpot + Salesforce credentials in .env — their discover calls the
    live APIs; without them, apply ../connectors-ddl/{hubspot,salesforce}.sql
    (relative to bootstrap-db/) to seed their bronze, then run the dbt step
cd src/ingestion/scripts/bootstrap-db
set -a; source pins.env; source .env; set +a
./bootstrap-db.sh connectors-config.yaml   # fresh ClickHouse 25.7.5
./dump-ddl.sh                              # writes scripts/connectors-ddl/*.sql

Commit the resulting scripts/connectors-ddl/*.sql diff. If nothing
changed, no snapshot update is needed. (Regeneration is manual for now.)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
docs/domain/presentation-layer/specs/DESIGN.md (1)

300-300: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Clarify that the tenant value is server-side bound.

“Literal tenant predicate” can imply SQL string interpolation. State that the compiler emits the predicate and binds the raw tenant UUID from SecurityContext as a server-side parameter.

  • docs/domain/presentation-layer/specs/DESIGN.md#L300-L300: describe the predicate as server-generated with a bound tenant value.
  • docs/domain/presentation-layer/specs/PRD.md#L212-L212: replace “literal” with wording that explicitly requires server-side binding.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/domain/presentation-layer/specs/DESIGN.md` at line 300, The tenant
predicate requirements must explicitly use a server-generated predicate with the
raw tenant UUID from SecurityContext bound as a server-side parameter, never SQL
interpolation. Update docs/domain/presentation-layer/specs/DESIGN.md lines
300-300 accordingly, and replace “literal” with explicit server-side binding
language in docs/domain/presentation-layer/specs/PRD.md lines 212-212.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/domain/presentation-layer/specs/DESIGN.md`:
- Line 55: Clarify the architecture-wide tenant guarantee in the overview and
the component-boundary wording near execute_metric_query to state that it
applies only to structured metric_results reads. Keep the legacy
execute_metric_query path explicitly outside this guarantee until it is
tenant-scoped, and ensure no surrounding text claims that every read is
currently isolated.

---

Nitpick comments:
In `@docs/domain/presentation-layer/specs/DESIGN.md`:
- Line 300: The tenant predicate requirements must explicitly use a
server-generated predicate with the raw tenant UUID from SecurityContext bound
as a server-side parameter, never SQL interpolation. Update
docs/domain/presentation-layer/specs/DESIGN.md lines 300-300 accordingly, and
replace “literal” with explicit server-side binding language in
docs/domain/presentation-layer/specs/PRD.md lines 212-212.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c9e139a4-b094-4a32-913c-6394d698d2dd

📥 Commits

Reviewing files that changed from the base of the PR and between 79d6307a220a9d2d37359903c8636a2894e314d6 and e39f06a.

📒 Files selected for processing (23)
  • docs/domain/presentation-layer/specs/DESIGN.md
  • docs/domain/presentation-layer/specs/PRD.md
  • src/backend/services/analytics/src/api/handlers.rs
  • src/backend/services/analytics/src/domain/metric_results/batch.rs
  • src/backend/services/analytics/src/domain/metric_results/builder.rs
  • src/backend/services/analytics/src/domain/metric_results/compiler.rs
  • src/backend/services/analytics/src/domain/metric_results/validation.rs
  • src/ingestion/tests/e2e/metrics/templates/bitbucket_git.yaml
  • src/ingestion/tests/e2e/metrics/templates/chatgpt_team_chat.yaml
  • src/ingestion/tests/e2e/metrics/templates/chatgpt_team_codex.yaml
  • src/ingestion/tests/e2e/metrics/templates/claude_enterprise_users.yaml
  • src/ingestion/tests/e2e/metrics/templates/claude_team_usage.yaml
  • src/ingestion/tests/e2e/metrics/templates/cursor_events.yaml
  • src/ingestion/tests/e2e/metrics/templates/cursor_usage.yaml
  • src/ingestion/tests/e2e/metrics/templates/git_activity.yaml
  • src/ingestion/tests/e2e/metrics/templates/jira_task.yaml
  • src/ingestion/tests/e2e/metrics/templates/m365_email.yaml
  • src/ingestion/tests/e2e/metrics/templates/m365_onedrive.yaml
  • src/ingestion/tests/e2e/metrics/templates/m365_teams.yaml
  • src/ingestion/tests/e2e/metrics/templates/outline_wiki_comments.yaml
  • src/ingestion/tests/e2e/metrics/templates/outline_wiki_pages.yaml
  • src/ingestion/tests/e2e/metrics/templates/people.yaml
  • src/ingestion/tests/e2e/metrics/templates/wiki_page_versions.yaml
🚧 Files skipped from review as they are similar to previous changes (5)
  • src/backend/services/analytics/src/domain/metric_results/batch.rs
  • src/backend/services/analytics/src/domain/metric_results/builder.rs
  • src/backend/services/analytics/src/domain/metric_results/validation.rs
  • src/backend/services/analytics/src/api/handlers.rs
  • src/backend/services/analytics/src/domain/metric_results/compiler.rs

| `cpt-presentation-fr-saved-query-crud` | The saved query (`presentation.queries` logically; the `saved_queries` table physically) is a SeaORM entity in the analytics **service database (MariaDB)**, like metric definitions; CRUD mutates that metadata, not ClickHouse. Only `/run` reaches ClickHouse — it reuses the existing read path and executes the stored SQL as `presentation_ro`, so no write grant on the contract is ever needed. Shipped (#1965) |
| `cpt-presentation-fr-query-params` | Named parameters, `tenant` always injected from context (not client SQL), `period` supported |
| `cpt-presentation-fr-tenant-filter` | Literal `insight_tenant_id = <ctx.tenant>` injected in one place — the compiler's shared `WHERE` — replacing the no-op |
| `cpt-presentation-fr-tenant-filter` | Literal leading `tenant_id = <ctx.tenant>` injected in one place — the compiler's shared `WHERE` (and the peer-cohort CTE reads) — replacing the no-op. `tenant_id` is the column the gold observation and cohort contract exposes (silver's `insight_tenant_id`, aliased to `tenant_id` in gold); filtering on it sidesteps the #1596 name drift, which affects other tables, not this read surface. Shipped for the structured `metric_results` read path (#1967). The legacy per-metric `query_ref` path (`execute_metric_query`) remains unscoped and is explicitly outside this guarantee until protected — see the component boundaries below. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Qualify the architecture-wide tenant guarantee.

The architecture overview states that every read is tenant-scoped, but this section and Line 308 exclude execute_metric_query. State that the guarantee applies to structured metric_results reads, or update the overview. Otherwise, the design claims stronger isolation than this PR provides.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/domain/presentation-layer/specs/DESIGN.md` at line 55, Clarify the
architecture-wide tenant guarantee in the overview and the component-boundary
wording near execute_metric_query to state that it applies only to structured
metric_results reads. Keep the legacy execute_metric_query path explicitly
outside this guarantee until it is tenant-scoped, and ensure no surrounding text
claims that every read is currently isolated.

@cyberantonz
cyberantonz added this pull request to the merge queue Jul 31, 2026
Merged via the queue into constructorfabric:main with commit 5c941d5 Jul 31, 2026
44 checks passed
@cyberantonz
cyberantonz deleted the pres/1967-tenant-filter branch July 31, 2026 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[pres] Flat tenant data-row filter

2 participants