feat: improve AI personal analytics queries - #1514
Merged
Merged
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds AI personal analytics ClickHouse views, bronze placeholder schema updates, two SeaORM migrations for metric queries and catalog seeds, and migrator registration. ChangesAI Personal Analytics Migrations
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
aleksdotbar
marked this pull request as ready for review
June 29, 2026 08:28
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@src/backend/services/analytics-api/src/migration/m20260623_000002_seed_ai_personal_metric_catalog.rs`:
- Around line 128-133: The INSERT_LINK_SQL in
m20260623_000002_seed_ai_personal_metric_catalog.rs currently links every
metric_catalog row matching metric_key, which can include tenant-specific rows.
Update the SELECT used by the catalog link insert so it only targets the global
seed rows created by the migration (the ones with tenant_id set to NULL),
keeping the link scoped to the global catalog entry and avoiding accidental
links to tenant-specific catalog rows.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 4ffe0f64-b1d8-4270-ad68-8d147a1c7f93
📥 Commits
Reviewing files that changed from the base of the PR and between da10446 and a9dba614007102dc4a616fca3c4032777ca37627.
📒 Files selected for processing (4)
src/backend/services/analytics-api/src/migration/m20260623_000001_ai_personal_queries.rssrc/backend/services/analytics-api/src/migration/m20260623_000002_seed_ai_personal_metric_catalog.rssrc/backend/services/analytics-api/src/migration/mod.rssrc/ingestion/scripts/migrations/20260623000000_ai_personal_gold_views.sql
This was referenced Jun 29, 2026
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
aleksdotbar
force-pushed
the
codex/ai-personal-gold
branch
from
June 29, 2026 21:49
5d6de19 to
2c3573e
Compare
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
src/ingestion/scripts/create-bronze-placeholders.sh (2)
942-960: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUpgrade existing
bronze_zoom.participantsplaceholders too.This table predated the PR, but the new columns are only present in the create path. On warm e2e/dev ClickHouse volumes, the old placeholder survives and dbt will still see the stale contract when it selects the newly added Zoom fields. Mirror the
jira_issuepattern here withADD COLUMN IF NOT EXISTSfor the added columns.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/ingestion/scripts/create-bronze-placeholders.sh` around lines 942 - 960, The bronze_zoom.participants placeholder is missing the newly added Zoom fields on existing ClickHouse volumes, so the create script needs to backfill them for warm environments. Update the create-bronze-placeholders.sh logic around the bronze_zoom.participants definition to mirror the jira_issue pattern by issuing ADD COLUMN IF NOT EXISTS statements for the new columns, using the bronze_zoom.participants table name and the added field names from the diff so stale contracts are upgraded in place.
192-227: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winReconcile the expanded
silver.class_ai_dev_usageschema on reruns.
CREATE TABLE IF NOT EXISTSwill not add these new columns to an older placeholder that already exists on a reused cluster. That leaves the pre-PR schema in place, so the new AI personal gold-view migration can still fail before dbt has a chance to replace the table. Add anelsebranch here withALTER TABLE ... ADD COLUMN IF NOT EXISTS ..., like thebronze_jira.jira_issuereconciliation below.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/ingestion/scripts/create-bronze-placeholders.sh` around lines 192 - 227, The `silver.class_ai_dev_usage` placeholder creation only handles first-time setup, so reruns on an existing cluster can leave the old schema unchanged. Add schema reconciliation in `create-bronze-placeholders.sh` by extending the existing `if ! ch_table_exists silver class_ai_dev_usage` block with an `else` branch that uses `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` for the newly added columns, matching the pattern used for `bronze_jira.jira_issue`. Make sure the fix is applied around the `CREATE TABLE IF NOT EXISTS` logic for `silver.class_ai_dev_usage` so older placeholders are upgraded in place.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@src/ingestion/scripts/create-bronze-placeholders.sh`:
- Around line 942-960: The bronze_zoom.participants placeholder is missing the
newly added Zoom fields on existing ClickHouse volumes, so the create script
needs to backfill them for warm environments. Update the
create-bronze-placeholders.sh logic around the bronze_zoom.participants
definition to mirror the jira_issue pattern by issuing ADD COLUMN IF NOT EXISTS
statements for the new columns, using the bronze_zoom.participants table name
and the added field names from the diff so stale contracts are upgraded in
place.
- Around line 192-227: The `silver.class_ai_dev_usage` placeholder creation only
handles first-time setup, so reruns on an existing cluster can leave the old
schema unchanged. Add schema reconciliation in `create-bronze-placeholders.sh`
by extending the existing `if ! ch_table_exists silver class_ai_dev_usage` block
with an `else` branch that uses `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` for
the newly added columns, matching the pattern used for `bronze_jira.jira_issue`.
Make sure the fix is applied around the `CREATE TABLE IF NOT EXISTS` logic for
`silver.class_ai_dev_usage` so older placeholders are upgraded in place.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6546120c-90c1-4e2a-93cc-5a4c6d4b397a
📥 Commits
Reviewing files that changed from the base of the PR and between a9dba614007102dc4a616fca3c4032777ca37627 and b9a0b7e.
📒 Files selected for processing (5)
src/backend/services/analytics-api/src/migration/m20260623_000001_ai_personal_queries.rssrc/backend/services/analytics-api/src/migration/m20260623_000002_seed_ai_personal_metric_catalog.rssrc/backend/services/analytics-api/src/migration/mod.rssrc/ingestion/scripts/create-bronze-placeholders.shsrc/ingestion/scripts/migrations/20260623000000_ai_personal_gold_views.sql
🚧 Files skipped from review as they are similar to previous changes (4)
- src/backend/services/analytics-api/src/migration/mod.rs
- src/ingestion/scripts/migrations/20260623000000_ai_personal_gold_views.sql
- src/backend/services/analytics-api/src/migration/m20260623_000001_ai_personal_queries.rs
- src/backend/services/analytics-api/src/migration/m20260623_000002_seed_ai_personal_metric_catalog.rs
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
aleksdotbar
force-pushed
the
codex/ai-personal-gold
branch
from
June 29, 2026 23:04
e68af32 to
0adfda5
Compare
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/ingestion/scripts/create-bronze-placeholders.sh`:
- Around line 229-243: The ALTER TABLE block in create-bronze-placeholders.sh
for silver.class_ai_dev_usage must only run for placeholder tables, not the real
dbt-backed model. Update the else branch that issues the schema reconciliation
so it first checks the table’s placeholder marker/comment
(INSIGHT_PLACEHOLDER_v1) before applying any ADD COLUMN statements. Keep the
guard close to the existing silver.class_ai_dev_usage reconciliation path so
reruns do not mutate a materialized production table.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 8f17162d-e400-42b4-8cfb-74ad598941db
📒 Files selected for processing (3)
src/backend/services/analytics-api/src/migration/m20260623_000001_ai_personal_queries.rssrc/backend/services/analytics-api/src/migration/m20260623_000002_seed_ai_personal_metric_catalog.rssrc/ingestion/scripts/create-bronze-placeholders.sh
🚧 Files skipped from review as they are similar to previous changes (2)
- src/backend/services/analytics-api/src/migration/m20260623_000002_seed_ai_personal_metric_catalog.rs
- src/backend/services/analytics-api/src/migration/m20260623_000001_ai_personal_queries.rs
Signed-off-by: Aleksandr Barkhatov <pm@aleks.bar>
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/ingestion/scripts/create-bronze-placeholders.sh (1)
241-252: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winReconcile the full
class_ai_dev_usageplaceholder contract.This
ALTER TABLEblock still leaves older placeholders missing several columns that the create branch now defines, includingtotal_lines_added,accepted_lines_added,spec_lines,session_count,total_chat_messages,cost_cents,prs_with_cc_count, andprs_total_count. On a warm cluster, rerunning this script will therefore keepsilver.class_ai_dev_usageout of sync with the new AI analytics schema instead of repairing it.Suggested fix
ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS lines_removed Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS total_lines_added Nullable(Float64); ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS total_lines_removed Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS accepted_lines_added Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS spec_lines Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS session_count Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS total_chat_messages Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS cost_cents Nullable(UInt32); ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS commits_count Nullable(UInt32); ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS pull_requests_count Nullable(UInt32); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS prs_with_cc_count Nullable(UInt32); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS prs_total_count Nullable(UInt32); ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS tool_action_breakdown_json Nullable(String);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/ingestion/scripts/create-bronze-placeholders.sh` around lines 241 - 252, The `ALTER TABLE` reconciliation for `silver.class_ai_dev_usage` is incomplete, so `create-bronze-placeholders.sh` does not fully match the newer placeholder contract. Update the `run_ch` SQL block in the `class_ai_dev_usage` section to add the missing columns that the create path already defines, including `total_lines_added`, `accepted_lines_added`, `spec_lines`, `session_count`, `total_chat_messages`, `cost_cents`, `prs_with_cc_count`, and `prs_total_count`, alongside the existing `ADD COLUMN IF NOT EXISTS` statements. Ensure the warm-cluster repair path mirrors the current schema exactly so reruns bring `silver.class_ai_dev_usage` back into sync.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/ingestion/scripts/create-bronze-placeholders.sh`:
- Around line 1052-1090: The `create-bronze-placeholders.sh` branch for
`bronze_cursor.cursor_daily_usage` only creates the table when it does not
already exist, so stale warm-cluster placeholders never get upgraded to the new
schema. Add a reconciliation path in this section that detects an existing table
and aligns it with the current `CREATE TABLE` definition, either by adding
missing columns or recreating/migrating the placeholder safely. Use the
`ch_table_exists` check and the `run_ch` block around
`bronze_cursor.cursor_daily_usage` as the entry point for the fix.
---
Outside diff comments:
In `@src/ingestion/scripts/create-bronze-placeholders.sh`:
- Around line 241-252: The `ALTER TABLE` reconciliation for
`silver.class_ai_dev_usage` is incomplete, so `create-bronze-placeholders.sh`
does not fully match the newer placeholder contract. Update the `run_ch` SQL
block in the `class_ai_dev_usage` section to add the missing columns that the
create path already defines, including `total_lines_added`,
`accepted_lines_added`, `spec_lines`, `session_count`, `total_chat_messages`,
`cost_cents`, `prs_with_cc_count`, and `prs_total_count`, alongside the existing
`ADD COLUMN IF NOT EXISTS` statements. Ensure the warm-cluster repair path
mirrors the current schema exactly so reruns bring `silver.class_ai_dev_usage`
back into sync.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 325baf61-ab07-407b-9a2e-b7a8d86367c1
📒 Files selected for processing (1)
src/ingestion/scripts/create-bronze-placeholders.sh
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/ingestion/scripts/create-bronze-placeholders.sh (1)
241-252: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winReconcile the full
class_ai_dev_usageplaceholder contract.This
ALTER TABLEblock still leaves older placeholders missing several columns that the create branch now defines, includingtotal_lines_added,accepted_lines_added,spec_lines,session_count,total_chat_messages,cost_cents,prs_with_cc_count, andprs_total_count. On a warm cluster, rerunning this script will therefore keepsilver.class_ai_dev_usageout of sync with the new AI analytics schema instead of repairing it.Suggested fix
ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS lines_removed Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS total_lines_added Nullable(Float64); ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS total_lines_removed Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS accepted_lines_added Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS spec_lines Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS session_count Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS total_chat_messages Nullable(Float64); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS cost_cents Nullable(UInt32); ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS commits_count Nullable(UInt32); ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS pull_requests_count Nullable(UInt32); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS prs_with_cc_count Nullable(UInt32); +ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS prs_total_count Nullable(UInt32); ALTER TABLE silver.class_ai_dev_usage ADD COLUMN IF NOT EXISTS tool_action_breakdown_json Nullable(String);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/ingestion/scripts/create-bronze-placeholders.sh` around lines 241 - 252, The `ALTER TABLE` reconciliation for `silver.class_ai_dev_usage` is incomplete, so `create-bronze-placeholders.sh` does not fully match the newer placeholder contract. Update the `run_ch` SQL block in the `class_ai_dev_usage` section to add the missing columns that the create path already defines, including `total_lines_added`, `accepted_lines_added`, `spec_lines`, `session_count`, `total_chat_messages`, `cost_cents`, `prs_with_cc_count`, and `prs_total_count`, alongside the existing `ADD COLUMN IF NOT EXISTS` statements. Ensure the warm-cluster repair path mirrors the current schema exactly so reruns bring `silver.class_ai_dev_usage` back into sync.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/ingestion/scripts/create-bronze-placeholders.sh`:
- Around line 1052-1090: The `create-bronze-placeholders.sh` branch for
`bronze_cursor.cursor_daily_usage` only creates the table when it does not
already exist, so stale warm-cluster placeholders never get upgraded to the new
schema. Add a reconciliation path in this section that detects an existing table
and aligns it with the current `CREATE TABLE` definition, either by adding
missing columns or recreating/migrating the placeholder safely. Use the
`ch_table_exists` check and the `run_ch` block around
`bronze_cursor.cursor_daily_usage` as the entry point for the fix.
---
Outside diff comments:
In `@src/ingestion/scripts/create-bronze-placeholders.sh`:
- Around line 241-252: The `ALTER TABLE` reconciliation for
`silver.class_ai_dev_usage` is incomplete, so `create-bronze-placeholders.sh`
does not fully match the newer placeholder contract. Update the `run_ch` SQL
block in the `class_ai_dev_usage` section to add the missing columns that the
create path already defines, including `total_lines_added`,
`accepted_lines_added`, `spec_lines`, `session_count`, `total_chat_messages`,
`cost_cents`, `prs_with_cc_count`, and `prs_total_count`, alongside the existing
`ADD COLUMN IF NOT EXISTS` statements. Ensure the warm-cluster repair path
mirrors the current schema exactly so reruns bring `silver.class_ai_dev_usage`
back into sync.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 325baf61-ab07-407b-9a2e-b7a8d86367c1
📒 Files selected for processing (1)
src/ingestion/scripts/create-bronze-placeholders.sh
🛑 Comments failed to post (1)
src/ingestion/scripts/create-bronze-placeholders.sh (1)
1052-1090: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Add a reconciliation path for pre-existing
bronze_cursor.cursor_daily_usage.This branch only fixes fresh clusters. If a warm environment already has the older 14-column placeholder,
CREATE TABLE IF NOT EXISTSis skipped and the new dbt inputs (tenant_id,source_id,unique_key,userId,day,totalLinesAdded, etc.) never get added. That leaves reruns broken in exactly the stale-placeholder scenario this comment describes.Suggested fix
if ! ch_table_exists bronze_cursor cursor_daily_usage; then echo " Creating placeholder: bronze_cursor.cursor_daily_usage" run_ch <<'SQL' CREATE TABLE IF NOT EXISTS bronze_cursor.cursor_daily_usage ( ... ) ENGINE = ReplacingMergeTree(_airbyte_extracted_at) ORDER BY unique_key; SQL +else + echo " Reconciling placeholder schema: bronze_cursor.cursor_daily_usage" + run_ch <<'SQL' +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS tenant_id String; +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS source_id String; +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS unique_key String; +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS userId Nullable(String); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS day Nullable(String); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS date Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS isActive Nullable(UInt8); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS chatRequests Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS cmdkUsages Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS composerRequests Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS agentRequests Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS bugbotUsages Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalTabsShown Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalTabsAccepted Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalAccepts Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalApplies Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalRejects Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalLinesAdded Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalLinesDeleted Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS acceptedLinesAdded Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS acceptedLinesDeleted Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS mostUsedModel Nullable(String); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS tabMostUsedExtension Nullable(String); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS applyMostUsedExtension Nullable(String); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS clientVersion Nullable(String); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS subscriptionIncludedReqs Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS usageBasedReqs Nullable(Float64); +ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS apiKeyReqs Nullable(Float64); +SQL fi📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.if ! ch_table_exists bronze_cursor cursor_daily_usage; then echo " Creating placeholder: bronze_cursor.cursor_daily_usage" run_ch <<'SQL' CREATE TABLE IF NOT EXISTS bronze_cursor.cursor_daily_usage ( tenant_id String, source_id String, unique_key String, userId Nullable(String), email String, day Nullable(String), date Nullable(Float64), isActive Nullable(UInt8), chatRequests Nullable(Float64), cmdkUsages Nullable(Float64), composerRequests Nullable(Float64), agentRequests Nullable(Float64), bugbotUsages Nullable(Float64), totalTabsShown Nullable(Float64), totalTabsAccepted Nullable(Float64), totalAccepts Nullable(Float64), totalApplies Nullable(Float64), totalRejects Nullable(Float64), totalLinesAdded Nullable(Float64), totalLinesDeleted Nullable(Float64), acceptedLinesAdded Nullable(Float64), acceptedLinesDeleted Nullable(Float64), mostUsedModel Nullable(String), tabMostUsedExtension Nullable(String), applyMostUsedExtension Nullable(String), clientVersion Nullable(String), subscriptionIncludedReqs Nullable(Float64), usageBasedReqs Nullable(Float64), apiKeyReqs Nullable(Float64), _airbyte_raw_id String DEFAULT toString(generateUUIDv4()), _airbyte_extracted_at DateTime64(3) DEFAULT now64(3), _airbyte_meta String DEFAULT '{}', _airbyte_generation_id UInt32 DEFAULT 0 ) ENGINE = ReplacingMergeTree(_airbyte_extracted_at) ORDER BY unique_key; SQL else echo " Reconciling placeholder schema: bronze_cursor.cursor_daily_usage" run_ch <<'SQL' ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS tenant_id String; ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS source_id String; ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS unique_key String; ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS userId Nullable(String); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS day Nullable(String); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS date Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS isActive Nullable(UInt8); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS chatRequests Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS cmdkUsages Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS composerRequests Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS agentRequests Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS bugbotUsages Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalTabsShown Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalTabsAccepted Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalAccepts Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalApplies Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalRejects Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalLinesAdded Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS totalLinesDeleted Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS acceptedLinesAdded Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS acceptedLinesDeleted Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS mostUsedModel Nullable(String); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS tabMostUsedExtension Nullable(String); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS applyMostUsedExtension Nullable(String); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS clientVersion Nullable(String); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS subscriptionIncludedReqs Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS usageBasedReqs Nullable(Float64); ALTER TABLE bronze_cursor.cursor_daily_usage ADD COLUMN IF NOT EXISTS apiKeyReqs Nullable(Float64); SQL fi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/ingestion/scripts/create-bronze-placeholders.sh` around lines 1052 - 1090, The `create-bronze-placeholders.sh` branch for `bronze_cursor.cursor_daily_usage` only creates the table when it does not already exist, so stale warm-cluster placeholders never get upgraded to the new schema. Add a reconciliation path in this section that detects an existing table and aligns it with the current `CREATE TABLE` definition, either by adding missing columns or recreating/migrating the placeholder safely. Use the `ch_table_exists` check and the `run_ch` block around `bronze_cursor.cursor_daily_usage` as the entry point for the fix.
ktursunov
approved these changes
Jun 30, 2026
ktursunov
approved these changes
Jul 1, 2026
ktursunov
pushed a commit
to ktursunov/insight
that referenced
this pull request
Jul 2, 2026
… connector) Upstream's AI-personal-gold work (constructorfabric#1514) added the ai_person_counter_daily catalog vector — 10 per-person AI daily counters (accepted/removed lines, cost, active days, assistant messages/actions, chat/dev conversations, tool acceptance). Like ai_bullet_rows it needs a Cursor/Claude/ChatGPT connector to have data, so add it to SKIP_TABLES. Metric-coverage gate back to 0 missing (44/106). Signed-off-by: Konstantin Tursunov <Konstantin.Tursunov@constructor.tech>
cyberantonz
added a commit
to cyberantonz/insight
that referenced
this pull request
Jul 2, 2026
Migration 20260623_ai_personal_gold_views (from constructorfabric#1514 ai-personal-gold) selects and GROUP BYs c.source / c.source_id from silver.class_ai_dev_usage, but the seed's placeholder DDL was never updated to add those columns. On a fresh ClickHouse the view creation fails with UNKNOWN_IDENTIFIER ("Identifier 'c.source' cannot be resolved"), breaking `seed silver`. Add source/source_id (String) to the class_ai_dev_usage placeholder, matching the real silver model — same placeholder-mirror gap the file already documents for the post-20260601 columns. Not gear-related; surfaced while validating the analytics-api gear on a clean k8s cluster. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Anton Zelenov <antonz@constructor.tech>
This was referenced Jul 2, 2026
cyberantonz
added a commit
that referenced
this pull request
Jul 2, 2026
* analytics-api: run as an idiomatic gears-rust gear
Migrate analytics-api from a self-hosted axum::serve service to a
gears-rust host on toolkit::bootstrap::run_server:
- main.rs: toolkit bootstrap (AppConfig::load_or_default → run_server /
gear::run_migrate); links the api-gateway REST host + auth/authz/tenant
system gears via inventory.
- gear.rs (new): AnalyticsApiGear #[toolkit::gear(capabilities=[rest])].
Gear::init reconstructs the full former run_server startup (self-managed
MariaDB pool + 45 migrations + CHECK/product-default probes, redis
catalog cache, ClickHouse + Identity clients, schema validator,
admin-CRUD service), stores AppState in a OnceLock, and tokio::spawns the
one-shot validate_all() scan. RestApiCapability::register_rest wires the
OperationBuilder routes into the host router + Extension(AppState).
- Auth: host runs auth_disabled (the platform api-gateway is the sole
authenticator); it injects a SecurityContext with DEFAULT_TENANT_ID
(== the single-tenant default the seeded data uses). The former
hand-rolled tenant_middleware becomes a thin, stateless layer that reads
toolkit_security::SecurityContext and overrides the tenant from
X-Insight-Tenant-Id when present. The local SecurityContext struct is
removed; handlers + domain services now use toolkit_security only.
- DB stays self-managed (no toolkit `db` capability; ClickHouse is not a
toolkit-db backend anyway).
- config.rs: figment loader removed; GearConfig is deserialized from
gears.analytics-api.config via ctx.config(). New config/insight.yaml in
the gears layout (auth_disabled: true), modeled on api-gateway's
no-auth.yaml.
Build, clippy (pedantic), and tests (325 pass) all green. Helm/compose
manifests still reference the old ANALYTICS__* env model and are updated
in a follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* analytics-api: wire deploy for the gears-rust config model
The gear migration switched analytics-api from figment (`ANALYTICS__*`
env, no config file) to toolkit AppConfig (a gears-layout YAML at `-c`
plus `APP__*` env overrides). Update both deploy paths to match:
- Dockerfile: bake config/ into the image; launch
`-c /app/config/insight.yaml run`.
- docker-compose.yml: bind-mount the config dir, add `-c … run`, and
rename every `ANALYTICS__<x>` env var to
`APP__gears__analytics-api__config__<x>`.
- Helm (services/analytics-api): new configmap.yaml (the gears structure
with the auth-disabled api-gateway host); deployment mounts it at
/app/config/analytics.yaml, launches `-c … run`, fixes the migrate
initContainer likewise, and rolls on config change via checksum/config.
- Umbrella secret (charts/insight) + compose-app-secrets.sh: rename the
generated `ANALYTICS__*` keys to `APP__gears__analytics-api__config__*`
(hyphenated names delivered via envFrom, same as api-gateway's
`APP__gears__oidc-authn-plugin__*`). Structural config now lives in the
mounted ConfigMap; the Secret carries only the leaf overrides
(DB/clickhouse/redis/identity URLs + tenant_default_id).
- Drop the now-host-owned `bind_addr` env var; refresh stale comments
(the /health-400 CrashLoop rationale no longer applies — the
auth-disabled host injects DEFAULT_TENANT_ID and /health is off the
auth path).
Validated: `docker compose config -q`, `helm template` of the chart, and
the gears config YAML all parse cleanly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* style(analytics-api): cargo fmt after the gear migration
Apply rustfmt to the files touched by the gears-rust migration — the
migration left several formatting violations (fmt is not covered by
clippy, so build/clippy/test were green while `cargo fmt --check` was
not). No behavioral change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* analytics-api: use identifier-safe env-override keys for the gear
Runtime fix: with the gears-rust host, config overrides arrive as
env vars under `gears.<name>.config.*`. The gear name is
`analytics-api` (kebab, macro-enforced), so the natural env key was
`APP__gears__analytics-api__config__*` — but hyphenated env-var names
are silently dropped by the compose `sh`/dash entrypoint (dash discards
env names that aren't valid identifiers) and skipped by Kubernetes
`envFrom`. The result: every override (DB URL, tenant, clickhouse,
redis, identity) was stranded at its YAML default, so the gear tried to
reach `localhost:3306` and failed init with "pool timed out". (The old
service worked because `ANALYTICS__*` had no hyphen.)
Fix, keeping the kebab gear name:
- Rename every override key to the identifier-safe alias
`APP__gears__analytics_api__config__*` (compose env, umbrella Secret,
gitops compose-app-secrets.sh) — survives dash and k8s envFrom.
- The toolkit does no kebab/snake normalization, so bridge it in
main.rs: `fold_gear_env_alias` deep-merges the `analytics_api` gears
alias into `analytics-api` right after config load (both `run` and
`migrate` paths). Verified via `--print-config`: the underscore env
keys now land in `gears.analytics-api.config`.
Build, clippy (-D warnings), fmt, and tests (325) all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* analytics-api: don't register /health (host gear owns it)
With the toolkit host, the api-gateway system gear's `rest_prepare`
already registers `/health` and `/healthz`. Our gear also registered
`/health`, so route composition panicked at boot with "Overlapping
method route. Handler for `GET /health` already exists".
Drop the gear's `/health` route (and the now-unused `handlers::health`).
Also build the gear's endpoints on a fresh sub-router so the
tenant-override middleware + `AppState` extension scope to analytics-api's
own routes and don't wrap the host's `/health`/`/healthz`/`/openapi.json`.
DB now connects (the earlier env-key fix landed); this was the next boot
panic. Build, clippy (-D warnings), fmt, tests (325) green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* fix(gitops): drop backticks from compose-app-secrets heredoc comments
The analytics-api/identity config Secrets are emitted via an unquoted
`cat <<EOF` heredoc (unquoted so ${MDB_USER} etc. expand). Backticks in
the heredoc *body* comments were therefore run as commands, printing
`keep: command not found`, `APP__gears__analytics_api__config__*: command
not found`, `ANALYTICS__*: command not found` during `make deploy`. The
`keep` one was pre-existing; the APP__/ANALYTICS__ ones came in with the
env-key rename.
Remove the backticks from the heredoc comments (shell comments elsewhere
in the file are unaffected — bash doesn't substitute in `#` lines). Only
comment text was corrupted, not the Secret data, but the noise is gone
and the script is correct.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* fix(seed): mirror source/source_id in class_ai_dev_usage placeholder
Migration 20260623_ai_personal_gold_views (from #1514 ai-personal-gold)
selects and GROUP BYs c.source / c.source_id from silver.class_ai_dev_usage,
but the seed's placeholder DDL was never updated to add those columns.
On a fresh ClickHouse the view creation fails with UNKNOWN_IDENTIFIER
("Identifier 'c.source' cannot be resolved"), breaking `seed silver`.
Add source/source_id (String) to the class_ai_dev_usage placeholder,
matching the real silver model — same placeholder-mirror gap the file
already documents for the post-20260601 columns. Not gear-related;
surfaced while validating the analytics-api gear on a clean k8s cluster.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* analytics-api: address CodeRabbit review (PR #1546)
- config/insight.yaml: drop the committed `mysql://insight:insight@…`
default; database_url/clickhouse_url are empty and injected per-deploy
via APP__* env overrides (no credential in the checked-in runnable config).
- gear.rs: redact userinfo from redis_url before logging (redis:// URLs
can embed credentials); add `check_config` that validates the gear
config section + required URLs without touching any backend.
- main.rs: `check` subcommand now runs `gear::check_config` instead of a
no-op success.
- charts secrets.yaml: percent-encode (urlquery) the MariaDB user/password
in the analytics DSN, matching the identity DSN, so operator credentials
with :/@/%/ chars produce a valid connection string.
Deferred (replied on PR): NetworkPolicy/trust-boundary + auth fail-closed
(the deliberate header-trust posture — infra follow-up), --print-config
redaction (operator debug flag, parity with api-gateway), bash DSN
url-encoding (pre-existing, also affects identity — needs shared helper),
Secret-change reload (Secret is externally managed in gitops).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* test(e2e): spawn analytics-api as a gears host
The gear migration changed how analytics-api boots — it's now a
toolkit::bootstrap host that needs `-c <gears-config> run` and reads
`APP__gears__analytics_api__config__*` (not the old `ANALYTICS__*`). The
e2e harness still launched the bare binary with `ANALYTICS__*` env, so the
new binary came up with empty config and the suite failed.
Update lib/analytics_api.py to spawn the way the deployment does:
- pass `-c <repo>/services/analytics-api/config/insight.yaml run`
- inject leaf overrides via APP__* (direct Popen execve preserves the
hyphenated gear-name segments, unlike the compose sh entrypoint)
- override the REST-host bind on the api-gateway gear to loopback:port
- give grpc-hub a per-spawn UDS so parallel workers/re-runs don't collide
- widen the health-wait to 60s (host inits all system gears + migrations)
- /health is now host-served (public); refresh the stale tenant comments
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* test(analytics-api): unit-test the gear's unit-testable new code
Adds tests for the pure/new logic introduced by the gear migration and
refactors the route table to be testable without a DB:
- api/mod.rs: split the OperationBuilder chains into a stateless
`build_operations(router, openapi)`; `register_routes` now just layers
state/middleware over it and merges into the host router. A test drives
`build_operations` so the full 20-endpoint route table + OpenAPI
registration is exercised with no AppState/DB (0% → ~95%).
- main.rs: tests for `fold_gear_env_alias` + `deep_merge` (alias→canonical
fold, object merge, leaf overwrite).
- gear.rs: tests for `redact_url` (strips userinfo) and `check_config`
(missing section / empty required URLs).
- config.rs reaches 100% via the above.
Patch coverage for the crate rises ~40% → 60%. The remainder is
MariaDB/ClickHouse-bound boot + handler code (gear init, infra::db
connect, AdminThresholdService, REST handler bodies) that needs a live
DB — exercised by the e2e suite + the #[ignore]d live_tests, neither of
which runs in the unit-coverage job.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* ci(analytics-api): run DB-backed live_tests in the rust job (#1564)
Wire the existing `rust:` CI job to exercise analytics-api's `#[ignore]`d
live_tests against a real MariaDB — no new job, no ClickHouse (CH-gated
tests skip themselves).
CI (`.github/workflows/ci.yml`, `scripts/ci/*`):
- Add a `live_db` component flag (components.py) threaded into the matrix
entry (changed.py); default false, true only for analytics-api.
- For live_db entries the rust job starts a MariaDB 11.4 container, applies
migrations once up front via `analytics-api migrate` (a failed migration
fails the job), then runs the live_tests in parallel with
INTEGRATION_TESTS_MARIADB_URL set (`--include-ignored`). Both the migrate
and the tests run under llvm-cov, so the DB paths now count toward coverage
(patch coverage 60% -> 73%).
Parallel-safe live_tests (no DB reset):
- Migrations run once before the suite; tests never migrate or reset the DB.
- Isolation is per-tenant / per-metric-key, so the suite is parallel-safe:
admin_threshold + catalog + schema_validator each own unique keys/tenants.
- Drop the per-test `reset_catalog` + `Migrator::up` blocks and the helper.
- Fix an admin_threshold key collision: use the full UUID (`metric_id.simple()`)
instead of the v7 timestamp prefix `[..8]`, which collided in same-ms runs.
- Delete migration/live_tests.rs: obsolete now that the CI migrate step (plus
the startup CHECK probe) covers "migrations apply on an empty DB".
Identity resolution already uses Testcontainers MariaDB (MariaDbFixture) in
the dotnet job — no CI change needed there.
Verified: 8/8 fresh-DB parallel runs green (362 tests, ~3s); no-DB run skips
all 24 live_tests cleanly; fmt + clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* test(analytics-api): HTTP-level live tests to close the handler coverage gap (#1564)
The service-layer live_tests exercise the domain logic directly but never
drive the axum handlers, leaving the extract -> delegate -> Result->Response
glue at 0% (new-code patch coverage stuck at 73%, under the 80% gate).
Add `api/http_live_tests.rs`: builds a full AppState against the live DB and
mounts the real route table via `register_routes`, then drives it with
`tower::oneshot`. ClickHouse/Identity clients point at an unreachable address
on purpose — DB-backed handlers return real 2xx; the CH/Identity ones exercise
their entry + error-mapping path. Covers the admin-threshold CRUD round-trip
(201/200/204 status mapping), the metric/column/catalog reads, and the
unknown-field rejection envelope. Same `#[ignore]` + `INTEGRATION_TESTS_MARIADB_URL`
gating as the domain live_tests; a fresh tenant per write keeps it parallel-safe.
Result: `admin/handlers.rs` and `catalog.rs` 100%, `api/mod.rs` (register_routes)
100%, patch coverage 73% -> 84% (clears the gate).
Also fix a latent bug the reset-removal exposed:
`response_link_map_omits_metrics_dropped_by_walk_all` DELETEs the GLOBAL
`product-default` seed threshold for `ic_kpis.tasks_closed` and never restored
it — `reset_catalog` used to rebuild the seed each run and hid this. Left as-is
it corrupts the shared seed (every subsequent run/reader sees the metric
permanently dropped) and fails on re-run. Fix: capture the row (as CHAR via CAST
— sqlx-mysql can't decode DECIMAL->f64), delete, resolve, then RESTORE it before
any assertion can unwind. The `fetch_links` surfaced-ids filter already keeps
metrics + links consistent during the delete window, so concurrent readers
never observe a phantom. Verified: full suite 3x on one DB, no reset, seed
intact (good/warn preserved), 372 passed each; no-DB run skips all 34
live_tests; fmt + clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* test(analytics-api): isolate the walk-drop test via tenant scoping, not shared mutation
Replace the capture->delete->restore approach (which mutated the GLOBAL
`product-default` seed row for `ic_kpis.tasks_closed`) with a fully isolated,
purely-additive fixture that proves the same ADR-003 `surfaced_ids` invariant
through tenant scoping alone — the model's own rule: a DEFAULT metric
(`tenant_id IS NULL`) is visible to everyone, and per-tenant threshold data
never intersects across tenants.
The test now inserts a GLOBAL catalog row with a UNIQUE key (catalog is
global-only in v1), links it into `metric_query_catalog`, and gives it a
threshold for ONE tenant. Resolving as that tenant surfaces it + its link;
resolving as any other (never-seen) tenant has no candidate, so `walk_all`
drops it and the surfaced-ids filter drops its link. No shared row is read or
mutated, nothing is deleted or restored.
Verified: shared seed untouched (tasks_closed product-default stays intact),
full suite 3x on one DB with no reset (372 passed each), no-DB run skips all
34 live_tests, fmt + clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* ci: scope a component's coverage report to its own code (fix dependency spillover)
`cargo llvm-cov --package X` reports every instrumented file, including
path-dependency crates compiled into X's test binary. So the analytics-api
report also carried `libs/insight-clickhouse` at ~15% (only `Client::new` is
hit from analytics-api) — and the gate buckets files into components via
`component_for`, so those lines land on the insight-clickhouse component.
Today this is masked: the gate merges all reports with max-hits, and this
branch also changes insight-clickhouse (its own job reports ~93%), so the union
wins. But on a PR that touches analytics-api and NOT insight-clickhouse, the
latter's job never runs — the only insight-clickhouse data is analytics-api's
15% spillover, which would fail the component's OVERALL_MIN (80%) gate. No
amount of insight-clickhouse tests fixes that; the spillover must not be
counted here in the first place.
Add a per-component `cover_ignore_regex` (threaded through changed.py) wired to
`cargo llvm-cov report --ignore-filename-regex`. Set `src/backend/libs/` on
analytics-api (the only rust service that links a shared lib). Each library
crate keeps self-reporting in its own job. Verified: the analytics-api report
no longer lists any insight-clickhouse file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
* test(insight-clickhouse): cover Client + Config Debug + Error Display (no ClickHouse)
Raise the crate's own coverage without a live ClickHouse by testing the pure,
connection-free surface that was untested: `Client::new` auth branches,
`query()` with/without a timeout, the `inner()`/`config()` accessors, and
`tenant_query` validation (lib.rs 46% -> 100% line); the password-redacting
`Config` Debug impl (config.rs -> 100%); and `Error` Display (error.rs ->
100%). Crate line coverage 88% -> 94%.
The remaining gap is `QueryBuilder::fetch_all` — query EXECUTION, which needs a
live ClickHouse and is out of scope here (cf/insight#1564 skips CH).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
---------
Signed-off-by: Anton Zelenov <antonz@constructor.tech>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mozhaev-dev
added a commit
that referenced
this pull request
Jul 6, 2026
…#1527) PR A of the #1527 split — the data-layer foundation. Establishes the shared gold view the other collab modalities (#1528–#1532) extend, and delivers the Messaging modality's two counters. - New gold view `insight.collab_person_counter_daily` (migration 20260702000000): honest-NULL UNION ALL over the per-vendor silver chat classes (M365 Teams · Slack · Zulip), outer GROUP BY (person_id, metric_date) with the `if(countIf(x IS NOT NULL) > 0, sumIf …, NULL)` wrapper so a person with no source on a day is NULL, never a fake 0. LEFT JOIN insight.people for org_unit_id. Clone of #1514's ai_person_counter_daily shape. · messages_sent — Σ total_chat_messages across the three vendors. · channel_posts — Σ (channel_posts + channel_replies) across M365 · Slack for vendor comparability (Slack folds replies in already; M365 splits them); Zulip has no channel split → honest-NULL. - dbt singular test `assert_collab_messaging_bounds.sql`: counters ≥ 0. - Register the view in silver/_shared/gold_sources.yml. Designed for extension: later modalities add columns + UNION branches, not new vendor rows. Backend query_ref + catalog seed (PR B), e2e YAML (PR C) and FE (insight-front) follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Sergei Mozhaev <mozhaev.dev@gmail.com>
cyberantonz
pushed a commit
to cyberantonz/insight
that referenced
this pull request
Aug 5, 2026
…tructorfabric#1527) PR D of the constructorfabric#1527 split — the frontend surface for the collaboration Messaging counters (backend: gold view constructorfabric#1567 + query/seed). Mirrors the AI-adoption peer-counters panel (constructorfabric#1514). - metric-registry: add V2_IC_COLLAB_PEER_COUNTERS (…0053), the seeded collab Messaging peer-counter query. - ic-extras: add useIcCollabPeerCounters (clone of useIcAiPeerCounters, same row shape + per-org_unit bands, different metric id). - new CollabMessagingPanel: renders messages_sent + channel_posts vs the person's department under a "Messaging" card via the shared PeerStorySection. Fully catalog-driven — any counter row whose metric_key is in the catalog renders; unknown keys are dropped. - mount it in the collaboration section drilldown (section-drilldown-sheet). - catalog-snapshot mock: add the two collab_person_counter_daily.* metrics. - vitest: renders both under "Messaging"; drops unseeded keys; renders nothing without person/range. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Sergey Mozhaev <Sergey.Mozhaev@constructor.tech>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the improved AI personal analytics query set, catalog rows, and ClickHouse gold views.
Supports the new AI drilldown counters and charts with unified person-level AI activity data.
Summary by CodeRabbit