Skip to content

feat: add alias column to materialized view and filter support - #4078

Merged
akshaydeo merged 18 commits into
devfrom
06-05-fix_bug_add_alias_filter_support_for_matview_queries
Jun 5, 2026
Merged

feat: add alias column to materialized view and filter support#4078
akshaydeo merged 18 commits into
devfrom
06-05-fix_bug_add_alias_filter_support_for_matview_queries

Conversation

@impoiler

@impoiler impoiler commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds alias as a dimension to the mv_logs_hourly materialized view, enabling filtering and grouping of log analytics by alias.

Changes

  • Added alias to the SELECT clause of the materialized view query using COALESCE(alias, '') AS alias
  • Updated the GROUP BY clause to include the new alias column
  • Added alias to the unique index mv_logs_hourly_uniq to maintain correctness for concurrent refreshes
  • Added alias to mvLogsHourlyRequiredColumns so schema validation and repair paths recognize it as a required column
  • Added filtering support for alias in applyMatViewFiltersOnly using the Aliases field from SearchFilters

Type of change

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Chore/CI

Affected areas

  • Core (Go)
  • Transports (HTTP)
  • Providers/Integrations
  • Plugins
  • UI (React)
  • Docs

How to test

go test ./...

After deploying, verify that:

  1. The materialized view is refreshed or recreated with the alias column present
  2. Querying log analytics with an alias filter returns correctly scoped results
  3. The unique index on mv_logs_hourly includes alias without conflicts

Breaking changes

  • Yes
  • No

The unique index definition has changed. The existing materialized view and its index will need to be dropped and recreated (or handled via the startup ensure/repair path) to reflect the new schema. Existing deployments should verify that the repair path correctly detects and applies the updated column set.

Related issues

#4071

Security considerations

No security implications. alias is an existing log attribute being surfaced as an analytics dimension.

Checklist

  • I read docs/contributing/README.md and followed the guidelines
  • I added/updated tests where appropriate
  • I updated documentation where needed
  • I verified builds succeed (Go and UI)
  • I verified the CI pipeline passes locally if applicable

Summary by CodeRabbit

  • New Features
    • Logs hourly aggregation now includes alias as a visible dimension for hourly summaries.
    • Added support for filtering/searching logs by alias, enabling more precise and efficient log queries.

@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@impoiler, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 10 minutes and 30 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: efbe867a-3dad-4ca9-8fe4-5a78e246566e

📥 Commits

Reviewing files that changed from the base of the PR and between c829950 and 107ef5b.

📒 Files selected for processing (1)
  • framework/logstore/matviews.go
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 06-05-fix_bug_add_alias_filter_support_for_matview_queries

Comment @coderabbitai help to get the list of available commands and usage tips.

impoiler commented Jun 5, 2026

Copy link
Copy Markdown
Contributor Author

@impoiler impoiler self-assigned this Jun 5, 2026
@roroghost17
roroghost17 marked this pull request as ready for review June 5, 2026 09:50
@greptile-apps

greptile-apps Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Confidence Score: 5/5

Safe to merge; the change is additive, follows the established scalar-column pattern, and the startup repair path correctly handles existing deployments via mvLogsHourlyRequiredColumns.

All five touched locations (DDL SELECT, GROUP BY, unique index, required-columns list, filter helper) are internally consistent. The alias column uses COALESCE(alias, '') matching every other nullable scalar column in the view. The repair path will detect any existing view missing the new column and drop-recreate it. No logic errors, data-correctness issues, or concurrency problems were found.

No files require special attention.

Important Files Changed

Filename Overview
framework/logstore/matviews.go Adds alias as a GROUP BY dimension to mv_logs_hourly, its unique index, required-columns list, and applyMatViewFiltersOnly; follows the established scalar-column pattern used by user_id/virtual_key_id; repair path correctly picks up the new column via mvLogsHourlyRequiredColumns.

Reviews (5): Last reviewed commit: "fix: [BUG] add alias filter support for ..." | Re-trigger Greptile

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
framework/logstore/matviews.go (1)

69-83: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Avoid rebuilding mv_logs_hourly in place during a rolling deploy.

Adding alias to mvLogsHourlyRequiredColumns forces repairMatViewShapes to DROP MATERIALIZED VIEW ... CASCADE on every upgrade from the old shape. Existing replicas can still have matViewsReady=true and keep routing eligible reads to mv_logs_hourly, so they can hit relation "mv_logs_hourly" does not exist between the drop and recreate/index phase. This needs a staged migration path or a temporary cluster-wide fallback to raw queries instead of an in-place repair.

As per coding guidelines, framework/** changes should preserve backward-compatible data formats and check migration behavior carefully.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@framework/logstore/matviews.go` around lines 69 - 83, The change to
mvLogsHourlyRequiredColumns (adding "alias") causes repairMatViewShapes to DROP
and recreate mv_logs_hourly in-place, which breaks rolling deploys; instead,
modify the migration approach: avoid dropping mv_logs_hourly inside
repairMatViewShapes when only adding a nullable/optional column—either implement
a staged migration that creates a new materialized view name (e.g.,
mv_logs_hourly_v2), backfills it, then atomically swap readers, or have
repairMatViewShapes detect incompatible shapes and set matViewsReady=false so
code routes reads to raw queries until all replicas are upgraded. Update
mvLogsHourlyRequiredColumns and repairMatViewShapes to follow this staged path
(or fall back to raw queries) rather than performing DROP ... CASCADE on
mv_logs_hourly during rolling upgrades.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@framework/logstore/matviews.go`:
- Around line 69-83: The change to mvLogsHourlyRequiredColumns (adding "alias")
causes repairMatViewShapes to DROP and recreate mv_logs_hourly in-place, which
breaks rolling deploys; instead, modify the migration approach: avoid dropping
mv_logs_hourly inside repairMatViewShapes when only adding a nullable/optional
column—either implement a staged migration that creates a new materialized view
name (e.g., mv_logs_hourly_v2), backfills it, then atomically swap readers, or
have repairMatViewShapes detect incompatible shapes and set matViewsReady=false
so code routes reads to raw queries until all replicas are upgraded. Update
mvLogsHourlyRequiredColumns and repairMatViewShapes to follow this staged path
(or fall back to raw queries) rather than performing DROP ... CASCADE on
mv_logs_hourly during rolling upgrades.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 72243e0e-f1de-46b3-887f-7ec70896c7f1

📥 Commits

Reviewing files that changed from the base of the PR and between e9ef1b2 and 5816c9f.

📒 Files selected for processing (1)
  • framework/logstore/matviews.go

@impoiler
impoiler force-pushed the 06-05-fix_bug_add_alias_filter_support_for_matview_queries branch from 5816c9f to c829950 Compare June 5, 2026 12:51
@impoiler
impoiler force-pushed the 06-05-feat_allow_viewing_filtered_logs_for_governance_entities branch from e9ef1b2 to 5138e35 Compare June 5, 2026 12:51
@impoiler
impoiler force-pushed the 06-05-feat_allow_viewing_filtered_logs_for_governance_entities branch from 5138e35 to 97fbee9 Compare June 5, 2026 13:48
@impoiler
impoiler force-pushed the 06-05-fix_bug_add_alias_filter_support_for_matview_queries branch from c829950 to 9be784d Compare June 5, 2026 13:48
@impoiler
impoiler force-pushed the 06-05-fix_bug_add_alias_filter_support_for_matview_queries branch from 9be784d to 513f10e Compare June 5, 2026 13:51
@impoiler
impoiler force-pushed the 06-05-feat_allow_viewing_filtered_logs_for_governance_entities branch from 97fbee9 to 5c1a600 Compare June 5, 2026 13:51
@impoiler
impoiler force-pushed the 06-05-feat_allow_viewing_filtered_logs_for_governance_entities branch from 5c1a600 to 186956b Compare June 5, 2026 13:53
@impoiler
impoiler force-pushed the 06-05-fix_bug_add_alias_filter_support_for_matview_queries branch from 513f10e to 107ef5b Compare June 5, 2026 13:53

akshaydeo commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Merge activity

  • Jun 5, 2:00 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Jun 5, 2:19 PM UTC: @akshaydeo merged this pull request with Graphite.

@akshaydeo
akshaydeo changed the base branch from 06-05-feat_allow_viewing_filtered_logs_for_governance_entities to graphite-base/4078 June 5, 2026 14:16
@akshaydeo
akshaydeo changed the base branch from graphite-base/4078 to dev June 5, 2026 14:19
@akshaydeo
akshaydeo merged commit c3610b3 into dev Jun 5, 2026
11 checks passed
@akshaydeo
akshaydeo deleted the 06-05-fix_bug_add_alias_filter_support_for_matview_queries branch June 5, 2026 14:19
@akshaydeo akshaydeo mentioned this pull request Jun 7, 2026
akshaydeo pushed a commit that referenced this pull request Jun 7, 2026
## Summary

Adds `alias` as a dimension to the `mv_logs_hourly` materialized view, enabling filtering and grouping of log analytics by alias.

## Changes

- Added `alias` to the `SELECT` clause of the materialized view query using `COALESCE(alias, '') AS alias`
- Updated the `GROUP BY` clause to include the new `alias` column
- Added `alias` to the unique index `mv_logs_hourly_uniq` to maintain correctness for concurrent refreshes
- Added `alias` to `mvLogsHourlyRequiredColumns` so schema validation and repair paths recognize it as a required column
- Added filtering support for `alias` in `applyMatViewFiltersOnly` using the `Aliases` field from `SearchFilters`

## Type of change

- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./...
```

After deploying, verify that:

1. The materialized view is refreshed or recreated with the `alias` column present
2. Querying log analytics with an alias filter returns correctly scoped results
3. The unique index on `mv_logs_hourly` includes `alias` without conflicts

## Breaking changes

- [x] Yes
- [ ] No

The unique index definition has changed. The existing materialized view and its index will need to be dropped and recreated (or handled via the startup ensure/repair path) to reflect the new schema. Existing deployments should verify that the repair path correctly detects and applies the updated column set.

## Related issues  
#4071

## Security considerations

No security implications. `alias` is an existing log attribute being surfaced as an analytics dimension.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

* **New Features**
  * Logs hourly aggregation now includes alias as a visible dimension for hourly summaries.
  * Added support for filtering/searching logs by alias, enabling more precise and efficient log queries.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo added a commit that referenced this pull request Jun 7, 2026
## ✨ Features

- **OpenAI Compaction** — Added OpenAI conversation compaction support
across core, framework, logging, and the API surface (#4053)
- **Multi-Customer & Org Hierarchy** — Logs and usage tracking now
support multiple customers, teams, and business units, including
business unit CRUD, team assignment, and governance endpoints in the
OpenAPI spec (#4066, #4041, #4082)
- **Provider-Level Governance** — Budgets & limits are now scope-aware
and can be applied at the virtual-key top level and per provider, wired
from the model configs table, with UI filters for scope and providers
(#3938, #3937, #3939, #3981, #3962)
- **Customer Budgets** — Customers support multiple budgets and
`calendar_aligned` budget windows (#3998, #3997)
- **Virtual Key Attribution & Controls** — Added a `created_by` user
attribution column and a `blacklisted_models` column for virtual key
provider configs (#3672, #3653)
- **Request Header Capture** — OTel and Maxim observability plugins
capture `request_headers` by pattern, with wildcard support (e.g.
`x-custom-*`); logging gained the same wildcard header capture (#4012,
#3958)
- **OTel Content Controls & Collectors** — New `disable_content_logging`
option drops message/tool content from exported spans, plus support for
multiple OTel collectors (#4064, #3894)
- **xAI x_search** — Added xAI `x_search` tool support (#3976)
- **URL Validation** — Added fetch URL validation with private-network
configuration and link-local blocking (#3947, #3991)
- **File Scheme Pricing URLs** — Pricing source URLs now accept the
`file://` scheme for air-gapped and self-hosted deployments (#4045)
- **Paginated Virtual Keys** — Virtual key fetching is paginated to
handle deployments with very large numbers of keys (#3957)
- **Client IP Resolution** — Resolve client IP from
`X-Forwarded-For`/`X-Real-IP` headers
- **SCIM Provisioning** — Added `attributeType`/`attributeValue` SCIM
provisioning fields
- **Helm/Config Schema** — Added `roles` RBAC governance config and
`per_user_oauth` MCP auth to the Helm chart and config schema (#4004,
#4009)
- **Log Navigation UI** — Added a "View logs" menu item to customer,
team, and virtual key tables, clickable links in log detail views, a
customer detail sheet, and a reusable `BudgetDisplay` component (#4073,
#4054, #4026, #4055)
- **Faster First Paint** — Added an inline loading shell to `#root`
before React mounts (#4063)
- **Materialized View Alias** — Added an `alias` column to the
materialized view with filter support (#4078)

## 🐞 Fixed

- **Fetch URL IP Checks** — Hardened fetch URL IP checks against SSRF
(#4092)
- **Mantle Model Matching** — Broadened Mantle model matching to all
`gpt` variants (#4091)
- **Empty Thinking Blocks** — Strip thinking blocks when the signature
is empty (#4079)
- **OpenAI Stream Usage** — Removed usage from the `responses.created`
event in the OpenAI stream (#4080)
- **Prompt Cache Key** — Set the prompt cache key from the Anthropic
integration (#4086)
- **Upstream Failure Status** — Map upstream connection failures to 502
instead of 400 (#3929) (thanks
[@chris-colinsky](https://github.com/chris-colinsky)!)
- **Gemini Schema Constraints** — Accept numeric schema integer
constraints for Gemini (#3994) (thanks
[@yanhao98](https://github.com/yanhao98)!)
- **Files Provider Param** — Accept the `?provider=` query param on `GET
/v1/files` (#3971) (thanks [@alexef](https://github.com/alexef)!)
- **Optional Batch Model** — Made the `model` field optional on `POST
/v1/batches` (#3973) (thanks [@alexef](https://github.com/alexef)!)
- **Helm Azure Config** — Added missing `azure_key_config` fields to the
Helm schema (#3996) (thanks
[@axelray-dev](https://github.com/axelray-dev)!)
- **Text Completion Chunk Model** — Added the missing `Model` field to
`TextCompletionChunkResponse` (#3970) (thanks
[@kuishou68](https://github.com/kuishou68)!)
- **MCP Inline stdio Env** — MCP stdio server configs accept inline
environment variable assignments (#3861) (thanks
[@Shushmitaaaa](https://github.com/Shushmitaaaa)!)
- **Orphaned Tool Results** — Orphaned tool results in the OpenAI to
Anthropic conversion flow are no longer rejected by the Anthropic API
(#3919)
- **Node Usage Reconciliation** — Added a monotonic `inc_number` log
cursor so node usage reconciliation does not skip late async log writes
(#3664)
- **Bedrock Output Assessments** — Corrected the type of
`outputAssessments` in Bedrock responses (#4028)
- **Model Pool Pricing Reloads** — Preserve non-pricing model pool
entries across pricing reloads (#3999)
- **Ghost Node Reconciliation** — Replicate the VK hierarchy flow for
ghost node reconciliation (#4088)
- **VK Double Usage Counting** — Fixed double usage counting when
creating a virtual key (#4070)
- **Model Config Lifecycle** — Cascade deletes for model configs and
removal of stale in-memory model configs (#4051, #4043)
- **FTS Index Cap** — Reduced the FTS index `left()` cap from 800k to
250k chars to stay within the tsvector limit (#4057)
- **Sync Worker Drift** — Reduced the sync worker ticker period to 5m to
prevent threshold drift (#4023)
- **Passthrough** — Fixed passthrough budgets, gated passthrough models
per VK, model extraction for Azure passthrough, and restricted
fallbacks/provider selection to the VK boundary (#3941, #3988, #3983,
#3924)
- **Provider Response Headers** — Strip provider response headers and
add a content-type filter (#3955, #4024)
- **Stream Handling** — Drain non-SSE stream readers and retry stale
connections (#3956, #3967)
- **Azure Claude** — Strip Azure diagnostic property for Claude models
(#3925)
- **Compat max_tokens** — Preserve chat `max_tokens` during param
filtering (#3992)
- **Raw Request Flag** — Removed the raw request flag from providers
that don't support it (#4058)
- **UI Fixes** — Standardized page container layout, virtual key model
configs UI, and dashboard chart tooltips (#4046, #4052, #4044)

## 🔧 Maintenance

- **Dependency Upgrades** — Bumped transitive `golang.org/x`
dependencies (crypto, net, sys, text) for Docker Scout CVE remediation
and `recharts` to 3.8.1; cascaded version bumps across all modules
(#3900, #4003)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants