fix: remove defaultFilterDataLimit cap from filter data queries

[impoiler]

Summary

Removes the defaultFilterDataLimit (500) cap that was previously applied to all GetDistinct* and GetAvailable* filter-data queries. This ensures that filter dropdowns and autocomplete fields return the full set of distinct values rather than silently truncating results at 500 entries.

Changes

• Removed the defaultFilterDataLimit = 500 constant and all .Limit(defaultFilterDataLimit) calls from both the materialized view query paths (matviews.go) and the direct RDB query paths (rdb.go).
• Affected queries: distinct models, aliases, stop reasons, routing engines, key pairs, tool names, server labels, and MCP virtual keys.
• The existing defaultFilterDataCutoffDays = 30 time-based filter remains in place to keep queries scoped to recent data.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Verify that filter endpoints return more than 500 distinct values when the dataset contains them, and that no results are truncated.

go test ./framework/logstore/...

Seed a test database with more than 500 distinct models (or aliases, etc.) within the 30-day window and confirm all values are returned by the corresponding GetDistinct* method.

Breaking changes

• Yes
• No

Related issues

Security considerations

Removing the result cap means larger payloads may be returned by filter-data endpoints. Ensure that the 30-day cutoff and existing authentication/authorization controls are sufficient to prevent abuse or excessive memory usage in environments with very high cardinality data.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 93fa9ae6-51a7-47f3-90b1-c4a9c34d98d3

📥 Commits

Reviewing files that changed from the base of the PR and between 5b2de1c and d7383b9.

📒 Files selected for processing (2)

• framework/logstore/matviews.go
• framework/logstore/rdb.go

🚧 Files skipped from review as they are similar to previous changes (2)

• framework/logstore/matviews.go
• framework/logstore/rdb.go

---

📝 Walkthrough

Summary by CodeRabbit

• Bug Fixes
  • Filter endpoints for models, aliases, stop reasons, key/value pairs, routing engines, available tools, server labels, and virtual keys now return the complete set of distinct available values instead of being capped, so filters present all eligible options for more comprehensive selection.

Walkthrough

This PR removes the defaultFilterDataLimit constant and eliminates the .Limit() clause from eleven distinct value query methods across two files. The queries now return the full distinct set for models, aliases, stop reasons, key pairs, routing engines, tool names, server labels, and MCP virtual keys while preserving existing time cutoff windows and WHERE predicates.

Changes

Remove distinct value limits

Layer / File(s)	Summary
Constant definition removal framework/logstore/rdb.go	The defaultFilterDataLimit constant is removed from the constants block.
Matview-backed distinct queries framework/logstore/matviews.go	Five matview query methods (getDistinctModelsFromMatView, getDistinctAliasesFromMatView, getDistinctStopReasonsFromMatView, getDistinctKeyPairsFromMatView, getDistinctRoutingEnginesFromMatView) no longer apply the limit clause, returning the full distinct set from materialized views.
RDB-backed distinct queries framework/logstore/rdb.go	Eight RDB query methods (GetDistinctModels, GetDistinctAliases, GetDistinctKeyPairs, GetDistinctRoutingEngines, GetDistinctStopReasons, GetAvailableToolNames, GetAvailableServerLabels, GetAvailableMCPVirtualKeys) remove the limit clause and return all distinct values within cutoff windows.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

A rabbit hops through limits old,
Removing caps on data bold,
Now filters fetch their complete tale—
No more truncation, none will fail! 🐰✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: removing the defaultFilterDataLimit cap from filter data queries, which aligns with the changeset modifications to both matviews.go and rdb.go.
Description check	✅ Passed	The pull request description is comprehensive, following the template structure with all major sections completed: summary, changes, type of change, affected areas, testing instructions, breaking changes, and security considerations are all present and well-documented.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 05-18-fix_remove_the_500_distinct_values_cap_from_the_matviews_filter_queries_to_avoid_confusion_when_data_is_missing

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[impoiler]

• feat: add search icon and fetching spinner to filter sidebar search inputs #3563
• feat: add server-side search to filter sidebar checkbox lists via debounced q query param #3568
• feat: add limit and query params to filter data endpoints for server-side search and pagination #3567
• fix: remove defaultFilterDataLimit cap from filter data queries #3561 👈 (View in Graphite)
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[greptile-apps]

Confidence Score: 5/5

Safe to merge; the change is a straightforward removal of a hard cap with no logic regressions on the critical query paths.

The removal of the 500-row limit is mechanical and well-scoped. The only behavioral gap introduced is a sort-order inconsistency on the key-pairs path between the matview and raw-table routes, which affects presentation order in the UI but not correctness of the data returned.

framework/logstore/rdb.go — the raw-table fallback for GetDistinctKeyPairs is missing the Order clause added to its matview counterpart.

Important Files Changed

Filename	Overview
framework/logstore/rdb.go	Removes .Limit(500) from all GetDistinct*/GetAvailable* query paths; the raw-table fallback for GetDistinctKeyPairs is missing the Order("name ASC") that was added to its matview counterpart in the same PR.
framework/logstore/matviews.go	Removes .Limit(500) from matview filter-data queries and adds Order("name ASC") to getDistinctKeyPairsFromMatView; one stale doc comment about the removed limit still references the old cap.

_{Reviews (2): Last reviewed commit: "fix: remove the 500 distinct values cap ..." | Re-trigger Greptile}

[akshaydeo]

Merge activity

• May 18, 1:40 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• May 18, 1:40 PM UTC: @akshaydeo merged this pull request with Graphite.