fix: stats for cancelled requests

[TejasGhatte]

Summary

Adds cancelled as a first-class terminal log status alongside success and error. Previously, cancelled requests were excluded from all aggregate queries, histograms, and materialized views. This change ensures cancelled requests are tracked, counted, and surfaced in charts and exports.

Changes

• Introduced a terminalLogStatuses constant (["success", "error", "cancelled"]) to replace all scattered inline []string{"success", "error"} slices across query filters in rdb.go and matviews.go.
• Added cancelled_count to the mv_logs_hourly materialized view definition and included cancelled in the WHERE status IN (...) clause so the view captures cancelled requests.
• Added cancelled_count to mvLogsHourlyRequiredColumns to enforce schema compatibility checks on startup.
• Added canUseMatViewStatusFilter to gate matview usage: non-terminal statuses (e.g. processing) force the raw query path, while terminal statuses (including cancelled) remain matview-eligible.
• Extended HistogramBucket and ModelUsageStats structs with a Cancelled field, and propagated it through all histogram query paths (raw DB and matview) for GetHistogram, GetModelHistogram, and their matview equivalents.
• Updated CSV export helpers (overviewVolumeToCSV, overviewModelUsageToCSV) to include cancelled counts.
• Updated UI chart components (logVolumeChart.tsx, modelUsageChart.tsx, logsVolumeChart.tsx, overviewTab.tsx) to render a stacked cancelled segment using zinc-400 (#a1a1aa), display it in tooltips and legends, and handle missing values with a ?? 0 fallback.
• Added cancelled to the CHART_COLORS palette and updated the HistogramBucket, ModelUsageStats, and MCPHistogramBucket TypeScript interfaces.
• Added TestCancelledStatusIncludedInLogAggregates to verify that cancelled rows appear in SearchLogs, GetStats, and GetHistogram results while processing rows are excluded from terminal aggregates.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

# Core/Transports
go test ./framework/logstore/...

# UI
cd ui
pnpm i
pnpm build

The new test TestCancelledStatusIncludedInLogAggregates covers the following scenarios:

• A cancelled log appears in SearchLogs when filtering by status=cancelled.
• GetStats with no filter counts all 4 rows as total requests but only 3 (terminal) as the cache-hit denominator.
• GetStats filtered to cancelled returns correct latency and a zero success rate.
• GetHistogram filtered to cancelled returns a bucket with Count=1, Success=0, Error=0, Cancelled=1.

Because mv_logs_hourly gains a new cancelled_count column, the materialized view must be recreated on first startup. The existing schema version/migration logic handles this via mvLogsHourlyRequiredColumns.

Breaking changes

• Yes
• No

The mv_logs_hourly materialized view schema changes (new cancelled_count column). On startup the store will detect the missing column and recreate the view. No manual migration is required, but there will be a brief period during view recreation where matview-backed queries fall back to raw table scans.

API consumers reading HistogramBucket or ModelUsageStats JSON will now receive an additional cancelled field. This is additive and backwards-compatible for consumers that ignore unknown fields.

Related issues

Security considerations

None. No new auth surfaces, secrets, or PII handling introduced.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds cancelled as a tracked terminal log status across matview and non-matview analytics, response types, tests, and dashboard/logs charts. Backend queries and matview aggregation now include cancelled counts, and the UI surfaces the new metric in charts, legends, tooltips, and CSV exports.

Changes

Cancelled status analytics and UI

Layer / File(s)	Summary
Matview schema and eligibility framework/logstore/matviews.go, framework/logstore/multi_team_filter_test.go	mv_logs_hourly adds cancelled_count and includes cancelled rows; required columns are updated; matview filter eligibility now uses terminal-status helpers and rejects processing status.
Matview-based aggregation framework/logstore/matviews.go	getStatsFromMatView, getHistogramFromMatView, and getModelHistogramFromMatView use terminal statuses and aggregate cancelled_count into histogram and model results.
Non-matview analytics queries framework/logstore/rdb.go	terminalLogStatuses is introduced and applied across stats, histograms, rankings, and provider/dimension aggregates; histogram and model results carry cancelled counts.
Response structs and aggregate test framework/logstore/tables.go, framework/logstore/rdb_perf_test.go	HistogramBucket and ModelUsageStats export Cancelled, and a log aggregate test verifies cancelled rows in stats, cache-hit denominators, and histograms.
Frontend type definitions ui/lib/types/logs.ts	LogEntry.status docs expand to include processing/cancelled, and HistogramBucket/ModelUsageStats add cancelled.
Dashboard charts and exports ui/app/workspace/dashboard/components/charts/logVolumeChart.tsx, .../modelUsageChart.tsx, .../overviewTab.tsx, .../utils/chartUtils.ts, .../utils/exportUtils.ts	Cancelled color, chart series, tooltip rows, legends, and CSV columns are added for volume and model usage views.
Logs page volume chart ui/app/workspace/logs/views/logsVolumeChart.tsx	Tooltip, bucket-filling logic, legend, and bar series are updated to carry and render cancelled counts.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

• maximhq/bifrost#3766: This PR’s terminal-status query changes directly overlap with the ranking and matview filter logic adjusted here.
• maximhq/bifrost#4831: This PR’s analytics changes depend on cancelled logs being persisted with status="cancelled".

Suggested reviewers: akshaydeo

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and directly summarizes the PR’s main change: adding cancelled-request stats.
Description check	✅ Passed	The PR description closely follows the template and includes the key sections for summary, changes, testing, breaking changes, and security.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 07-05-fix_stats_for_cancelled_requests

---

_{Comment @coderabbitai help to get the list of available commands.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

tejas ghatte seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[TejasGhatte]

• fix: stats for cancelled requests #4930 👈 (View in Graphite)
• fix: tier costs evaluated via input tokens instead of total tokens #4917
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[greptile-apps]

Confidence Score: 5/5

Safe to merge — the changes are additive, internally consistent, and well-tested on the raw-query path.

The cancelled promotion is correct end-to-end: the matview DDL groups by status (making WHERE status IN ('cancelled') valid on the view), the canUseMatViewStatusFilter gate correctly routes processing to raw scans, and every histogram/stats query path is updated symmetrically. No query correctness issues were found.

No files require special attention. The matview rebuild on startup is expected and documented.

Important Files Changed

Filename	Overview
framework/logstore/matviews.go	Adds cancelled_count to mv_logs_hourly DDL and updates histogram matview paths; adds canUseMatViewStatusFilter gate.
framework/logstore/rdb.go	Introduces terminalLogStatuses constant replacing all inline slices; extends all histogram and ranking queries with cancelled_count.
framework/logstore/tables.go	Adds Cancelled int64 field to HistogramBucket and ModelUsageStats Go structs.
framework/logstore/rdb_perf_test.go	New TestCancelledStatusIncludedInLogAggregates covers SearchLogs, GetStats, and GetHistogram for cancelled status on SQLite.
ui/app/workspace/logs/views/logsVolumeChart.tsx	Adds cancelled Bar segment and legend; hardcodes fill color instead of CHART_COLORS.cancelled (noted in prior review).

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[GetHistogram / GetStats / GetModelHistogram] --> B{canUseMatViewFilters?}
    B -- "ContentSearch/Metadata present" --> RAW[Raw table scan]
    B -- "All filters matview-compatible?" --> C{canUseMatViewStatusFilter}
    C -- "non-terminal status e.g. processing" --> RAW
    C -- "empty or all terminal statuses" --> MV[mv_logs_hourly path]
    MV --> D[applyMatViewFilters WHERE status IN ...]
    D --> E[SUM count + success_count + error_count + cancelled_count]
    RAW --> F[WHERE status IN terminalLogStatuses + user filter]
    F --> G[COUNT star + SUM CASE per status]
    E --> OUT[HistogramBucket .Count .Success .Error .Cancelled]
    G --> OUT

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[GetHistogram / GetStats / GetModelHistogram] --> B{canUseMatViewFilters?}
    B -- "ContentSearch/Metadata present" --> RAW[Raw table scan]
    B -- "All filters matview-compatible?" --> C{canUseMatViewStatusFilter}
    C -- "non-terminal status e.g. processing" --> RAW
    C -- "empty or all terminal statuses" --> MV[mv_logs_hourly path]
    MV --> D[applyMatViewFilters WHERE status IN ...]
    D --> E[SUM count + success_count + error_count + cancelled_count]
    RAW --> F[WHERE status IN terminalLogStatuses + user filter]
    F --> G[COUNT star + SUM CASE per status]
    E --> OUT[HistogramBucket .Count .Success .Error .Cancelled]
    G --> OUT

Loading

_{Reviews (2): Last reviewed commit: "fix: stats for cancelled requests" | Re-trigger Greptile}

[coderabbitai]

🧹 Nitpick comments (1)

ui/app/workspace/logs/views/logsVolumeChart.tsx (1)

161-167: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Color duplicated as a magic string instead of reusing the shared constant.

fill="#a1a1aa" / bg-zinc-400 duplicate CHART_COLORS.cancelled from chartUtils.ts. This mirrors the pre-existing pattern in this file (success/error are also hardcoded here rather than using CHART_COLORS), so it's not a new inconsistency introduced by this PR, but consolidating on the shared constant would prevent future drift if the cancelled color is updated in one place but not the other.

Also applies to: 225-292, 389-392, 460-469

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ui/app/workspace/logs/views/logsVolumeChart.tsx` around lines 161 - 167, The
cancelled chart color is hardcoded in logsVolumeChart instead of reusing the
shared CHART_COLORS.cancelled constant, which can drift from chartUtils.ts.
Update the cancelled indicator/legend and any related cancelled series rendering
in logsVolumeChart to reference CHART_COLORS.cancelled rather than inline color
values, following the existing chart color usage patterns in this file.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@ui/app/workspace/logs/views/logsVolumeChart.tsx`:
- Around line 161-167: The cancelled chart color is hardcoded in logsVolumeChart
instead of reusing the shared CHART_COLORS.cancelled constant, which can drift
from chartUtils.ts. Update the cancelled indicator/legend and any related
cancelled series rendering in logsVolumeChart to reference
CHART_COLORS.cancelled rather than inline color values, following the existing
chart color usage patterns in this file.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 1705d35f-7eeb-4526-ae8b-1a3d821527dd

📥 Commits

Reviewing files that changed from the base of the PR and between 4c98869 and aa77187.

📒 Files selected for processing (12)

• framework/logstore/matviews.go
• framework/logstore/multi_team_filter_test.go
• framework/logstore/rdb.go
• framework/logstore/rdb_perf_test.go
• framework/logstore/tables.go
• ui/app/workspace/dashboard/components/charts/logVolumeChart.tsx
• ui/app/workspace/dashboard/components/charts/modelUsageChart.tsx
• ui/app/workspace/dashboard/components/overviewTab.tsx
• ui/app/workspace/dashboard/utils/chartUtils.ts
• ui/app/workspace/dashboard/utils/exportUtils.ts
• ui/app/workspace/logs/views/logsVolumeChart.tsx
• ui/lib/types/logs.ts

[akshaydeo]

Merge activity

• Jul 5, 5:35 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jul 5, 5:36 PM UTC: @akshaydeo merged this pull request with Graphite.

The base branch was changed.

[Pratham-Mishra04]

Arent text n image handled differently?

[TejasGhatte]

but the input tokens for tier would be combination of all input tokens right