feat(flare-workflow): instance/step metrics query layer (workflow_metrics) - #509
Conversation
Add MetricsFilter/WorkflowMetrics/StepMetrics types, a workflow_metrics method on StateStore (SqliteStore via GROUP BY aggregation, InMemoryStore by iterating the map), and a thin WorkflowEngine::metrics passthrough. SqliteStore migration adds duration_ms/input_tokens/output_tokens columns to step_state so per-step aggregates don't require deserializing run JSON. Agentflare-Agent: claude-code Agentflare-Branch: task/128-flare-workflow-instance-step-metrics-que Agentflare-Item: 128
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 59 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
getappz
changed the title
Agentflare-Agent: claude-code Agentflare-Branch: task/128-flare-workflow-instance-step-metrics-que Agentflare-Item: 128
* feat(workflow): surface instance/step metrics via CLI and MCP The workflow_metrics query layer (PR #509) had no caller outside flare-workflow's own tests -- no CLI command, MCP action, or dashboard route ever called engine.metrics(). Debugging a failing pipeline meant hand-querying workflows.db directly. Add `agentflare workflow metrics` and the MCP `workflow` tool's `metrics` action, both wired to the existing engine.metrics()/ StateStore::workflow_metrics() layer, with --workflow-id/--status/ --since filters. Metrics alone only gives counts, not the actual error text, so also add a recent_failures list: the most recent failed runs with their failing step and its real last_error message. This queries workflow_runs/step_state directly by raw SQL rather than through StateStore::list_all::<D>() -- the store is shared by several WorkflowData types (PipelineData, WorkItemData, ...), and deserializing through a fixed D silently skips every run of a different type, same class of bug as the list_active/list_all fix. workflow_metrics's own SQL aggregation already takes this same type-agnostic approach for the same reason. Agentflare-Agent: claude-code_2-1-233_agent Agentflare-Branch: feat/workflow-metrics-surfacing * fix(workflow): status hard-fails for any run not typed PipelineData agentflare workflow status / mcp__flare__workflow(action="status") went through engine.get_status -> StateStore::load::<PipelineData>, which hard-fails for a run of any other WorkflowData type (e.g. the internal SDD work-item pipeline's WorkItemData) with "storage error: deserialize state: invalid type: map, expected unit struct PipelineData". Reproduced live against a real agentflare-work-item run while narrowing metrics down to a single job. Every field this view reports (run_id, workflow_id, status, current_step, step_states, input, output, error, variables) lives outside context.data, so read the raw state_json generically via serde_json::Value instead of through the typed store -- same reasoning as recent_failures_async's raw-SQL query in the prior commit. Agentflare-Agent: claude-code_2-1-233_agent Agentflare-Branch: feat/workflow-metrics-surfacing * fix(workflow): address CodeRabbit review findings on #517 - recent_failures_async's LIMIT bounded the joined run x failed-step rows, not distinct runs -- a run with several failed steps (fan-out) could crowd out more recent failed runs. Bound the run-level query first, then join to failed step states. - Its since filter checked updated_at; MetricsFilter's since means created_at (sqlite_store::metrics_where). Match that semantics. - Both raw-SQL helpers opened a bare Connection with no busy_timeout, unlike agentflare-db-kit::open_file, and ran synchronously on the calling async runtime. Add the same 5-second busy_timeout and move the blocking work into spawn_blocking. - Extend workflow_metrics's test with workflow_id/since filter cases (matching and non-matching) and invalid-input cases, exercising recent_failures_async's parameter binding order. Agentflare-Agent: claude-code_2-1-233_agent Agentflare-Branch: feat/workflow-metrics-surfacing
1 participant
Working tree is clean and the commit is in place.
Summary
The implementation from the prior (interrupted) run was already complete and correct against the design spec — it just hadn't been committed due to a
gitinvocation failure. I verified it against all 8 acceptance criteria, fixed one trivial clippy warning (20 + 40 + 0→20 + 40in the test), and committed:duration_ms/input_tokens/output_tokenscolumns tostep_state(5thM::up), andwrite_state's UPSERT now writes them.MetricsFilter,WorkflowMetrics,StepMetricsintypes.rs, plusHashderives onWorkflowStatus/StepStatusneeded forHashMapkeys.StateStoretrait: newworkflow_metrics(&self, filter: MetricsFilter) -> WorkflowResult<WorkflowMetrics>.SqliteStore: real SQL aggregation (GROUP BY status/step_id, statuswith a sharedWHEREbuilder for the filter) — no full-row JSON deserialization.InMemoryStore: mirrors the same signature via in-memory iteration.WorkflowEngine::metrics: thin passthrough to the store, no CLI/HTTP/GraphQL surface added.tests/metrics_test.rs(3 tests) covering mixed-status aggregation, step breakdown/averages,workflow_idscoping, and the empty-store case.Full crate test suite (70 tests across all test files) and
cargo clippy --all-targetsboth pass clean. Confirmed no other crate implementsStateStore(only imports the trait), so the new trait method doesn't break anything downstream.