feat(layer-4): Prometheus metrics + provisioned Grafana dashboard - #4
Merged
Conversation
The proxy now exposes GET /covenant/metrics (text exposition format):
- covenant_calls_total{tool,outcome} - ok / error / blocked
- covenant_call_latency_seconds{tool} - forwarded calls only
- covenant_drift_total{severity} - per breaking tool on refresh
- covenant_quarantined_tools - gauge, synced on refresh and in-band detection
One CollectorRegistry per app (the global registry would collide across
create_app calls in tests). prometheus-client rides the [proxy] extra.
Metric writes are in-process and non-throwing - never on the store path.
docker compose up -d prometheus grafana provisions a dashboard: calls by
outcome, p95 latency, quarantine stat (green 0 / red >=1), drift events.
OTel spans deferred (heavy dep tree, no trace consumer yet) - stated in the
Layer 4 spec.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (12)
📝 WalkthroughWalkthroughThis PR adds Layer 4 observability to Covenant via a Prometheus ChangesLayer 4 Observability
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant ProxyServer
participant Metrics
participant Upstream
Client->>ProxyServer: POST /mcp tools/call
ProxyServer->>ProxyServer: check quarantine
alt tool quarantined
ProxyServer->>Metrics: record_call(tool, "blocked")
ProxyServer-->>Client: error response
else tool allowed
ProxyServer->>Upstream: forward call
Upstream-->>ProxyServer: JSON-RPC response
ProxyServer->>Metrics: record_call(tool, outcome, latency_s)
ProxyServer-->>Client: response
end
Client->>ProxyServer: GET /covenant/metrics
ProxyServer->>Metrics: render()
Metrics-->>ProxyServer: Prometheus text
ProxyServer-->>Client: metrics payload
Possibly related PRs
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Owner
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
Mhemd139
added a commit
that referenced
this pull request
Jul 4, 2026
feat(layer-4): Prometheus metrics + provisioned Grafana dashboard
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@
What
Layer 4 of the roadmap: observability for the proxy.
GET /covenant/metrics— Prometheus text exposition:covenant_calls_total{tool,outcome}(ok/error/blocked),covenant_call_latency_seconds{tool}(forwarded calls only — blocked calls never reach upstream),covenant_drift_total{severity},covenant_quarantined_toolsgauge (synced on refresh + in-band detection).docker compose up -d prometheus grafanagives calls-by-outcome, p95 latency, a quarantine stat that flips green→red within one scrape of a drift, and drift events. Dashboard + datasource fully provisioned, zero clicks.Named decisions (Layer 4 spec)
CollectorRegistryper app — the global registry collides when tests create several proxy apps in one process.prometheus-clientrides the[proxy]extra — metrics are meaningless without the proxy; core CLI stays dependency-light./covenant/metrics, not/metrics— everything Covenant owns lives under/covenant/*; the proxy stays transparent everywhere else.rate()is reset-aware); the Layer 2 store stays the durable record. No double-write.Verified
docker compose config -qvalid🤖 Generated with Claude Code
@
Summary by CodeRabbit
New Features
Bug Fixes