Skip to content

feat(backend): Prometheus /metrics endpoint + request middleware - #875

Merged
LucasSantana-Dev merged 2 commits into
release/v2.12.0from
feat/backend-observability
May 16, 2026
Merged

LucasSantana-Dev merged 2 commits into
release/v2.12.0from
feat/backend-observability

Conversation

@LucasSantana-Dev

Copy link
Copy Markdown
Owner

Summary

PR C of the observability rollout. Sentry was already wired in bootstrap.ts; this adds the Prometheus scrape surface to match the bot (#873).

What

Endpoint

  • GET /metrics → Prometheus text exposition (mounted on the existing Express app, before the /api rate limiter, no auth — restrict at the network layer)
  • METRICS_DISABLED=true skips the route entirely

Metrics

  • lucky_backend_http_requests_total{method,route,status} — per-request counter, labelled with the resolved Express route TEMPLATE (e.g. /api/guilds/:guildId), not the raw path, so cardinality stays bounded
  • lucky_backend_http_request_duration_seconds{method,route,status} — histogram with buckets tuned for typical API latencies (5ms – 10s)
  • lucky_backend_http_server_errors_total{method,route} — separate 5xx counter for easier alerting
  • prom-client defaults: process_cpu, eventloop_lag, heap, GC, etc.

Middleware

  • New metricsMiddleware records duration on response 'finish'. Mounted right after requestLogger, before express.json(), so it observes the full server time, not just handler time. Unmatched routes (404s, static) collapse to "unmatched" to keep cardinality finite.

Test plan

  • metrics.test.ts (middleware, 6 cases): counters/histograms recorded on finish, 5xx-only error counter, route template label, unmatched + baseUrl fallbacks
  • metrics.test.ts (route, 2 cases): /metrics serves text exposition with default Node metrics + service="lucky-backend" label; METRICS_DISABLED no-op
  • All 72 existing middleware tests still pass
  • Backend tsc --noEmit clean
  • Manual scrape on staging once deployed: curl backend:5000/metrics | grep lucky_backend_http_requests_total

Deployment notes

  • Same port as the API (PORT / WEBAPP_PORT, default 5000) — no extra port to open
  • If Prometheus is in a separate network, route /metrics through nginx or expose the backend port internally
  • Add to scrape config: job_name: lucky-backend, static_configs.targets: [backend:5000], metrics_path: /metrics

Known CI hazard

CI lint may show a transient youtube-dl-exec postinstall failure unrelated to this PR — tracked in #874 (GitHub API rate limit on shared runners). Re-run the affected check or land #874's fix first.

Out of scope

PR C of the observability rollout. Sentry was already wired in
bootstrap.ts; this adds the Prometheus scrape surface to match the bot
(PR #873).

Endpoint
- GET /metrics → Prometheus text exposition (mounted on the same Express
  app, before the /api rate limiter, no auth — restrict at the network
  layer)
- METRICS_DISABLED=true skips the route entirely

Metrics
- lucky_backend_http_requests_total{method,route,status}: per-request
  counter labelled with the resolved Express route TEMPLATE (e.g.
  /api/guilds/:guildId), not the raw path, so cardinality stays bounded
- lucky_backend_http_request_duration_seconds{method,route,status}:
  histogram with buckets tuned for typical API latencies (5ms – 10s)
- lucky_backend_http_server_errors_total{method,route}: separate 5xx
  counter for easier alerting
- prom-client defaults: process_cpu, eventloop lag, heap, GC, etc.

Middleware
- New metricsMiddleware records duration on response 'finish'. Mounted
  before express.json so it observes the full server time, not just
  handler time. Unmatched routes (404s, static) collapse to "unmatched"
  to keep cardinality finite.

Tests
- metrics.test.ts (middleware): records counters/histograms on response
  finish; 5xx-only error counter; route template label; unmatched +
  baseUrl fallbacks
- metrics.test.ts (route): /metrics serves text exposition with default
  Node metrics + service label; METRICS_DISABLED no-op
- All 72 existing middleware tests still pass

Next: PR D (frontend @sentry/react) and PR E (Grafana dashboards using
these metrics + lucky_bot_guilds_total from #873).
@vercel

vercel Bot commented May 15, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lucky Ready Ready Preview, Comment May 16, 2026 0:12am

Request Review

@coderabbitai

coderabbitai Bot commented May 15, 2026 •

Copy link
Copy Markdown

Warning

Rate limit exceeded

@LucasSantana-Dev has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 51 minutes and 47 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f681106c-9191-46e5-bc9a-91f9335e2d20

📥 Commits

Reviewing files that changed from the base of the PR and between ea886cc and 84aaeb7.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json, !**/package-lock.json
📒 Files selected for processing (8)
  • packages/backend/package.json
  • packages/backend/src/middleware/index.ts
  • packages/backend/src/middleware/metrics.ts
  • packages/backend/src/routes/index.ts
  • packages/backend/src/routes/metrics.ts
  • packages/backend/src/utils/prometheus.ts
  • packages/backend/tests/unit/middleware/metrics.test.ts
  • packages/backend/tests/unit/routes/metrics.test.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/backend-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LucasSantana-Dev has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@github-actions github-actions Bot added dependencies Pull requests that update a dependency file backend size/xl labels May 15, 2026
@github-actions

Copy link
Copy Markdown

Failed to generate code suggestions for PR

@github-actions

github-actions Bot commented May 15, 2026 •

Copy link
Copy Markdown

Size Change: 0 B

Total Size: 422 kB

ℹ️ View Unchanged
Filename Size
packages/frontend/dist/assets/ActionPanel-BJm2-UwJ.js 401 B
packages/frontend/dist/assets/Admin-Cyx3qV4m.js 1.99 kB
packages/frontend/dist/assets/api-AUPUM2Qa.js 3.37 kB
packages/frontend/dist/assets/authStore-D18Icsqi.js 559 B
packages/frontend/dist/assets/AutoMessages-CMmB2nQX.js 2.68 kB
packages/frontend/dist/assets/AutoMod-IIkWFCyV.js 4.09 kB
packages/frontend/dist/assets/badge-BKfLlLik.js 498 B
packages/frontend/dist/assets/Card-DHfizKnr.js 506 B
packages/frontend/dist/assets/Changelog-Bzb0gps0.js 26.6 kB
packages/frontend/dist/assets/CommandsConfig-C9m8Jowf.js 1.47 kB
packages/frontend/dist/assets/Config-CNGVeQEL.js 1.72 kB
packages/frontend/dist/assets/CustomCommands-DUWlpKZm.js 2.16 kB
packages/frontend/dist/assets/DashboardOverview-DjFB3k-Y.js 3.93 kB
packages/frontend/dist/assets/dialog-D4GXigsF.js 948 B
packages/frontend/dist/assets/Docs-DlizujZO.js 17.6 kB
packages/frontend/dist/assets/DocsShell-CcljC3vb.js 1.42 kB
packages/frontend/dist/assets/EmbedBuilder-C9Zm2P7k.js 3.34 kB
packages/frontend/dist/assets/Features-CTsJFBTt.js 754 B
packages/frontend/dist/assets/GuildAutomation-BkkcqQ-5.js 2.92 kB
packages/frontend/dist/assets/guildStore-il-poOxg.js 791 B
packages/frontend/dist/assets/index-B6IXjcnV.css 17.6 kB
packages/frontend/dist/assets/index-eFskPvjF.js 53.9 kB
packages/frontend/dist/assets/input-DFOdST7m.js 465 B
packages/frontend/dist/assets/label-CHNOsjRF.js 488 B
packages/frontend/dist/assets/Landing-DS5COV02.js 5.07 kB
packages/frontend/dist/assets/LastFm-BpjclAYg.js 1.74 kB
packages/frontend/dist/assets/legalNav-Dg0pcNaV.js 270 B
packages/frontend/dist/assets/Levels-Cwq3UvvY.js 2.63 kB
packages/frontend/dist/assets/Login-Djv5bPUR.js 2.5 kB
packages/frontend/dist/assets/Lyrics-h-V2IrJO.js 1.33 kB
packages/frontend/dist/assets/Moderation-CX9QU1JK.js 3.85 kB
packages/frontend/dist/assets/Music-DMqcd96v.js 7.09 kB
packages/frontend/dist/assets/MusicConfig-CQYyqa3R.js 1.61 kB
packages/frontend/dist/assets/PreferredArtists-QvjX_VBy.js 3.76 kB
packages/frontend/dist/assets/PrivacyPolicy-Dj58jJqO.js 1.77 kB
packages/frontend/dist/assets/ReactionRoles-DJU4ywNC.js 1.89 kB
packages/frontend/dist/assets/rolldown-runtime-BYbx6iT9.js 471 B
packages/frontend/dist/assets/SectionHeader-D6idj02z.js 893 B
packages/frontend/dist/assets/select-Bo9ZxOQ5.js 1.23 kB
packages/frontend/dist/assets/ServerLogs-xpeRhEqQ.js 2.91 kB
packages/frontend/dist/assets/ServerSettings-UfbcIO3v.js 4.21 kB
packages/frontend/dist/assets/ServersPage-CpgN-80x.js 2.92 kB
packages/frontend/dist/assets/shim-Qmczh487.js 510 B
packages/frontend/dist/assets/Skeleton-BJSvKI8c.js 236 B
packages/frontend/dist/assets/Spotify-hp1neso5.js 1.75 kB
packages/frontend/dist/assets/Starboard-Bvvb_b68.js 2.08 kB
packages/frontend/dist/assets/StatTile-DQ3lYBMP.js 636 B
packages/frontend/dist/assets/switch-C5G39PBt.js 544 B
packages/frontend/dist/assets/TermsOfService-DFdRongw.js 1.61 kB
packages/frontend/dist/assets/TrackHistory-CBCxIeII.js 2.17 kB
packages/frontend/dist/assets/TwitchNotifications-3ItXaqam.js 2.44 kB
packages/frontend/dist/assets/useActiveHeading-BmLnT6P1.js 1.35 kB
packages/frontend/dist/assets/useFeatures-BTKzapaa.js 2.06 kB
packages/frontend/dist/assets/usePageMetadata-DVjH87aq.js 329 B
packages/frontend/dist/assets/utils-Denu3VwY.js 146 B
packages/frontend/dist/assets/vendor-forms-VJTTJemd.js 25.6 kB
packages/frontend/dist/assets/vendor-radix-DKbKEiWx.js 38.9 kB
packages/frontend/dist/assets/vendor-react-DNaNL-Gj.js 55.6 kB
packages/frontend/dist/assets/vendor-state-B_fYO9zl.js 24.1 kB
packages/frontend/dist/assets/vendor-ui-23gi6Zpr.js 65.5 kB

compressed-size-action

LucasSantana-Dev added a commit that referenced this pull request May 16, 2026
)

## Summary

Fixes #874 by:

1. **Bumping the pinned ref of the org reusable \`quality.yml\`** to the
merge commit of
[LucasSantana-Dev/.github#2](LucasSantana-Dev/.github#2).
That PR adds \`GITHUB_TOKEN\` + \`YOUTUBE_DL_SKIP_DOWNLOAD=true\` to the
install step in the \`lint\` and \`deadcode\` jobs, eliminating the 60
req/hr unauthenticated cap that's been failing PRs since the
observability rollout started.

2. **Applying the same env vars to local \`release.yml\`** for parity.
Release builds don't strictly need the yt-dlp binary, so skipping the
fetch is harmless.

## Why this scope

| Workflow | npm ci flags | Status |
|---|---|---|
| \`ci.yml\` | \`--legacy-peer-deps --ignore-scripts\` | already safe |
| \`sonarcloud.yml\` | \`--legacy-peer-deps --ignore-scripts\` | already
safe |
| \`bundle-size.yml\` | runs postinstall | already sets both env vars |
| \`release.yml\` | bare \`npm ci\` | **fixed here** |
| reusable \`quality.yml\` | runs postinstall | **fixed via SHA bump** |

## Closes

- #874

## Unblocks

- PR #873 (bot observability metrics)
- PR #875 (backend observability)
- PR #876 (frontend Sentry)
- PR #877 (paired Lucky-side network for homelab #135)

## Test plan

- [ ] \`quality / Lint\` passes on this PR's first run
- [ ] After merge: re-run failed \`quality / Lint\` on #873 / #875 /
#876 — they should pass without retry

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LucasSantana-Dev has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@sonarqubecloud

Copy link
Copy Markdown

@LucasSantana-Dev
LucasSantana-Dev merged commit a187390 into release/v2.12.0 May 16, 2026
28 checks passed
@LucasSantana-Dev
LucasSantana-Dev deleted the feat/backend-observability branch May 16, 2026 02:01
LucasSantana-Dev added a commit that referenced this pull request May 16, 2026
)

## Summary

Adds the `lucky-monitoring` Docker network and joins both `bot` and
`backend` services to it, so the homelab Prometheus container can
scrape:

- `lucky-bot:9091/metrics` (added in #873)
- `lucky-backend:5000/metrics` (added in #875)

…without exposing those ports to the host.

Lucky owns/creates the network; homelab joins it as `external: true` in
`compose/monitoring.yml` (homelab PR #135).

## Why this PR exists separately

PR #135 in the homelab repo declares `lucky-monitoring` as an
**external** network. Without this Lucky-side change, the homelab stack
would fail to start (network does not exist) and the new `LuckyBotDown`
/ `LuckyBackendDown` alerts would fire immediately on deploy because the
scrape targets would not be DNS-resolvable.

## Deploy order

1. Merge this PR (creates the network on the Lucky compose project on
next deploy)
2. Redeploy Lucky → `docker network ls` shows `lucky-monitoring`
3. Merge homelab #135
4. Redeploy homelab monitoring stack → Prometheus container joins the
existing network
5. Verify in Grafana that `up{job=~"lucky-.*"} == 1`

## Test plan

- [ ] CI passes (lint / Docker build / compose validate)
- [ ] After deploy: `docker network inspect lucky-monitoring` shows both
`lucky-bot` and `lucky-backend` attached
- [ ] After homelab redeploy: Prometheus target page shows both Lucky
targets as `UP`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated Docker Compose configuration to enable monitoring
infrastructure connections for bot and backend services.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/LucasSantana-Dev/Lucky/pull/877?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
LucasSantana-Dev added a commit that referenced this pull request May 16, 2026
## Summary

PR D of the observability rollout. Adds browser-side error reporting +
performance tracing to the dashboard SPA, completing the data-layer
coverage across bot (#873), backend (#875), and frontend.

## What

- **`Sentry.init`** on app boot via `initSentry()` called from
`main.tsx` before React mounts so early errors are captured
- **`reactRouterV7BrowserTracingIntegration`** wired to
`react-router-dom` v7 so route changes become Sentry transactions; LCP /
CLS / INP captured automatically per transaction
- **`replayIntegration`**: 10% of sessions, 100% of sessions that hit an
error
- **`captureFrontendException(err, context?)`** helper for catch-block
reporting where the existing UI already surfaces the error state

## Configuration

All optional Vite env vars — missing `VITE_SENTRY_DSN` makes
`initSentry` a no-op (dev / preview / self-hosted without telemetry):

| Env | Default | Meaning |
|---|---|---|
| `VITE_SENTRY_DSN` | — | Enables Sentry when set |
| `VITE_SENTRY_ENVIRONMENT` | `import.meta.env.MODE` | dev / staging /
prod |
| `VITE_SENTRY_RELEASE` | `VITE_COMMIT_SHA` | Release identifier |
| `VITE_SENTRY_TRACES_SAMPLE_RATE` | `0.1` | % of route changes traced |
| `VITE_SENTRY_REPLAYS_SESSION_SAMPLE_RATE` | `0.1` | % of sessions
replayed |
| `VITE_SENTRY_REPLAYS_ON_ERROR_SAMPLE_RATE` | `1.0` | % of error
sessions replayed |

## Test plan

- [x] `sentry.test.ts` (7 cases):
  - no-op without DSN (privacy-by-default for self-hosted)
  - `Sentry.init` shape with DSN + env + release
  - Router v7 + replay integrations wired
  - custom sample rates honored
  - default sample rates applied when env missing
  - `captureFrontendException` with and without extra context
- [x] Frontend \`type:check\` clean
- [x] Frontend \`lint\` clean (\`--max-warnings 0\`)
- [ ] Manual: set \`VITE_SENTRY_DSN\` in \`.env.local\`, click around,
confirm a route-change transaction lands in Sentry

## Deployment notes

- Set \`VITE_SENTRY_DSN\` (and the rest of the \`VITE_SENTRY_*\` env, as
needed) in the production build secrets / \`.env.production\`
- Set \`VITE_SENTRY_RELEASE\` to the commit SHA from CI so errors are
tagged with the right release
- No DSN exposed at build time → completely off in self-hosted forks
unless the operator opts in

## Known CI hazard

The yt-dlp postinstall flake (#874) may trip the quality / Lint job.
Re-run or land #874's fix first.

## Out of scope

- Source maps upload via \`@sentry/vite-plugin\` — wire in PR E once the
release pipeline is firmed up
- Standalone Web Vitals package — Sentry's \`browserTracingIntegration\`
already captures the core vitals on the first navigation transaction, no
need for a duplicate path
- **PR E**: Grafana dashboards + alert rules consuming the metrics from
#873 + #875 + this PR's Sentry data
LucasSantana-Dev added a commit that referenced this pull request May 21, 2026
## Summary

Cut v2.13.0 of Lucky. Bumps root + 4 workspaces from `2.11.0` → `2.13.0`
(skipping the archived `2.12.0`) and promotes the CHANGELOG
`[Unreleased]` block to `[2.13.0] - 2026-05-21`.

## Headline changes since v2.11.0

**Added**
- Guild Automation Module Executor seam + AutoMessages pilot (#901)
- Sentry React SDK + Router v7 tracing/replay on frontend (#876)
- Prometheus `/metrics` on backend (#875) + bot (#873)
- Guild join/leave history tracking (#872)
- Trivy image-scan on docker-publish, Phase A audit-only (#883)
- Self-hosted developer-tooling register on landing page (#868)

**Changed**
- Backend migrated to Zod 4 API (#919) — unblocked the CVE patch + ended
the lockfile fragility loop
- 3 bot circular-deps clusters broken (#885, #886, #888)

**Fixed**
- brace-expansion DoS + ws uninit-memory CVEs patched (#921)
- nginx-alpine CVEs (#881)
- CI postinstall rate limit + madge actionlint (#878, #905)

Full list in CHANGELOG.md.

## Next steps (after this PR merges)

1. Open `release/v2.13.0 → main` PR with merge-commit method
2. Tag `v2.13.0` on the merge commit
3. Cut next `release` (homelab-style bare branch) — Lucky's bare-release
migration is still pending the user removing protection on
`release/v2.11.0`
@LucasSantana-Dev LucasSantana-Dev mentioned this pull request May 21, 2026
3 tasks
LucasSantana-Dev added a commit that referenced this pull request May 21, 2026
## Release v2.13.0

Promotes \`release/v2.13.0\` to \`main\` for the v2.13.0 cut.

**$AHEAD commits across all merged PRs since v2.11.0 ship.**

(Skipping v2.12.0 — the branch existed but its work was rolled forward
into v2.13.0 alongside this session's Zod migration + CVE patches +
standards adoption.)

## Headline changes

**Added** — Guild Automation Module Executor pilot (#901), Sentry
frontend (#876), Prometheus metrics on bot+backend (#873, #875), guild
membership history (#872), Trivy image-scan Phase A (#883), landing
redesign (#868).

**Changed** — Backend migrated to Zod 4 API (#919), 3 bot circular-deps
clusters broken (#885/#886/#888).

**Fixed** — brace-expansion + ws moderate CVEs (#921), nginx-alpine CVEs
(#881), CI postinstall rate limit (#878), madge actionlint (#905).

**Internal** — shared coverageThreshold gate (#909/#914),
Feature-removal sweep checklist + dangerfile guard (#908/#913),
monitoring network, AI-doc policy, 4 new ADRs.

Full list in [CHANGELOG.md](./CHANGELOG.md).

## Merge method

This PR should land via **merge commit** (NOT squash) to preserve the
individual PR SHAs in main's history. After merge:

1. Tag \`v2.13.0\` on the merge commit
2. Create GitHub release with notes from CHANGELOG.md
3. Fast-forward \`release/v2.13.0\` to match the new main HEAD

## Test plan

- [ ] All 30 checks green except infra (snyk plan cap)
- [ ] Verify \`gh pr view 922 --json mergeCommit\` shows the chore-bump
commit on release tip
- [ ] After merge: confirm \`origin/main\` contains the full $AHEAD
commits

This branch was successfully deployed

1 active deployment
Preview — 84aaeb71 Deployed May 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend dependencies Pull requests that update a dependency file size/xl

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant