fix(health): opt-in deep check sampling completions surface

[maxmad64bis]

⚠️ base-red inherited: #13866

Summary

The system health endpoint gains an opt-in deep check (?deep=1, gated on DEEP_HEALTH_CHECK_ENABLED=1 plus management authentication) that samples the completions surface with a minimal one-token non-streaming request and serves the verdict from a 30-second cache. Only 502/503 answers raise the failover signal while all other failures keep the existing payload unchanged; off by default and inert without configuration, anonymous callers never trigger a probe.

Related Issues

• No linked issue — no open upstream issue covers a completions-sampling deep check (neighbors feat(docker): separate liveness from readiness probes #10316, fix(docker): use lifecycle healthz for the Docker HEALTHCHECK #10311 are closed liveness/readiness items).

Validation

• Change type: other
• Focused tests and category gates from the golden path (10 deep-health tests + 52 focused, tsc 0, eslint 0 on touched files)
• npm run lint — touched files clean locally; the full run is red on the base (inherited, non-blocking, see Reviewer Notes)
• Reconciled with the current active release base; focused checks rerun afterward (0 behind, no realignment needed)
• Production-code changes include a new or updated automated test in this PR

Tests Added Or Updated

• tests/unit/monitoring-deep-health.test.ts (new, 10 tests: 502/503 set failover, 4xx/timeout/network stay fail-open, one-token request shape, TTL, anonymous never probed, flag-off inert, status intact, cached verdict identical).

Coverage Notes

• src/lib/monitoring/observability.ts (new probeDeepHealth plus two timing constants) is covered by the probe unit tests across all status classes.
• src/app/api/monitoring/health/route.ts (additive deepHealth key on the full view only, own cache) is covered by the route gating tests including TTL double-serve.

Reviewer Notes

• Split from a three-concern draft: this PR keeps only the deep-health concern (1 commit, 4 files: probe, route wiring, new test file, changelog fragment), with target and credential read from named settings keys no schema supplies yet — this phase ships the probe plus verdict plumbing, not an activatable feature. Fail-open is strict: failover is informative only with no readers, the verdict never alters status, and the probe never throws and never logs bodies.
• The route grew by about 60 lines: mostly pure helpers plus a test-only seed with two additive call sites, and the 1-second payload cache and anonymous view are untouched. DEEP_HEALTH_CHECK_ENABLED stays undocumented (no featureFlagDefinitions.ts, FEATURE_FLAGS.md, .env.example, or ENVIRONMENT.md entry) until the settings phase activates ?deep=1. Fail-open edge cases: flag off or parameter absent keeps the current path, anonymous plus ?deep=1 skips the probe, and refresh runs off the request path with an in-flight guard and 3-second timeout; touched files are clean (tsc 0, ESLint 0) while API Route Typecheck stays red only on rerankProviderNodes.ts and antigravity.ts (TS2677 each, merge-base unchanged, lists identical to fix(proxies): classify refusal cause and hang, add bounded opt-in recovery pass #14233).
• CI reds are inherited from the red base (🔴 Release branch not green: release/v3.8.51 #13866), not from this diff: the third-party PR fix(sse): treat antigravity empty completions with a normal stop as valid 200s (#14160) #14243 on the same base release/v3.8.51 fails the same 9 jobs (API Route Typecheck, Docs Gates, Fast Quality Gates, Merge integrity, ESLint, Unit fast-path 1-4/4); every file cited by the failing gates is outside this diff. Non-blocking for this PR.