fix(health): keep liveness probes free of store I/O - #102
cursor[bot] wants to merge 11 commits into
Conversation
RED: the domain health snapshot is not yet exposed as GET /live and GET /ready with fail-closed readiness and a safe as-built OpenAPI 3.2 contract.
Map RuntimeHealthSnapshot to GET /live and GET /ready with fail-closed readiness, RFC 9457 problem details for unsupported requests, and an as-built OpenAPI 3.2.0 document that lists only those operations.
Exercise HealthHttpResponse::content_type and GET /ready?capability= from the library test module so Linux line coverage can reach the remaining unexecuted instantiations.
* feat(health): bind a TCP listener for operator probes GET /live and GET /ready already exist as a request translator. Bind a blocking listener that serves one HTTP/1.1 request per accepted connection without adding public product routes, TLS, or SLO values. * docs(traceability): name Active PR #92 on the bound health listener * feat(health): answer probes from a PostgreSQL operational snapshot * test(health): require a composed PostgreSQL operational snapshot RED: runtime and relation probes exist, but callers still assemble liveness, backlog, integrity, and the postgres capability by hand. * feat(health): compose PostgreSQL probes into one operational snapshot Map runtime and relation probes plus caller-supplied backlog into a RuntimeHealthSnapshot. Probe failure becomes unknown/unready evidence and never returns a driver error. * feat(health): answer probes from a PostgreSQL operational snapshot Rebuild #95 on the current #92 listener plus the #93 snapshot composer. handle_postgres_health_http_request and accept_one_postgres_health_http observe the caller-owned store and reuse the existing probe translator. * docs(health): name #95 snapshot wiring on the current listener stack Restore the architecture mappings the rewrite dropped: TRACEABILITY module and capability rows, OPERABILITY fail-closed snapshot readiness, ADR-0014 as-built status, and the composed-snapshot changelog line. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
The inherited #72 recovery fixture inserted a processing consumption row without claim_deadline_at. Migration 0019 requires that column for processing rows, and the deadline trigger is UPDATE-only, so exact-head CI failed closed. Seed a valid persisted claim and assert the deadline survives COPY restore. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
…port RED: GET /live must not observe PostgreSQL, bare GET /ready must fail closed on a read-only store, incomplete or oversized requests must return 400 without echoing input, and the as-built OpenAPI /live operation must document HTTP 503. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Observe PostgreSQL only for GET /ready, after accept. Bare /ready requires postgres_operational_store. The listener applies a 2-second I/O timeout, rejects incomplete or oversized requests without echo, and adds Cache-Control: no-store plus Allow: GET on 405. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
There was a problem hiding this comment.
Stale comment
Current-head review (
79aa953)The #91 production defects are addressed on this head.
handle_postgres_health_http_requestreturns a process-liveness snapshot for every non-GET-/readyrequest, so GET/liveand 400/404/405 do not callobserve_postgres_operational_snapshot.accept_one_postgres_health_httpreads the request first throughaccept_one_health_http_with, then observes PostgreSQL only if the accepted request is GET/ready.- Bare GET
/readyinjectspostgres_operational_store. A read-only or unsupported-major store therefore cannot return HTTP 200 to a default Kubernetes readiness path.- The listener sets a 2-second read/write timeout. Incomplete or oversized requests become empty request text and fail closed as HTTP 400 without echoing input. 405 includes
Allow: GET. Responses sendCache-Control: no-store.Residual, not blocking this slice
RFC 9457 problem bodies still use
"type":"about:blank". #87 requires an explicit HTTPS or URN problem type before that primitive lands. Keep that on #87; do not block operator probes on it.Do not merge until exact-head required checks and independent last-push approval are satisfied. Close or supersede #91 after this lands. Point liveness at GET
/liveand readiness at GET/ready(bare or?capability=postgres_operational_store). Do not treat/liveHTTP 200 as write-readiness.Sent by Cursor Automation: Fix Issues
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
There was a problem hiding this comment.
Stale comment
Verdict: comment on
79aa953The four #91 production defects are fixed in this head, not only restated:
/live, 400, 404, and 405 no longer observe PostgreSQL.accept_one_health_http_withis now accept → I/O timeout → read → handler.- Bare
/readyon the postgres adapter requirespostgres_operational_store. The generic translator still usesis_ready_for(&[])— do not composeobserve_postgres_operational_snapshotwithaccept_one_health_httpor the old default returns.- Read/write timeouts are 2s before the request is read. A hung client gets 400 instead of a wedged accept loop.
- An oversized buffer or a missing
\r \rbecomes an empty request and 400, including a valid/livefirst line plus padding. The body does not echo input.Not approving: the PR is still Draft, TRACEABILITY/ADR-0014 still name Active PR #91, and
/readystill has no statement timeout (a serial accept loop can head-of-line-block/livewhile one/readywaits on PostgreSQL). That last item is caller-owned session state, not a fail-open in this slice.Do not merge #91 at
ec32f969. Land this successor after required checks.Sent by Cursor Automation: Fix Issues
There was a problem hiding this comment.
Verdict: comment on 79aa953
The four #91 production defects are fixed on this head.
- GET
/live, 400, 404, and 405 never callobserve_postgres_operational_snapshot.handle_postgres_health_http_requestreturns a process-liveness snapshot unless the request is GET/ready. accept_one_postgres_health_httpis accept → 2s I/O timeout → read → handler. Store observation cannot start before accept.- Bare GET
/readyon the PostgreSQL adapter requirespostgres_operational_store. A read-only or unsupported-major store cannot return HTTP 200 to a default Kubernetes readiness path. - Incomplete or oversized requests become empty request text and HTTP 400 without echoing input. 405 includes
Allow: GET. Responses sendCache-Control: no-store. OpenAPI documents/live503.
TRACEABILITY and ADR-0014 still name Active PR #91. That is not a correctness failure: #91 and #102 share this SHA, and the documented probe contract matches the code. Close or supersede the non-merge vehicle after landing.
A PostgreSQL statement_timeout on observe is not required for this one-shot acceptor. Add it only when a long-running serial accept loop would otherwise let one hung /ready starve /live.
Do not merge until exact-head required checks and independent last-push approval are satisfied. Never self-approve. Point kubelet liveness at GET /live and readiness at GET /ready. Do not treat /live HTTP 200 as write-readiness.
Sent by Cursor Automation: Fix Issues
|
Closing as a proven superseded duplicate/predecessor. This PR's exact head |


Why
PR #91 exposes GET
/liveand GET/ready, but the PostgreSQL adapter observes the store before parsing the request and beforeaccept(). A hung store can take down liveness, and a stale snapshot can return HTTP 200 on/readyafter the store becomes read-only. Bare/readyalso ignoredpostgres_operational_store, so a default Kubernetes readiness path could keep sending writes to a store that cannot persist. The listener had no I/O timeout and treated an oversized request with a valid first line as a successful/liveprobe.TDD
RED
90a2a7erequired store-free/live, fail-closed bare/readyon a read-only transaction, HTTP 400 for incomplete and oversized requests without echo, and an as-built OpenAPI/live503. GREEN79aa953is the narrow fix.Scope
/liveanswers process liveness only and does not perform store I/O./readyobserves PostgreSQL after accept; a bare/readyrequirespostgres_operational_store.Cache-Control: no-store, and includesAllow: GETon 405.openapi/health-probes.yamldocuments/live503.This branch continues the #91 operator-probe stack with those production-safety corrections. Do not merge until exact-head checks and independent last-push approval are satisfied. Never self-approve.
Operator next action
Review the probe contract tests, then use GET
/livefor process liveness and GET/ready(bare or?capability=postgres_operational_store) for write-readiness. Do not point a load balancer at/liveas a readiness signal.