feat(deploy): verify each tenant /buildinfo matches published SHA after redeploy - #2398
Merged
Merged
Conversation
…er redeploy Closes the gap that let issue #2395 ship: redeploy-fleet workflows reported ssm_status=Success based on SSM RPC return code alone, while EC2 tenants silently kept serving the previous :latest digest because docker compose up without an explicit pull is a no-op when the local tag already exists. Wire: - new buildinfo package exposes GitSHA, set at link time via -ldflags from the GIT_SHA build-arg (default "dev" so test runs without ldflags fail closed against an unset deploy) - router exposes GET /buildinfo returning {git_sha} — public, no auth, cheap enough to curl from CI for every tenant - both Dockerfiles thread GIT_SHA into the Go build - publish-workspace-server-image.yml passes GIT_SHA=github.sha for both images - redeploy-tenants-on-main.yml + redeploy-tenants-on-staging.yml curl each tenant's /buildinfo after the redeploy SSM RPC and fail the workflow on digest mismatch; staging treats both :latest and :staging-latest as moving tags; verification is skipped only when an operator pinned a specific tag via workflow_dispatch Tests: - TestGitSHA_DefaultDevSentinel pins the dev default - TestBuildInfoEndpoint_ReturnsGitSHA pins the wire shape that the workflow's jq lookup depends on Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HongmingWang-Rabbit
requested a review
from hongmingwang-moleculeai
as a code owner
April 30, 2026 17:55
This was referenced Apr 30, 2026
HongmingWang-Rabbit
pushed a commit
that referenced
this pull request
Apr 30, 2026
Workspace-server has GET /buildinfo (PR #2398) — `curl https://<slug>. moleculesai.app/buildinfo` returns the live git SHA. Canvas had no parallel: debugging "is this the deployed code?" required reading Vercel's UI or response headers (deployment ID, not git SHA). Add canvas /api/buildinfo returning {git_sha, git_ref, vercel_env} sourced from VERCEL_GIT_COMMIT_SHA / _REF / VERCEL_ENV — Vercel injects these at build time from the deploying commit. Outside Vercel (local `next dev`, harness) all three are unset and the endpoint returns `git_sha: "dev"`, the same sentinel workspace-server uses pre-ldflags- injection. Now both surfaces speak the same vocabulary: curl https://<slug>.moleculesai.app/buildinfo curl https://canvas.moleculesai.app/api/buildinfo 3 tests cover dev-fallback, Vercel-injected SHA pass-through, and JSON content type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Apr 30, 2026
HongmingWang-Rabbit
pushed a commit
that referenced
this pull request
Jun 12, 2026
…ce/scope/namespace context (#2398) Replace the generic log.Printf with operator-diagnosis context: - workspace=%s — the calling workspace - scope=%s — requested scope (LOCAL|TEAM|GLOBAL) - namespace=%s — resolved v2 plugin namespace - err_class=%T — Go concrete type for log-aggregator filtering - err=%q — full quoted error message The HTTP 500 response body is unchanged ('failed to store memory') to preserve the no-client-leak discipline. Fixes #2398. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
HongmingWang-Rabbit
pushed a commit
that referenced
this pull request
Jun 12, 2026
…2398) Replace the bare log.Printf("Commit memory error (plugin): %v", err) with operator-diagnosis context: workspace=%s scope=%s namespace=%s err_class=%T err=%q The HTTP 500 response body stays the generic literal "failed to store memory" — zero client-side leak. Only the server-side log is enriched so recurring incidents can be distinguished in the log aggregator. Closes #2398
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #2395. The redeploy-fleet workflows currently report
ssm_status=Successbased on the SSM RPC return code alone — but on hongmingwang.moleculesai.app's tenant, that "Success" was firing 30+ minutes after every push while the EC2 instance kept serving the previous:latestdigest. Root cause:docker compose up -dis a no-op when the local tag already resolves locally; without an explicitdocker pull, "the SSM RPC didn't error" and "the new code is actually running" are different statements.This PR makes them the same statement.
How
internal/buildinfopackage exposesGitSHA, set at link time via-ldflags "-X .../buildinfo.GitSHA=$GIT_SHA". Default value"dev"so test runs without ldflags fail closed against an unset deploy rather than rounding-tripping"".GET /buildinfo → {git_sha}. Public, no auth, cheap enough to curl from CI for every tenant.GIT_SHAfrom a build-arg into the Go build line.publish-workspace-server-image.ymlpassesGIT_SHA=${{ github.sha }}for both images.redeploy-tenants-on-main.ymlandredeploy-tenants-on-staging.ymladd a "Verify each tenant /buildinfo matches published SHA" step that curls every tenant after the redeploy RPC and fails the workflow on mismatch. Staging treats both:latestand:staging-latestas moving tags. Verification is skipped only when the operator pinned a specific tag viaworkflow_dispatch.Why a verify step rather than
docker pullin the SSM scriptBecause the verify step also catches:
Adding
docker pullwould only address one of the four. The verify step is the load-bearing fix.Tests
TestGitSHA_DefaultDevSentinelpins the dev default — guarantees an unset deploy always fails closed.TestBuildInfoEndpoint_ReturnsGitSHApins the wire shape ({"git_sha": "..."}) that the workflow's jq lookup depends on.go build ./...,go vet ./..., both touched packages green.Test plan
🤖 Generated with Claude Code