ci(publish): gate PyPI on server.json registry-schema validation#89
Conversation
Closes #44. Adds a `validate-server-json` job to publish.yml that runs before publish-pypi on every release tag, mirroring the same check that runs on every PR via ci.yml. Without this gate, a malformed server.json (new required field in a future registry schema, type change, etc.) would PyPI-publish cleanly and then fail at the registry leg — leaving a discoverable PyPI release that isn't in the MCP registry, with no way to roll back PyPI short of yanking. v0.5.1 hit the registry-leg-fails-after-PyPI scenario for a different reason (mcp-publisher OIDC audience mismatch, fixed in #79), so the failure mode isn't theoretical. The new job uses the same `./.github/actions/install-mcp-publisher` composite that publish-registry uses, so the validating publisher version always matches the publishing one. The job runs in parallel with `build` (no dep on it), and publish-pypi now lists both as `needs:` so a validation failure stops the entire pipeline before any external side-effect. The optional `publish-manifest` artifact named in #44 is deferred — it's a tooling-version reconciliation aid that's only useful if a CI-PR / release-tag mismatch *does* fire, and is straight- forward to add later when there's a concrete failure to debug. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
cmeans
left a comment
There was a problem hiding this comment.
QA Round 1 — PASS (no findings)
CI-only change on head 3153fdb. 2 files / +23 / -1. Branch is on post-#87 main (merge-base 4e22e35).
Verified
Workflow change (publish.yml):
- New
validate-server-jsonjob runs./mcp-publisher validate server.jsonusing the same./.github/actions/install-mcp-publishercomposite aspublish-registry— so the validating publisher version is exact-pinned to the publishing publisher version. Schema-drift detection isn't subject to a CI-vs-release version mismatch. publish-pypinow listsneeds: [build, validate-server-json]— validation must succeed before any PyPI side-effect; sincepublish-registryandgithub-releasebothneeds: publish-pypi, the gate transitively blocks the entire downstream pipeline.validate-server-jsonandbuildare dep-free, so they run in parallel — gate adds ~10s to the critical path, not a serial round-trip.- yaml.safe_load parses cleanly. Job graph confirmed via Python:
build (no deps) || validate-server-json (no deps) → publish-pypi (needs both) → publish-registry (needs publish-pypi) || github-release (needs publish-pypi).
Same-validator-as-PRs invariant:
ci.yml:40runs./mcp-publisher validate server.jsonon every PR. The new release-time job runs the byte-identical command via the byte-identical composite action. So a PR that passes CI cannot regress the schema validation at tag-time unless mcp-publisher's pinned version moves between merge and tag-push (the exact scenario #44's body called out as the residual risk).- Current branch's
validate-server-json(in CI) is SUCCESS on3153fdb, confirming the validation works on the currentserver.json.
Issue #44 scope:
- AC 1 (validate-server-json before publish-pypi using same pinned mcp-publisher): ✓
- AC 2 (validation failure blocks all jobs): ✓ via the
needs:chain - AC 3 (optional
publish-manifestartifact): explicitly deferred in PR body. Issue marks it "Optional"; deferral rationale (debug aid only useful when a mismatch fires; trivially additive later) is honest.
CI
12/12 required checks green incl. vdsm integration tests SUCCESS on this exact head. validate-server-json green confirms the validation logic is functional today on server.json.
Local sanity (workflow-only PR, no code paths touched)
uv run pytest→ 599 passed, 112 deselected, 96.25% coverage. Post-#87 baseline preserved exactly.uv run ruff check,ruff format --check,mypy src/all clean.
CHANGELOG
New ### Added entry under ## Unreleased, correctly placed above the existing #87 entry. Honest framing: notes that v0.5.1 hit the registry-leg-fails-after-PyPI scenario for a different reason (#79's audience mismatch), so the failure mode isn't theoretical.
PR-body checkboxes
Boxes 1–2 flipped (yaml parse + CI green). Box 3 is post-merge (next release-tag run will exercise the new gate). Box 4 is the informal negative test that the PR body itself rules out running on main — accepted as documented.
Disposition
Ready for QA Signoff applied as the final act. With this in, #51's MEDIUM block is fully closed (#41 via #87, #44 via #89). Roadmap state moves to LOW/docs/chore (#42, #43, #45, #46), architectural ADR (#47), post-baggage features (#48–50), and the newer follow-ups (#25, #75, #76).
|
Applying Ready for QA Signoff — clean CI-only change. New validate-server-json job uses same install-mcp-publisher composite as publish-registry (publisher-version-pinned consistency); publish-pypi needs both build+validate-server-json so a schema failure stops the pipeline before any external side-effect; downstream publish-registry / github-release transitively gated. yaml parses, current ci.yml validate-server-json job is already SUCCESS on 3153fdb confirming the validation works on today's server.json. Local 599/96.25% (post-#87 baseline preserved), ruff/format/mypy clean, CI 12/12 green incl. vdsm. Boxes 1-2 flipped; #3 post-merge, #4 informal-only per PR body. Closes #44 — fully clears #51's MEDIUM block. |
…ublish-registry workflow (#120) Closes #114. Five-file registry-publish setup ported from mcp-synology. No runtime code change. ## Files added | File | Role | |---|---| | `server.json` (root) | Registry manifest. `io.github.cmeans/mcp-clipboard` name (the GitHub-OIDC namespace), starts at version `2.4.0` to match what just shipped to PyPI. | | `scripts/sync-server-json.py` | Single-source-of-truth sync from `pyproject.toml`'s `[project].version` to `server.json`'s two version fields (top-level + `packages[0]`). Stdlib only — runs before any `uv sync` in CI. `--check` mode exits 1 on drift. | | `.github/actions/install-mcp-publisher/action.yml` | Composite action that downloads a pinned `mcp-publisher` (default `v1.7.6`). The pin is load-bearing: the registry rolled out a new GitHub-OIDC audience on 2026-04-30; earlier publisher releases (incl. `v1.5.0`) fail with `401 invalid audience: expected https://registry.modelcontextprotocol.io, got [mcp-registry]`. | ## Workflow changes **`ci.yml` gains two jobs:** - `version-sync` — runs `python scripts/sync-server-json.py --check`. Fails PRs that bump `pyproject.toml` without re-running the sync. - `validate-server-json` — runs `./mcp-publisher validate server.json` against the live registry schema. Catches schema drift at PR time rather than at tag-push time. **`publish.yml` gains:** - `validate-server-json` (release-time backstop). `publish-pypi` is now `needs: [build, validate-server-json]`. Job renamed from `publish` → `publish-pypi` for symmetry with the new `publish-registry` sibling. (Workflow runs only on tag pushes, no PR branch protection references this job name, so the rename is safe.) - `publish-registry` (runs after `publish-pypi`). Uses GitHub OIDC token exchange — no API key, no secret. Idempotent on duplicate-version errors so a partially-failed tag rerun is a no-op; if the upstream error text changes, falls through to `exit $status` and surfaces the real error rather than silently swallowing it. ## Why this matters PyPI publishes are **irreversible per-version**: a yanked release still occupies the version slot. If the registry schema changes between PR-merge and tag-push and we discover it only on the publish run, PyPI ships fine but the registry leg fails — leaving a discoverable PyPI release that isn't in the MCP registry, with no way to fix without bumping the version. Both gates (`ci.yml` PR-time + `publish.yml` release-time) close that window. Reference for the same pattern landing in mcp-synology: - cmeans/mcp-synology#16 — initial `server.json` + first registration attempt. - cmeans/mcp-synology#44 — gating story (why validate runs at both PR time and release time). - cmeans/mcp-synology#79 — the `v1.5.0 → v1.7.6` mcp-publisher bump after v0.5.1's registry leg failed on the OIDC audience mismatch. - cmeans/mcp-synology#89 — `publish-pypi` wired to `needs: [build, validate-server-json]`. - cmeans/mcp-synology#92 — `sync-server-json.py` ergonomics improvements. ## Verification - `python scripts/sync-server-json.py --check` → `server.json in sync with pyproject.toml (2.4.0)`. - `uv run pytest -q`: **548 passed**, 16 deselected, 5 xfailed (no runtime code changed). - `uv run ruff check src/ tests/`: clean. - New CI jobs (`version-sync`, `validate-server-json`) will run on this PR and serve as the first smoke test of the gates themselves. ## First-run plan server.json starts at `2.4.0` (the just-shipped version). After this PR merges: - The next release tag (whether `v2.4.1`, `v2.5.0`, or whatever comes next) fires `publish.yml` → `validate-server-json` → `publish-pypi` → `publish-registry`. - `publish-registry` registers `io.github.cmeans/mcp-clipboard` with the registry for the first time. - Subsequent releases update the entry in place. Optionally we could re-tag `v2.4.0` after merge to register immediately — `publish-pypi` will hit a duplicate-version no-op on PyPI, and `publish-registry` will register the entry. Not strictly necessary; can wait for the next bump. ## Test plan - [x] CI green across `lint`, `typecheck`, `test (3.11/3.12/3.13)`, `integration-x11`, `version-sync`, `validate-server-json`. - [ ] First post-merge tag-push lands all three publish jobs green. - [ ] `curl -s 'https://registry.modelcontextprotocol.io/v0/servers?search=mcp-clipboard'` returns a hit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: cmeans-claude-dev[bot] <272174644+cmeans-claude-dev[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
Closes #44.
Adds a
validate-server-jsonjob topublish.ymlthat runs beforepublish-pypion every release tag, mirroring the same check that already runs on every PR viaci.yml.Why
Without this gate, a malformed
server.json(new required field in a future registry schema, type change, etc.) would PyPI-publish cleanly and then fail at the registry leg — leaving a discoverable PyPI release that isn't in the MCP registry, with no way to roll back PyPI short of yanking. Re-running the failed registry job can't fix that asymmetry.v0.5.1 hit the registry-leg-fails-after-PyPI scenario for a different reason (mcp-publisher OIDC audience mismatch, fixed in #79). The failure mode isn't theoretical — it just hasn't yet fired for a schema reason.
What changed
validate-server-jsonjob inpublish.ymlruns./mcp-publisher validate server.jsonusing the same./.github/actions/install-mcp-publishercomposite thatpublish-registryuses. Validating publisher version always matches publishing publisher version.publish-pypinow listsneeds: [build, validate-server-json]so a validation failure stops the entire pipeline before any external side-effect.validate-server-jsonruns in parallel withbuild— no dep on it — so the gate adds ~10s to the critical path, not a serial round-trip.Why not the optional manifest artifact
The
publish-manifestartifact (recordmcp-publisher/uv/Python versions for CI-PR vs release-tag reconciliation) is named as optional in #44. Deferring — it's a debugging aid that's only useful if a CI-PR / release-tag mismatch does fire, and is trivially additive later when there's a concrete failure to investigate.Test plan
python -c "import yaml; yaml.safe_load(open('.github/workflows/publish.yml'))"parsesvalidate-server-jsonjob in the publish.yml run, gatingpublish-pypiserver.jsonon a branch and tag-push, the publish workflow stops at validate-server-json with no PyPI side-effect — won't actually run this since it requires breaking server.json on main; the validate step is the same./mcp-publisher validateinvocation that ci.yml exercises on every PR, so it's already proven to fail loudly on schema errors🤖 Generated with Claude Code