Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 113 additions & 0 deletions docs/CONTRACT_TESTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Contract Testing for Edge Functions

This document describes the contract-test pattern adopted in PR
`claude/webhook-contract-tests-swzNU` and the migration roadmap for the
remaining Edge Functions.

## Why

The project has ~80 Supabase Edge Functions. Before this work:

- Schema coverage was inconsistent (~40 with Zod, ~40 without).
- Webhook `webhook-inbound` accepted any JSON.
- Error responses had different shapes per function.
- There was no contract versioning, so any breaking schema change would hit
external clients (n8n, logistics APIs, etc.) at once.

This PR introduces:

1. A **shared error helper** with two flavors:
- V1: legacy `{error, details}` with HTTP 400 (byte-for-byte compatible
with the previous `parseBodyWithSchema` behavior — n8n keeps working).
- V2: new `{code, message, fields: [{path, code, message}]}` with HTTP 422
and `Content-Type: application/problem+json`.
2. A **path-based version dispatcher** (`/v1` / `/v2`) that defaults to v1
when no version is present (back-compat).
3. **Contract tests** covering happy path, missing fields, wrong types, empty
values, invalid enums/UUIDs, and unknown keys (v2 strict mode).
4. An **auto-discovery runner** (`npm run test:contracts`) that picks up any
`supabase/functions/<name>/contract.json` manifest.
5. A **coverage gate** (`npm run check:contract-coverage`) that fails CI when
webhook-tier functions miss schema/test/manifest.

## The three shared helpers

```
supabase/functions/_shared/version-dispatch.ts
→ resolveVersion(req) reads URL pathname; falls back to ?_v=2; default v1.
→ withVersionHeader / VERSION_SERVED_HEADER (X-Contract-Version-Served).

supabase/functions/_shared/error-response.ts
→ buildV1ValidationError(err, cors) // 400 / {error, details}
→ buildV2ValidationError(err, cors) // 422 / {code, message, fields}
→ buildV2Error(code, msg, status, cors, fields?)

supabase/functions/_shared/zod-validate.ts
→ parseBodyWithSchema (unchanged — legacy 400)
→ parseBodyVersioned (new — dispatches v1/v2)
```

## Pattern: applying contract versioning to a new function

```ts
// supabase/functions/my-fn/schemas.ts
import { z } from "https://esm.sh/zod@3.23.8";

export const RequestV1 = z.object({ /* current shape */ });
export const RequestV2 = RequestV1.extend({
idempotency_key: z.string().min(8),
}).strict();

export function adaptV1ToCanonical(d) { /* ... */ }
export function adaptV2ToCanonical(d) { /* ... */ }
```

```ts
// supabase/functions/my-fn/index.ts
import { parseBodyVersioned } from "../_shared/zod-validate.ts";
import { RequestV1, RequestV2, adaptV1ToCanonical, adaptV2ToCanonical } from "./schemas.ts";

export const handler = async (req) => {
if (req.method === "OPTIONS") return new Response(null, { headers: corsHeaders });
const parsed = await parseBodyVersioned(req, { v1: RequestV1, v2: RequestV2 }, corsHeaders);
if ("error" in parsed) return parsed.error;
const canonical = parsed.version === "v2"
? adaptV2ToCanonical(parsed.data)
: adaptV1ToCanonical(parsed.data);
// Business logic operates on the canonical shape only.
};
Deno.serve(handler);
```

Then add `index.test.ts` and `contract.json` (see existing webhooks for
examples) or run `npm run generate:contract-stub <name> [--v2]` to scaffold.

## Running the tests

| Command | What it does |
|---------|--------------|
| `npm run test:contracts` | Simulate mode — validates manifest shape. No network. |
| `npm run test:contracts:live` | Hits Supabase deployment with all matrix scenarios. |
| `npm run test:contracts:baseline` | Records v1 response hashes to `scripts/__contracts__/v1-baseline.json`. Commit the file. |
| `npm run test:contracts:deno` | Runs Deno unit tests for the 3 webhooks + shared helpers. |
| `npm run check:contract-coverage` | Reports which functions are missing schema/test/manifest. Fails only on webhook tier in this PR. |
| `npm run check:contract-coverage:strict` | Fail also on public tier (future PRs). |
| `npm run generate:contract-stub <name> [--v2]` | Scaffold the 3 files for a new function. |

## Tier plan (this PR vs future PRs)

| Tier | Functions | Versioning | Status |
|------|-----------|------------|--------|
| T0 | Shared helpers + tests | n/a | ✅ this PR |
| T1 | product-webhook, webhook-inbound, webhook-dispatcher | v1+v2 | ✅ this PR |
| T2 | 13 `verify_jwt=false` non-webhook functions | v1 | Future PRs (use `generate:contract-stub`) |
| T3 | 12 cron functions | v1 | Future PRs (most are exempt — see `contract-exceptions.json`) |
| T4 | ~52 JWT-protected functions | v1 | Future PRs |

## Anti-regression for V1 (n8n & external clients)

- `error-response.test.ts` includes a snapshot test that asserts
`{error, details}` 400 shape stays byte-for-byte stable.
- `scripts/__contracts__/v1-baseline.json` stores response-shape hashes per
endpoint per case. CI compares against this file when run with `--live`.
- A PR that intentionally alters V1 must add the `breaking-v1` label.
10 changes: 9 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,15 @@
"check:toast-leaks": "node scripts/check-toast-leaks.mjs",
"test:stress": "node scripts/massive-load-test.mjs",
"test:fuzz:full": "node scripts/fuzz-testing.mjs",
"test:contract": "node scripts/contract-testing.mjs"
"test:contract": "node scripts/contract-testing.mjs",
"test:contracts": "node scripts/contract-testing.mjs",
"test:contracts:live": "node scripts/contract-testing.mjs --live",
"test:contracts:baseline": "node scripts/contract-testing.mjs --baseline",
"test:contracts:verbose": "node scripts/contract-testing.mjs --verbose",
"test:contracts:deno": "deno test --allow-env --allow-net --allow-read supabase/functions/_shared/version-dispatch.test.ts supabase/functions/_shared/error-response.test.ts supabase/functions/product-webhook/index.test.ts supabase/functions/webhook-inbound/index.test.ts supabase/functions/webhook-dispatcher/index.test.ts",
"check:contract-coverage": "node scripts/check-contract-coverage.mjs",
"check:contract-coverage:strict": "node scripts/check-contract-coverage.mjs --strict",
"generate:contract-stub": "node scripts/generate-contract-stub.mjs"
},
"lint-staged": {
"src/**/*.{ts,tsx}": [
Expand Down
46 changes: 46 additions & 0 deletions scripts/__contracts__/contract.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Edge Function Contract Manifest",
"type": "object",
"required": ["endpoint", "versions", "samples"],
"properties": {
"$schema": { "type": "string" },
"endpoint": {
"type": "string",
"description": "Function slug as deployed at /functions/v1/<endpoint>"
},
"verifyJwt": { "type": "boolean" },
"kind": {
"type": "string",
"enum": ["webhook", "public", "cron", "jwt"]
},
"auth": {
"type": "object",
"description": "Free-form auth descriptor; consumed by the runner"
},
"versions": {
"type": "array",
"items": { "enum": ["v1", "v2"] },
"minItems": 1
},
"samples": {
"type": "object",
"patternProperties": {
"^v[12]$": {
"type": "object",
"properties": {
"valid": {},
"invalid_missing_field": {},
"invalid_wrong_type": {},
"invalid_empty_value": {}
},
"additionalProperties": true
}
}
},
"expectedResponses": {
"type": "object",
"additionalProperties": true
}
}
}
5 changes: 5 additions & 0 deletions scripts/__contracts__/v1-baseline.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"$comment": "Baseline of V1 contract response hashes (status + body shape). Updated by `npm run test:contracts:baseline`. PRs that intentionally alter V1 responses must add the `breaking-v1` label.",
"generated_at": null,
"endpoints": {}
}
Loading