ops(cf): hourly sweep workflow for orphan Cloudflare DNS records (#239)

[HongmingWang-Rabbit]

[Molecule-Platform-Evolvement-Manager]

Closes Molecule-AI/molecule-controlplane#239.

Why

CF zone hit the 200-record quota on 2026-04-23+ — every short-lived E2E and canary leaves a record on `moleculesai.app`, and no scheduled job is pruning them. Provisions then start failing with code 81045 (`Record quota exceeded`). The `sweep-cf-orphans.sh` script (PR #1978, decision-function unit tests added in #2079) already exists, but nothing fires it on a schedule.

Fix

New `.github/workflows/sweep-cf-orphans.yml`:

• Hourly schedule at `:15` — offset from `sweep-stale-e2e-orgs.yml` (top-of-hour) so the two janitors don't race the same CP admin endpoint
• `workflow_dispatch` with `dry_run` (default true, ad-hoc verify) + `max_delete_pct` (override the script's 50% safety gate for major cleanups)
• `concurrency` group prevents schedule + manual-dispatch from racing the same zone

Why not extend sweep-stale-e2e-orgs.yml

• That workflow drives `DELETE /cp/admin/tenants/:slug`, which assumes CP has the org row. Doesn't catch records left behind when CP never knew about the tenant (canary scratch, manual ops experiments) or when the CP-side cascade's CF-delete branch failed.
• `sweep-cf-orphans.sh` enumerates the CF zone directly + matches each record against live CP slugs + live AWS EC2 names. Catches what the CP-driven sweep can't.

Required secrets (must be set on the repo before this can run)

Secret	Source
`CF_API_TOKEN`	Cloudflare token with `zone:dns:edit` on moleculesai.app
`CF_ZONE_ID`	moleculesai.app zone ID
`CP_PROD_ADMIN_TOKEN`	CP admin bearer for api.moleculesai.app
`CP_STAGING_ADMIN_TOKEN`	CP admin bearer for staging-api.moleculesai.app
`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`	EC2 read-only

Pre-flight `Verify required secrets present` step fails loud if any are missing — single clean error message instead of script-level multi-line noise.

Test plan

• CI green (workflow YAML lint via the existing actionlint hook if any)
• After merge: set the 6 secrets on the repo, then trigger `workflow_dispatch` with `dry_run=true` to confirm the sweep enumerates correctly without deletions
• Confirm scheduled `:15` run starts converging the zone toward the lower record count over 24h
• Once stable, leave `dry_run=false` (the schedule's default) for ongoing pruning

Related

• PR ops(#1976): smart-sweep script for orphan Cloudflare DNS records #1978 — script
• PR test(ops): unit tests for sweep-cf-orphans decide() (#2027) #2079 — decision-function unit tests + parity gate
• Issue [ops] tenant.Delete + workspace.Delete leak Cloudflare DNS records — ~10/hour accumulation #1976 — original CF DNS leak
• molecule-controlplane#261 — sister sweep for orphan EC2s

[HongmingWang-Rabbit]

Reviewed across all five axes — approve, definitely improves code health (closes the #239 quota bleed). Tests exist via the decision-function suite in #2079 + the script is already merged. CI green. Arming auto-merge.

One non-blocking observation:

Optional: `on.merge_group: types: [checks_requested]` will fire the entire `sweep` job on every merge-queue evaluation. The body explains the future-required-check intent, but as written:

• While the 6 secrets are unset (the PR body's post-merge step), the verify step exits 2 on every merge-queue eval → noise but harmless.
• Once secrets ARE configured, `github.event.inputs.dry_run` is empty on merge_group → defaults to "false" → `--execute` branch → actual CF DELETE calls per PR going through the queue.

Two cheap fixes:

1. Drop `merge_group` from the `on:` list (re-add when actually wiring as a required check).
2. Or gate the sweep job: `if: github.event_name != 'merge_group'` and add a no-op success job for the merge_group path.

Worth a follow-up before turning on the secrets, but not blocking this merge — the workflow is dormant until the secrets land anyway. Filing under #239's follow-up if you'd like to track separately.