Repository navigation
test(characterization): fix the public surface of cloudAgent, evals, webhooks and gamification before the v4 extraction (rail 3.8.55) - #16192
Merged
Conversation
…rail 3.8.55) Snapshot the runtime exports of src/lib/cloudAgent (barrel + every module) and pin the observable contracts used by the /api/v1/agents routes: registry, listSources/createTask/getStatus with a fake fetch, task persistence, serialisation, credentials and Zod schemas. Current quirks are pinned under 'characterization: ... currently ...' names.
…3.8.55) Snapshot the runtime exports of src/lib/evals (evalRunner, runtime, builtinSuites) and the built-in suite catalogue, then pin the grading contract with a fake-judge suite (custom strategy), scorecards, custom DB suites, runtime request building and pre-flight errors.
…racts (rail 3.8.55) Snapshot the runtime exports of webhookDispatcher and src/lib/webhooks, then pin delivery against a real HTTP server on an ephemeral loopback port: JSON payload, event headers, HMAC-SHA256 signature, 4xx/5xx retry policy, SSRF fail-closed, per-kind routing in dispatchEvent, delivery bookkeeping, auto-disable and the Slack/Discord/Telegram builders.
…ontracts (rail 3.8.55) Snapshot the runtime exports of src/lib/gamification (barrel + every module), XP_REWARDS and the built-in badge catalogue, then pin the XP curve/levels and the award pipeline for deterministic fixtures (points, badges, streak bonus, leaderboard scopes), plus anti-cheat, sharing, invites and servers. Current quirks are pinned, not fixed.
This was referenced Oct 10, 2026
Contributor
CI Coverage Report
Coverage artifact was not available for this run. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rail 3.8.55, Task 11 ("caracterização A"). In v4,
cloudAgent,evals,webhooksandgamificationmove out of the core into@omniroute/mod-*modules. To prove that extraction byte-for-byte, the LTS first needs characterization tests that pin each subsystem's public surface and how it behaves today. This PR adds four of them. No production code changes.What changed
Four new test files in
tests/unit/(the root, so the existing runner glob picks them up). There is one commit per subsystem:characterization-cloudAgent.test.tslistSources/createTask/getStatus/approvePlanwith a fakefetch(URL, headers, body, mapping); task DB round trip;serializeCloudAgentTask; CORS without Origin; credentials save/get/list/delete; Zod schemascharacterization-evals.test.tsevalRunner/runtime/builtinSuites; the built-in suite catalogue;runSuiteon a minimal suite graded by a fake judge (customstrategy); errored-case forcing (#13137); scorecards; custom DB suites; eachevaluateCasestrategy and its error path; runtime request building; pre-flight errors ofrunEvalSuiteAgainstTargetcharacterization-webhooks.test.tswebhookDispatcher+webhooks/**; the event catalogue;deliverWebhookagainst a real HTTP server on an ephemeral loopback port (payload, event headers, HMAC-SHA256 signature, 4xx final, 5xx retry/backoff, SSRF fail-closed without retries, connection refused);dispatchEventrouting by subscription and kind, delivery rows,last_status/failure_count, auto-disable at 10 failures; the Slack/Discord/Telegram builderscharacterization-gamification.test.tsXP_REWARDS; the 23-badge catalogue; the XP curve, level inversion and titles/tiers; the award pipeline on deterministic fixtures (fresh key → 11 XP +first-token; extending a 2-day streak → 27 XP +daily-user); recognition-onlyradar_supporter; anti-cheat; sharing; invites; serversSurface snapshots are literal lists inside each test, so any change to them is a deliberate edit, never an automatic one. Quirks in today's behaviour are pinned, not fixed, and their test names start with
characterization: … currently …:isCloudAgentProviderusesin, so it also matchesObject.prototypekeys such astoString. It has no production caller.CloudAgentBase.mapStatuscheckspendingbeforeapproval, so"pending approval"maps toqueued.CursorCloudAgent.listSourcesturns a non-OK upstream response into[].updateCloudAgentTasksilently drops columns that are not on its whitelist.customeval strategy passes the judge's raw truthy value through aspassed, without converting it to a boolean.deliverWebhookreturnsstatus: 0, "Max retries exceeded"and the last upstream status is lost.TypeErroron an unknown event (events are typed but never validated at runtime).calculateLevelreturns 1, whilexpToNextLevelreturnsNaNor-Infinity.transferTokens(NaN)gets past validation and returns the raw SQLite NOT NULL error. The route and the MCP tool both reject NaN through Zod, so this is not reachable from outside.revokeInvitereturnstruefor an id that does not exist.evaluateBadges(no production caller) rejects withSyntaxErrorwhen any badge definition has malformed criteria.Validation
node --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit tests/unit/characterization-cloudAgent.test.tscharacterization-evals.test.tscharacterization-webhooks.test.tscharacterization-gamification.test.ts--test-concurrency=4 tests/unit/characterization-*.test.tsnpx eslint tests/unit/characterization-*.test.tsnpx prettier --check tests/unit/characterization-*.test.tsnpm run typecheck:coretsconfig.typecheck-core.jsonwith only the 4 filessrc/files, none in the new tests)npm run check:test-discoverynpm run check:file-sizeThe full unit suite and the build were not run, as the campaign preamble requires (the devbox is shared).