fix(submit): harden trusted submission ingestion

[ShivamB25]

Summary

• quarantine suspicious usage before it can affect competitive submissions
• return explicit trust decisions (trusted, rejected, or review_required) with machine-readable reason codes
• preserve accepted days while queueing only review-eligible days
• surface queued state in the Rust CLI instead of reporting unconditional success

Rebase onto current main

• re-authored the trust gate around the current submit transaction, preserving device-aware merges, first-write race recovery, savepoints, device-time totals, MCP server fields, cache invalidation, and username revalidation
• renumbered the trust-gate migration to 0015_submit_trust_gate.sql and regenerated 0015_snapshot.json plus meta/_journal.json
• removed the stale submit_count column migration and schema change because main already provides it through 0010_submit_count_safety.sql
• widened submission_reviews.total_cost to numeric(18,4) to match 0014_widen_cost_columns.sql
• kept main's stricter validation: client/token and cost-sum mismatches remain errors, including cost-without-tokens and token-breakdown checks
• preserved the timestamp guard and database CHECK constraints from f0abfbe
• corrected first-write accounting so the first accepted submission remains submit_count = 1
• persisted a monotonic competitive_write_applied review marker so the stacked adjudication flow can avoid counting a partial submission twice

Design decisions

1. Model/date mismatch → review, not rejection. The date extracted from a client-supplied model ID is heuristic; a parse or naming mismatch must not discard otherwise recoverable data.
2. Old schema-v0 days without timestampMs → day-scoped review. Only affected days are queued. Trusted days from the same payload continue through the normal transaction, avoiding whole-submission withholding for legitimate historical backfills.
3. CLI-visible queued state. Review responses return trustState: "review_required", submissionId: null, separate metrics and reviewMetrics, and competitiveWriteApplied. The Rust CLI prints Submitted for review. or Submitted; flagged days queued for review. rather than Successfully submitted!.

Additional ingestion bounds

• cap raw request bodies before JSON parsing
• validate printable identifiers, real calendar dates, integer timestamps, persisted cost ranges, and bounded MCP server arrays
• cap unresolved pending reviews per user and serialize the quota check with a transaction-scoped advisory lock
• keep review insert and trusted-day competitive writes in one transaction
• make the competitive-write marker sticky across pending-review upsert retries

Verification

• Frontend Vitest after latest-main rebase: 66 files, 643 tests passed
• Frontend typecheck: bun run typecheck
• Frontend production build: bun run build
• Migration runtime check against fresh PostgreSQL: contiguous journal 0..15, all migrations applied, schema/constraints/indexes verified, rollback and concurrent pending-review checks passed
• Rust CLI after main.rs conflict resolution: 1004 tests passed, 1 ignored
• Rust core tests in a hermetic environment: 1206 passed, 3 ignored
• Local app + PostgreSQL smoke: fresh payload → trusted; duplicate-date → rejected; future-date → rejected; old day without timestampMs → review_required; model/date mismatch → review_required; no scenario returned 500
• Submit-count regression: first accepted submission persists submit_count = 1

Stack

#445 remains separate and is rebased onto this exact branch head. #444 remains untouched and out of scope.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
tokscale	Ready	Preview, Comment	Jul 14, 2026 9:02am

[cubic-dev-ai]

2 issues found across 14 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/frontend/src/lib/validation/submissionTrust.ts">

<violation number="1" location="packages/frontend/src/lib/validation/submissionTrust.ts:33">
P1: Unvalidated `timestampMs` conversion can throw `RangeError` and crash trust assessment instead of returning structured trust-state output.</violation>
</file>

<file name="packages/frontend/src/lib/db/migrations/0005_concerned_justin_hammer.sql">

<violation number="1" location="packages/frontend/src/lib/db/migrations/0005_concerned_justin_hammer.sql:5">
P2: `submission_reviews` lacks DB integrity constraints for trust state/domain and numeric/date invariants, allowing invalid records despite ingestion hardening.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[ShivamB25]

@junhoyeo @IvGolovach sorry to bother you I may be missing context here, but I noticed the public all-time leaderboard still appears to show some unusually large entries (hundreds of billions of tokens / very large cost totals across a relatively small number of submits).

Since this PR is focused on hardening submit trust/persistence, could you share whether there has already been any review/backfill/removal plan for existing inflated leaderboard rows, or if there are follow-up steps planned? Thanks for taking a look.

[junhoyeo]

@ShivamB25 Hi, thank you for all your efforts! I’ll definitely review them when I have some spare time.

[ShivamB25]

For temporary measure please check the first 2-3 on leaderboard and permanent ban would be nice I believe.

…

On 2 May 2026, at 8:11 PM, Junho Yeo ***@***.***> wrote: junhoyeo left a comment (junhoyeo/tokscale#443) <#443 (comment)> @ShivamB25 <https://github.com/ShivamB25> Hi, thank you for all your efforts! I’ll definitely review them when I have some spare time. — Reply to this email directly, view it on GitHub <#443 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY6KJAOZOWIYRQGRETBNY7T4YYCKLAVCNFSM6AAAAACX6EKEF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DGNRUGA2TCMJQHE>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.

[junhoyeo]

Hey @ShivamB25 — first, apologies that this sat for so long; it's been open since April and main moved a lot underneath it in the meantime. That's on us, not you. The direction here is exactly right and still needed: trust-state decisioning with a review queue is the remediation path for the inflated-leaderboard entries you reported, and nothing on main has implemented it another way. Your latest commit (f0abfbe) also already addressed both cubic findings (timestamp guard + CHECK constraints), and the 55-test suite passes on the branch. Great work.

That said, this needs a real rebase from you rather than a mechanical resolve on our side — the submit pipeline was substantially rewritten on main since you branched, and a few pieces of this PR now conflict with shipped work. Concretely:

Migrations (blocker)

• main now has migrations 0005–0014, so 0005_concerned_justin_hammer.sql needs renumbering to 0015 with a regenerated drizzle snapshot + meta/_journal.json entry.
• Drop the ALTER TABLE "submissions" ADD COLUMN "submit_count" ... statement (and the matching schema.ts hunk) entirely — main already ships submit_count via 0010_submit_count_safety.sql (with IF NOT EXISTS). As written, the migration would abort with column "submit_count" already exists in prod.
• Widen submission_reviews.total_cost from numeric(12,4) to numeric(18,4) to match main's 0014_widen_cost_columns.sql (fix(db): widen total_cost and daily cost columns to avoid numeric overflow #692). This queue will hold exactly the inflated submissions most likely to overflow the narrower type, which would 500 the very submissions it's meant to quarantine.

Validation (src/lib/validation/submission.ts)

• main promoted the client-token/cost-sum mismatch from a warning to an error (see the "must not remain warnings" comment around line 402) and added cost-without-tokens + tokenBreakdown checks. When resolving, please keep main's stricter side — repo policy is to never loosen server-side validation.

Submit route (src/app/api/submit/route.ts)

• The transaction body was rebuilt on main: device-aware merges (fix(submit): make usage merges device-aware #517), first-write race recovery (fix(submit): recover first-write races #616), savepoint handling (fix: invite privilege escalation, submit savepoint race, leaderboard pagination drift, CSRF origin gate, CI coverage floors #611), device time totals (fix(submit): aggregate device time totals #627), and MCP fields (feat: display MCP servers in profile Statistics panel #675). The trust-gate needs re-authoring on top of that structure (and __tests__/api/submit.test.ts mirrors it).

Design questions to settle in the rebase

1. extractDatedModelAvailability hard-REJECTs (400, data discarded) when a day predates the date embedded in the modelId. Since that date is a heuristic parsed from a client-supplied string, would REVIEW_REQUIRED be safer than outright rejection?
2. Any day older than 30 days without timestampMs forces the whole submission into review — schemaVersion-0 CLIs never send timestampMs, so legitimate historical backfills from older CLI versions get withheld entirely. Since the adjudication API only arrives in feat(frontend): isolate public trust and add adjudication #445, we'd like to land fix(submit): harden trusted submission ingestion #443 and feat(frontend): isolate public trust and add adjudication #445 close together so nothing gets stranded; alternatively consider scoping review to only the affected days.
3. The route returns HTTP 200 for review_required, and the Rust CLI (which tolerates the new fields fine — no deny_unknown_fields) will print "Successfully submitted!" with no hint the data is queued. Worth surfacing trustState in the CLI output, or keeping submissionId: null rather than omitting it.

How this fits with the rest of your stack: once #443 is rebased, #445 (adjudication) needs the same treatment on top of it — see the notes I left there. We're closing #444 for now since main's regression-guarded merge covers the worst of the resubmit problem and the replay design needs a rethink on the new architecture (details on that PR), but #443 and #445 are the pieces we actively want to land. Happy to review promptly this time — this is work we want in.

[ShivamB25]

Thanks for the breakdown, this is useful.

Main moving fast makes sense, I've got other things on my plate too, so not blaming the delay. I'll handle all four blockers: renumber the migration, drop the duplicate submit_count, widen total_cost, keep validation on main's stricter side. Will re-author the route on top of #517, #611, #616, #627, #675.

On the three design questions: agree on REVIEW_REQUIRED over hard-reject for the modelId date heuristic. Will scope the 30-day rule to just the affected days. Will surface trustState in the response instead of silently returning submissionId: null.

One ask. The leaderboard still shows inflated entries live in prod while this is open. Once I push the rebase, can we get a faster turnaround on review this time? I'll ping directly when it's ready. Planning to land #443 and #445 close together as you suggested. (example https://tokscale.ai/u/grenadeoftacoss )

[ShivamB25]

@junhoyeo

[ShivamB25]

@junhoyeo Rebase and re-authoring are complete against current main, following your conflict notes. The branch now has a clean four-commit history, the migration is 0015, the duplicate submit_count change is gone, review cost is numeric(18,4), current validation remains strict, and the trust gate is rebuilt around the current submit transaction. All requested runtime outcomes passed without 500s; frontend tests/typecheck/build, migration checks, and Rust suites are green. #445 has also been rebased as a separate layer on this exact head. Ready for re-review.

[ShivamB25]

@junhoyeo One final cross-stack correctness pass found and fixed submit-count accounting: the first accepted write now remains submit_count = 1, and pending reviews persist a monotonic competitive_write_applied marker for #445 so approving a partial submission cannot count the same request twice. The 0015 SQL and snapshot both include the marker, and the retry upsert uses SQL OR so a later retry cannot reset true to false.

Exact #443 head is now 3a4d529f. On that branch: frontend Vitest 65 files / 624 tests passed, typecheck passed, and the focused first-write/retry regression tests passed. #445 was restacked on this exact head. The PR description has the updated contract and evidence.

[ShivamB25]

@junhoyeo Rebased again onto current main (bfb16de8) and resolved the crates/tokscale-cli/src/main.rs conflict. The resolution keeps both sides: current-main’s local-calendar submit behavior (no client-side UTC capping) and #443’s trustState / queued-review confirmation output and tests.

New #443 head: 095a6975. GitHub now reports MERGEABLE. Verification on the rebased stack: full Rust CLI 1004 passed, 1 ignored; exact #443 frontend 66 files / 643 tests passed; frontend typecheck passed; Rust formatting passed.