Skip to content

Reject invalid structured values in storage - #1865

Merged
stefan-burke merged 19 commits into
mainfrom
normalize/validated-storage
Jul 20, 2026
Merged

Reject invalid structured values in storage#1865
stefan-burke merged 19 commits into
mainfrom
normalize/validated-storage

Conversation

@stefan-burke

@stefan-burke stefan-burke commented Jul 19, 2026

Copy link
Copy Markdown
Member

What changed

  • Added one schema-backed JSON format for database reads and writes.
  • Moved stored listing options, attendee details, site details, contact statistics, payment failures, checkout metadata, token attempts, listing defaults, and admin feature settings onto that shared validation path.
  • Added row and field context to errors so corrupt stored values point to the record that needs repair.
  • Redacts rejected stored values from validation error messages and causes.
  • Uses only the one-way SumUp reference index in corrupt checkout repair errors.
  • Rejects non-text payment metadata before a paid session can create a booking.
  • Rejects unsafe integers, unsafe day prices, and non-canonical day-price keys such as 01.
  • Returns clear 400 errors when listing API requests contain fractional durations, non-text bookable days, or unsafe day prices.
  • Clamps valid whole booking durations where intended and rejects malformed numbers.
  • Keeps supported older stored formats, including address results, pre-versioned attendee details, and site details without a version field.

Why

Structured values were often parsed from storage without checking their shape. Bad data could then fail later in unrelated code or be accepted with the wrong types. Reads and writes now check the same declared format and stop at the boundary with a useful error that does not expose the rejected value.

Tests

  • Full deno task precommit passed, including lint, typecheck, duplication checks, edge build, full tests, and deterministic coverage.
  • Targeted mutation tests passed at 100% for the stored JSON codec, table integration, number validation, listing storage, listing defaults, admin features, built sites, payment helpers, SumUp checkout storage, and the admin listings API.
  • Added regressions for malformed JSON, wrong shapes, invalid writes, redacted errors, supported older shapes, row context, duration clamping, malformed numbers, unsafe integer strings, unsafe day prices, colliding day-price keys, non-text signed payment metadata, corrupt SumUp metadata, and malformed listing API fields.

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This change adds schema-backed stored JSON validation, replaces duration normalization with bounded clamping, strengthens admin API and payment metadata validation, expands database and integration tests, and updates equivalent-mutant coordinates.

Changes

Validation and persistence migration

Layer / File(s) Summary
Validation contracts and duration policies
src/shared/validation/*, src/shared/types.ts, src/shared/address-lookup/types.ts, src/features/admin/catalog-transfer/schema.ts
Adds bounded integer helpers, stored JSON codecs, reusable schemas, day-price validation, and clamped duration handling.
JSON column and persistence integration
src/shared/db/table.ts, src/shared/db/*, src/shared/admin-features.ts, src/shared/listing-defaults.ts
Applies schema-backed serialization and parsing to database columns, encrypted blobs, settings, caches, payments, contacts, tokens, and built sites.
Duration handling adoption
src/features/*, src/shared/*, src/ui/templates/*
Uses clampDurationDays across booking, capacity, attendee, listing, date, email, administrative, and ticket paths.
Admin API and payment validation
src/features/admin/api.ts, src/shared/payment-helpers.ts, test/features/admin/*, test/integration/*, test/shared/payment-helpers/*
Validates mapped request fields and payment metadata, with expanded listing, child-ID, lifecycle, metadata, and logging coverage.
Regression and mutation coverage
test/shared/*, test/test-utils/debug-log.ts, scripts/mutation/equivalent-mutants.txt
Adds stored-JSON, table, cache, validation, test-isolation, and mutation metadata coverage.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main change: adding validation to reject invalid structured values in storage.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch normalize/validated-storage

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 44b5e87805

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/features/api/payment-processing/metadata.ts Outdated
Comment thread src/shared/types.ts
Comment thread src/shared/validation/number.ts Outdated
Comment thread src/shared/types.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/features/admin/attendees.ts`:
- Around line 208-210: Update the comment describing clampDurationDays near the
daily booking duration logic to state that valid whole numbers outside the
allowed range are clamped, while fractions, NaN, and unsafe integers are
rejected. Keep the distinction between customizable daily bookings and
fixed-duration bookings accurate.

In `@src/shared/db/built-sites.ts`:
- Around line 68-81: Update SiteDataBlobSchema and the built_sites
restore/parsing flow so legacy site_data blobs missing v are backfilled with
SITE_DATA_BLOB_VERSION before strict validation, or migrate them before parsing.
Preserve strict validation for all other fields and ensure current blobs
continue using the existing v value.

In `@src/shared/types.ts`:
- Around line 214-230: Update DayPriceKeySchema to accept only canonical decimal
day-count keys, rejecting leading-zero values such as "01" while preserving
valid keys like "1". Add a regression case covering a record containing both
"01" and "1", ensuring the schema rejects this collision.
- Around line 203-204: Update the exported rejectDurationDays declaration to
include an explicit function parameter and return type, while preserving its
existing delegation to durationDays.reject and validation behavior.

In `@src/shared/validation/number.ts`:
- Around line 13-14: Declare an explicit return type for the exported
integerAtLeast function, using the appropriate validator/schema type for the
value returned by its v.pipe expression. Preserve the existing number,
safe-integer, and minimum-value validation behavior.
- Around line 47-52: Update NonNegativeIntSchema to reject digit-only values
outside JavaScript’s safe-integer range before or during the Number
transformation, while preserving valid non-negative integer behavior and
PositiveIntSchema’s existing constraint. Add a regression test using a
digit-only value greater than Number.MAX_SAFE_INTEGER and assert that validation
returns null.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3555d1ea-6b0a-401c-9e24-84de03a1bb98

📥 Commits

Reviewing files that changed from the base of the PR and between 054111a and 44b5e87.

📒 Files selected for processing (53)
  • scripts/mutation/equivalent-mutants.txt
  • src/features/admin/attendee-form-model.ts
  • src/features/admin/attendees.ts
  • src/features/admin/catalog-transfer/import.ts
  • src/features/admin/catalog-transfer/schema.ts
  • src/features/api/payment-processing/metadata.ts
  • src/features/public/pages.ts
  • src/shared/address-lookup/types.ts
  • src/shared/admin-features.ts
  • src/shared/booking-date-fields.ts
  • src/shared/booking/model.ts
  • src/shared/dates.ts
  • src/shared/db/address-cache.ts
  • src/shared/db/attendees/capacity/range.ts
  • src/shared/db/attendees/create.ts
  • src/shared/db/attendees/pii.ts
  • src/shared/db/attendees/servicing.ts
  • src/shared/db/attendees/update.ts
  • src/shared/db/built-sites.ts
  • src/shared/db/capacity.ts
  • src/shared/db/contact-preferences.ts
  • src/shared/db/groups.ts
  • src/shared/db/listings/table.ts
  • src/shared/db/processed-payments.ts
  • src/shared/db/sumup-checkouts.ts
  • src/shared/db/table.ts
  • src/shared/db/token-attempts.ts
  • src/shared/email-renderer.ts
  • src/shared/listing-defaults.ts
  • src/shared/listing-parents-rules.ts
  • src/shared/listings-actions.ts
  • src/shared/types.ts
  • src/shared/validation/number.ts
  • src/shared/validation/stored-json.ts
  • src/shared/validation/string.ts
  • src/ui/templates/admin/listings/details.tsx
  • src/ui/templates/admin/listings/form-values.tsx
  • src/ui/templates/tickets.tsx
  • test/shared/admin-features.test.ts
  • test/shared/db/address-cache.test.ts
  • test/shared/db/attendees/pii.test.ts
  • test/shared/db/built-sites.test.ts
  • test/shared/db/contact-preferences.test.ts
  • test/shared/db/listings/table.test.ts
  • test/shared/db/processed-payments.test.ts
  • test/shared/db/sumup-checkouts.test.ts
  • test/shared/db/table.test.ts
  • test/shared/db/table/cache.test.ts
  • test/shared/db/table/write-row.test.ts
  • test/shared/db/token-attempts.test.ts
  • test/shared/listing-defaults.test.ts
  • test/shared/validation/number.test.ts
  • test/shared/validation/stored-json.test.ts

Comment thread src/features/admin/attendees.ts Outdated
Comment thread src/shared/db/built-sites.ts
Comment thread src/shared/types.ts Outdated
Comment thread src/shared/types.ts
Comment thread src/shared/validation/number.ts Outdated
Comment thread src/shared/validation/number.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37ffc03162

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/shared/db/listings/table.ts
Comment thread src/shared/db/listings/table.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/shared/db/built-sites.test.ts`:
- Around line 211-223: Update the “toDbValues identifies invalid site data” test
to handle the asynchronous rejection from builtSitesCrudTable.toDbValues: pass
its returned promise directly to expect(...).rejects and assert the existing
error message, rather than using synchronous toThrow.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 46c70b38-979a-4579-8d2a-83d20716ffa5

📥 Commits

Reviewing files that changed from the base of the PR and between 37ffc03 and 703c76d.

📒 Files selected for processing (17)
  • scripts/mutation/equivalent-mutants.txt
  • src/features/admin/attendees.ts
  • src/shared/db/built-sites.ts
  • src/shared/payment-helpers.ts
  • src/shared/types.ts
  • src/shared/validation/number.ts
  • test/lib/server-webhooks/single-ticket-refunds.test.ts
  • test/shared/db/built-sites.test.ts
  • test/shared/db/listings/table.test.ts
  • test/shared/logger/log-output.test.ts
  • test/shared/payment-helpers/build-items.test.ts
  • test/shared/payment-helpers/dispatch.test.ts
  • test/shared/payment-helpers/limits.test.ts
  • test/shared/payment-helpers/metadata.test.ts
  • test/shared/payments.test.ts
  • test/shared/validation/number.test.ts
  • test/test-utils/debug-log.ts

Comment thread test/shared/db/built-sites.test.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e9d82c7545

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/shared/validation/number.ts Outdated
stefan-burke and others added 2 commits July 19, 2026 19:21
* Fix reservation balance summaries

* Extend external screenshot scenarios

* Fix mobile booking controls

* Address reservation balance review

* Derive balance query arguments

* Clarify reservation order totals

* Improve paid checkout screenshots

* Start Stripe mock for screenshots

* Clean up failed screenshot startup

* Clean up all screenshot resources

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/ui/client/admin/order-gallery.test.ts`:
- Around line 120-121: Update the afterEach teardown in the test to restore
clock.time with the repository’s invariant-enforcing non-null assertion instead
of optional chaining. Add a brief comment documenting that beforeEach always
installs FakeTime, so the value must exist and teardown should fail loudly if it
does not.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f0b33177-c991-43a2-9e72-5027e50dcc21

📥 Commits

Reviewing files that changed from the base of the PR and between e9d82c7 and 1111bfa.

📒 Files selected for processing (4)
  • src/shared/types.ts
  • src/shared/validation/number.ts
  • test/shared/validation/number.test.ts
  • test/ui/client/admin/order-gallery.test.ts

Comment thread test/ui/client/admin/order-gallery.test.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 209e1e6267

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/shared/types.ts
stefan-burke and others added 3 commits July 19, 2026 21:13
* Normalize declarative form inputs

* Consolidate declarative input handling

* Address declarative input review feedback

* Unify declarative input sources

* Address final declarative input review

* Use functional column layout parsing
* Normalize lookup and result contracts

* Move changed integration tests into mapped suites

* Map admin API tests to their source

* Map listing API tests to their source

* Add branch code line task

* Keep required lookups at domain boundaries

* Address lookup contract review feedback

* Preserve optional provider and write-back outcomes

* Keep cache hits stable during batch loads
@stefan-burke
stefan-burke added this pull request to the merge queue Jul 19, 2026
@stefan-burke
stefan-burke removed this pull request from the merge queue due to a manual request Jul 19, 2026
@stefan-burke
stefan-burke added this pull request to the merge queue Jul 19, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Jul 19, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 47c9451747

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/shared/db/sumup-checkouts.ts Outdated
Comment thread src/shared/validation/stored-json.ts Outdated
@stefan-burke
stefan-burke added this pull request to the merge queue Jul 20, 2026
Merged via the queue into main with commit 53aa3bf Jul 20, 2026
3 checks passed
@stefan-burke
stefan-burke deleted the normalize/validated-storage branch July 20, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant