Skip to content

refactor: clear raw-regex and process.env backlogs#2313

Merged
andrew-bierman merged 6 commits into
developmentfrom
feat/checks-backlog-2
Apr 26, 2026
Merged

refactor: clear raw-regex and process.env backlogs#2313
andrew-bierman merged 6 commits into
developmentfrom
feat/checks-backlog-2

Conversation

@andrew-bierman
Copy link
Copy Markdown
Collaborator

Chained off #2312. Clears two more check categories and promotes them to hard gates.

Summary

  • 84 → 0 raw regex violations — all inline regex literals extracted to named top-level constants across 18 files. no-raw-regex now blocks at pre-push.
  • 19 → 0 process.env violations — 5 legitimate uses added to the allowlist (the check script itself, analytics env module, api scripts, guides sync); apps/admin gets a proper Zod-parsed env shim (apps/admin/lib/env.ts) mirroring the pattern from packages/env/src/next.ts.
  • CI hardenedlint:custom (no-raw-typeof + no-raw-regex + no-raw-process-env) now runs without continue-on-error. Only check-type-casts remains soft-gated (130 casts, next PR).
  • Pre-push gate expandedno-raw-regex and no-raw-process-env added alongside the other clean checks.

Commits

🔒 Clear process.env backlog — exempt scripts + add admin env shim
🔧 Hoist inline regex → named constants (batch 1: format-ai-response, csv-utils, mcp tools, chart, guides scripts)
🔧 Hoist inline regex → named constants (batch 2: reader.ts HTML→MD, trailConditions, wildlife, r2-bucket, embeddingService, entity-resolver)
🔧 Promote no-raw-regex + no-raw-process-env to hard gates

Test Plan

  • bun scripts/lint/no-raw-regex.ts → 0 violations
  • bun packages/env/scripts/no-raw-process-env.ts → 0 violations
  • bun lint:custom passes clean (all 3 checks)
  • bun check:all — only check-type-casts and check-react-doctor still fail
  • Spot-check csv-utils.ts — 15 named regex constants grouped at top of file
  • Spot-check reader.ts — 29 HTML→Markdown patterns at top, chain reads cleanly

Copilot AI review requested due to automatic review settings April 26, 2026 20:34
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 26, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 45f75fac-6ad7-48dc-a360-9f0c1f37c1dd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • ✅ Review completed - (🔄 Check again to review again)
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/checks-backlog-2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 26, 2026

Coverage Report for Expo Unit Tests Coverage (./apps/expo)

Status Category Percentage Covered / Total
🔵 Lines 81.43% 522 / 641
🔵 Statements 81.43% (🎯 75%) 522 / 641
🔵 Functions 92.85% 52 / 56
🔵 Branches 92.55% 199 / 215
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
apps/expo/utils/format-ai-response.ts 100% 100% 100% 100%
Generated in workflow #833 for commit d443422 by the Vitest Coverage Report Action

@cloudflare-workers-and-pages
Copy link
Copy Markdown
Contributor

cloudflare-workers-and-pages Bot commented Apr 26, 2026

Deploying packrat-guides with  Cloudflare Pages  Cloudflare Pages

Latest commit: d443422
Status: ✅  Deploy successful!
Preview URL: https://abfea625.packrat-guides-6gq.pages.dev
Branch Preview URL: https://feat-checks-backlog-2.packrat-guides-6gq.pages.dev

View logs

@cloudflare-workers-and-pages
Copy link
Copy Markdown
Contributor

cloudflare-workers-and-pages Bot commented Apr 26, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
packrat-admin d443422 Commit Preview URL

Branch Preview URL
Apr 26 2026, 09:35 PM

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the monorepo to eliminate remaining no-raw-regex and no-raw-process-env custom-lint backlogs, then promotes both checks to hard gates in CI and the pre-push hook. This includes hoisting inline regex literals out of string method calls and introducing a typed Zod-based env shim for the Admin Next.js app.

Changes:

  • Hoist inline regex literals used in string method calls into named top-level constants across multiple packages/apps.
  • Introduce apps/admin/lib/env.ts (Zod-parsed env shim) and migrate Admin callers off raw process.env.
  • Harden enforcement: add new checks to pre-push and remove continue-on-error for lint:custom in CI; expand no-raw-process-env allowlist for legitimate script/test usages.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/web-ui/src/components/chart.tsx Hoists colon-stripping regex into a constant for chart ID normalization.
packages/mcp/src/tools/trips.ts Hoists UUID hyphen-stripping regex into a constant.
packages/mcp/src/tools/trail-conditions.ts Hoists UUID hyphen-stripping regex into a constant.
packages/mcp/src/tools/packs.ts Hoists UUID hyphen-stripping regex into a constant for pack/item IDs.
packages/env/scripts/no-raw-process-env.ts Expands allowlist for legitimate process.env usage sites and test directories.
packages/api/test/setup.ts Hoists markdown extension regex used in replace into a constant.
packages/api/src/utils/format-ai-response.ts Hoists formatting regex literals into top-level constants.
packages/api/src/utils/csv-utils.ts Hoists many sanitization regexes into constants and updates string normalization pipeline.
packages/api/src/services/r2-bucket.ts Hoists ETag quote-stripping regex into a constant.
packages/api/src/services/executeSqlAiTool.ts Hoists join keyword regex into a constant for complexity validation.
packages/api/src/services/embeddingService.ts Hoists newline replacement regex into a constant for embedding normalization.
packages/api/src/routes/wildlife/index.ts Hoists slug normalization regexes into constants.
packages/api/src/routes/trailConditions/reports.ts Hoists LIKE-escape regexes into constants for query input escaping.
packages/api/src/routes/packTemplates/index.ts Hoists UUID hyphen-stripping regex into a constant for template/item IDs.
packages/api/src/routes/knowledgeBase/reader.ts Hoists HTML→Markdown conversion regexes into constants.
packages/analytics/src/core/entity-resolver.ts Hoists normalization regexes into constants and simplifies brand normalization.
packages/analytics/src/core/data-export.ts Hoists timestamp filename-sanitization regex into a constant.
lefthook.yml Adds no-raw-regex + no-raw-process-env to the pre-push hard gate list.
apps/guides/scripts/enhance-content.ts Hoists timestamp regex into a constant and replaces filename filtering regex with includes.
apps/expo/utils/format-ai-response.ts Hoists formatting regex literals into top-level constants.
apps/admin/lib/env.ts Adds Admin env shim: Zod-parse process.env once and export typed env.
apps/admin/lib/api.ts Switches API base URL sourcing from raw process.env to adminEnv.
apps/admin/app/login/page.tsx Switches API base URL sourcing from raw process.env to adminEnv.
.github/workflows/checks.yml Makes lint:custom a hard CI gate (removes continue-on-error).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +7 to +9
// ── Script regex constants ──
const TIMESTAMP_SAFE_CHARS = /[:.]/g;

Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIMESTAMP_SAFE_CHARS is named as if it matches characters that are safe to keep, but the pattern /[:.]/g matches characters that are typically unsafe in filenames and are being replaced. Renaming this to something like TIMESTAMP_UNSAFE_CHARS (or similar) would better reflect its purpose and align with the naming used elsewhere (e.g., TIMESTAMP_UNSAFE_CHARS in analytics).

Copilot uses AI. Check for mistakes.
Comment on lines 250 to +253
// Normalize smart/special quotes to standard quotes
.replace(/[‘’‛‹›]/g, "'")
.replace(/[“”„‟«»]/g, '"')
.replace(/[`]/g, '')
.replace(CURLY_SINGLE_QUOTES, '’')
.replace(CURLY_DOUBLE_QUOTES, '')
.replace(BACKTICK_CHARS, '')
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

normalizeJsonString is supposed to normalize smart/special quotes to standard quotes, but it currently replaces them with curly right quotes (/). That will break downstream JSON.parse (and also conflicts with the existing csv-utils tests that expect straight quotes). Additionally, the CURLY_SINGLE_QUOTES / CURLY_DOUBLE_QUOTES character classes look incorrect (they include straight quotes instead of the actual curly quote codepoints), so many smart quotes won't be normalized at all. Suggest changing the replacements to straight quotes (' and ") and updating the regex character classes to match the intended curly quote characters (e.g. ‘ ’ “ ” etc.).

Copilot uses AI. Check for mistakes.
@andrew-bierman andrew-bierman force-pushed the feat/checks-backlog-2 branch from 20539af to a0f05f6 Compare April 26, 2026 21:03
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 26, 2026

Coverage Report for API Unit Tests Coverage (./packages/api)

Status Category Percentage Covered / Total
🔵 Lines 75.77% 610 / 805
🔵 Statements 75.77% (🎯 65%) 610 / 805
🔵 Functions 95.91% 47 / 49
🔵 Branches 88.23% 270 / 306
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/api/src/services/embeddingService.ts 100% 100% 100% 100%
packages/api/src/utils/csv-utils.ts 95.43% 87.5% 100% 95.43% 66-67, 69-70, 88-89, 132-133, 149-150, 171-172
packages/api/src/utils/format-ai-response.ts 100% 100% 100% 100%
Generated in workflow #833 for commit d443422 by the Vitest Coverage Report Action

Adds 5 entries to the allowlist (the check script itself, api startup
validator, analytics env module, guides sync script, api test directory)
and introduces apps/admin/lib/env.ts so NEXT_PUBLIC_API_URL is typed
via Zod rather than read raw.
format-ai-response (expo + api), csv-utils, enhance-content,
chart.tsx, mcp tools, api test setup — 40 patterns extracted.
Regex now compiles once and is self-documenting at the call site.
reader.ts (29 HTML→Markdown patterns), trailConditions, wildlife,
packTemplates, r2-bucket, embeddingService, executeSqlAiTool,
entity-resolver, data-export — 44 patterns extracted.
no-raw-regex now passes clean (0 violations).
Both checks now pass clean — move them from continue-on-error CI
and add them to the pre-push gate. Only check-type-casts remains
in the backlog (130 casts, next PR).
- Rename TIMESTAMP_SAFE_CHARS → TIMESTAMP_UNSAFE_CHARS (matches its purpose)
- Fix CURLY_SINGLE_QUOTES: add U+2018/U+2019 which were missing
- Fix CURLY_DOUBLE_QUOTES: add U+201C/U+201D which were missing
- Fix normalizeJsonString: replace with straight quotes (was replacing with
  smart quotes U+2019/U+201D, which would still break JSON.parse)
@andrew-bierman andrew-bierman force-pushed the feat/checks-backlog-2 branch from 2fe85f1 to d443422 Compare April 26, 2026 21:33
Base automatically changed from feat/checks-backlog to development April 26, 2026 21:46
@andrew-bierman andrew-bierman merged commit 9a34a54 into development Apr 26, 2026
10 checks passed
@andrew-bierman andrew-bierman deleted the feat/checks-backlog-2 branch April 26, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants