Skip to content

feat(marketing): add six local tools and update build tooling - #281

Merged
abhineetjain13 merged 5 commits into
mainfrom
feat/marketing-free-tools
Oct 6, 2026
Merged

abhineetjain13 merged 5 commits into
mainfrom
feat/marketing-free-tools

Conversation

@abhineetjain13

@abhineetjain13 abhineetjain13 commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Adds a public /tools hub and six browser-only utilities for crawler rules, robots.txt generation, page directives, structured data, sitemap comparison, and social previews. Inputs stay local, reports state their limits, and crawler tools reuse the existing public catalog and robots parser.

Adds navigation and sitemap discovery, bounded inputs, safe exports and workflow coverage. Pending sitemap uploads cannot overwrite a loaded example; parser boundaries preserve crawler scope and reject malformed sitemap namespaces. Docker packaging includes the crawler catalog. This is additive; no existing behavior is retired.

Aligns root/frontend manifests, the lockfile and all Docker pnpm install stages on 12.9.1, and includes the project Sonar editor connection. A one-line pnpm patch makes Astro 7.3.5's Windows cleanup fallback compatible with Node 26. The fix is still an open upstream PR, so the documented removal condition is upgrading to a release that includes it. Every Docker dependency stage receives the patch for frozen installs.

Disposable bootstrap/seed test databases close their pool before normal deletion and always close the administrative connection. Forced database deletion previously raced a closing PostgreSQL socket and failed an otherwise-passing API suite with an unhandled error; test assertions and gates remain intact.

Testing

  • pnpm --dir frontend install --frozen-lockfile --ignore-scripts — exit 0 with pnpm 12.9.1.
  • .\scripts\check.ps1 -All — exit 0: backend, frontend, contract and API static checks, plus all production builds. Marketing uses its normal output directory after stale local dev processes were stopped.
  • docker build --target native --file Dockerfile --tag citeladder-native-build-check:local . — exit 0: migration-image native stage, including the frozen production install and runtime deploy, receives the patch and uses pnpm 12.9.1.
  • From frontend/, pnpm exec vp test run --config ../.git/ship-tool-tests.config.mts components/marketing/tools/tools.test.tsx lib/free-tools/tools.test.ts — exit 0, 14 tests passed. The temporary config extends the checked-in Vite config with envDir: false; inherited provider credentials were removed from the test process.
  • pnpm --dir frontend/services/api test test/dev-seed.test.ts test/bootstrap-sequence.test.ts — exit 0, 8 tests passed against a newly created, isolated PostgreSQL 16 container. Dotenv was disabled and provider credentials cleared; the container was removed afterward.
  • From frontend/, pnpm exec vp check --fix services/api/test/disposable-database.ts and pnpm --filter @citeladder/api typecheck — exit 0 for the fixture follow-up.
  • git diff --check and git diff --cached --check — exit 0. Commit hooks run staged formatting/lint.

CI owns the full native suites, E2E and clean-clone Compose smoke; these are not reproduced locally. No live providers or deployments were invoked.

SonarCloud S5332 on the standard sitemap XML namespace is classified FALSE_POSITIVE in SonarCloud: this identifier is compared as text and never fetched. Its required HTTP namespace remains intact; no source suppression or weakened quality gate was added.

@codeant-ai

codeant-ai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Reviewed your PR 23ebd20 Oct 06, 2026 · 11:01 11:05

@codeant-ai

codeant-ai Bot commented Oct 6, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@abhineetjain13

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@codeant-ai codeant-ai Bot added the size:XXL This PR changes 1000+ lines, ignoring generated files label Oct 6, 2026
@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5c449bb2-def7-487e-9989-21d2751114ce
📥 Commits

Reviewing files that changed from the base of the PR and between 9eb6763 and 23ebd20.

⛔ Files ignored due to path filters (1)
  • frontend/pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (23)
  • docs/frontend-architecture.md
  • frontend/apps/marketing/src/pages/sitemap.xml.ts
  • frontend/apps/marketing/src/pages/tools/[slug].astro
  • frontend/apps/marketing/src/pages/tools/index.astro
  • frontend/components/marketing/chrome/footer.tsx
  • frontend/components/marketing/tools/markup-tools.tsx
  • frontend/components/marketing/tools/robots-tools.tsx
  • frontend/components/marketing/tools/sitemap-tool.tsx
  • frontend/components/marketing/tools/social-tool.tsx
  • frontend/components/marketing/tools/tool-form.tsx
  • frontend/components/marketing/tools/tools-index.tsx
  • frontend/components/marketing/tools/tools.test.tsx
  • frontend/e2e/marketing-pages.spec.ts
  • frontend/lib/config/free-tools.ts
  • frontend/lib/free-tools/generators.ts
  • frontend/lib/free-tools/input.ts
  • frontend/lib/free-tools/markup.ts
  • frontend/lib/free-tools/robots.ts
  • frontend/lib/free-tools/sitemap.ts
  • frontend/lib/free-tools/tools.test.ts
  • frontend/lib/marketing-content/nav.ts
  • frontend/lib/marketing-content/tools.ts
  • frontend/package.json

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The marketing site adds a /tools hub and six browser-based utilities for crawler rules, markup, structured data, sitemaps, and social previews. The tools use bounded pasted inputs and local files. Their pages, navigation links, sitemap entries, and tests are also added.

Changes

Free Tools

Layer / File(s) Summary
Tool catalog and shared input handling
frontend/lib/marketing-content/tools.ts, frontend/lib/config/free-tools.ts, frontend/lib/free-tools/input.ts, frontend/components/marketing/tools/tool-form.tsx
Adds metadata for six tools, shared input limits and validation helpers, and form components that manage results, errors, copying, and downloads.
Robots rule checking and generation
frontend/lib/free-tools/robots.ts, frontend/components/marketing/tools/robots-tools.tsx, frontend/package.json, frontend/lib/free-tools/tools.test.ts
Adds crawler-rule evaluation and robots.txt generation, their browser interfaces, and tests for parsing, rule precedence, and generation.
Markup inspection and tag generation
frontend/lib/free-tools/markup.ts, frontend/lib/free-tools/generators.ts, frontend/components/marketing/tools/markup-tools.tsx, frontend/lib/free-tools/tools.test.ts
Adds pasted-markup inspection, JSON-LD and social-tag generation, corresponding interfaces, and validation and escaping tests.
Sitemap parsing and comparison
frontend/lib/free-tools/sitemap.ts, frontend/components/marketing/tools/sitemap-tool.tsx, frontend/lib/free-tools/tools.test.ts
Adds sitemap XML validation and comparison, file input handling, and tests for comparison results and invalid input.
Tool interfaces and result handling
frontend/components/marketing/tools/robots-tools.tsx, frontend/components/marketing/tools/markup-tools.tsx, frontend/components/marketing/tools/sitemap-tool.tsx, frontend/components/marketing/tools/social-tool.tsx, frontend/components/marketing/tools/tools.test.tsx
Connects tool inputs and results to browser components. Adds a local image preview for social tags and workflow tests for all six tools.
Tools pages and site discovery
frontend/apps/marketing/src/pages/tools/*, frontend/components/marketing/tools/tools-index.tsx, frontend/lib/marketing-content/nav.ts, frontend/components/marketing/chrome/footer.tsx, frontend/apps/marketing/src/pages/sitemap.xml.ts, docs/frontend-architecture.md, frontend/e2e/marketing-pages.spec.ts
Adds the tools hub and slug pages, navigation links, sitemap routes, architecture documentation, and end-to-end coverage for the hub, checker, and unknown-tool 404.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  actor User
  participant CrawlerChecker
  participant ToolForm
  participant testRobots
  User->>CrawlerChecker: Enter robots.txt and page URL
  CrawlerChecker->>ToolForm: Submit supplied inputs
  ToolForm->>testRobots: Check rules for crawler tokens
  testRobots-->>ToolForm: Return statuses and matching-line evidence
  ToolForm-->>User: Display the current result
Loading

Merge Risk: ⚪ Minimal · up to 23ebd

The generator requires a page URL to test its rules before offering a download. No identified issue needs resolution before merge.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 24 functions across 19 files. (4 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: adding six local marketing tools. The build-tooling reference is secondary and does not obscure the feature.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 24 functions across 19 files. (4 skipped: 4 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codeant-ai

codeant-ai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

🏁 CodeAnt Quality Gate Results

Commit: d028fe6d
Scan Time: 2026-10-06 11:53:06 UTC

✅ Overall Status: PASSED

Quality Gate Details

Quality Gate Status Details
Secrets ✅ PASSED 0 secrets found, 1 false positive secret suppressed
Duplicate Code ✅ PASSED 0.0% duplicated
SAST ✅ PASSED No security issues
Bugs ✅ PASSED Rating B: 1 bugs (1 medium)
IAC ✅ PASSED Rating B: 1 issues (1 medium)

View Full Results

@codeant-ai

codeant-ai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

CodeAnt Nitpicks

5 code suggestions

1. A compressed image under this byte limit can still have enormous pixel dimensions; rendering it can use hundreds of megabytes and freeze or crash the browser.

Performance · frontend/components/marketing/tools/social-tool.tsx:99-102


2. Valid crawler-specific names such as googlebot-news and googlebot-image are ignored, so the report can miss supplied indexing restrictions.

Incomplete implementation · frontend/lib/free-tools/markup.ts:27


3. A valid comments-only robots.txt has no User-agent line, so this check rejects it instead of reporting the default permission.

Incorrect condition logic · frontend/lib/free-tools/robots.ts:11


4. The raw-text check matches harmless XML comments containing <!DOCTYPE or <!ENTITY, so it rejects valid sitemap files despite having no declarations.

Incorrect condition logic · frontend/lib/free-tools/sitemap.ts:6


5. A <url> or <sitemap> element in the wrong namespace is silently skipped, so declared URLs disappear and comparisons return incomplete results instead of rejecting malformed input.

Incorrect condition logic · frontend/lib/free-tools/sitemap.ts:18-20


1 custom suggestion

1. Build the displayed limits from FREE_TOOL_LIMITS so the hint stays aligned with the owning frontend configuration.

Custom_rule · frontend/components/marketing/tools/sitemap-tool.tsx:23

Comment on lines +72 to +75
sample={() => {
setBefore(xml(['', 'old-guide']));
setAfter(xml(['', 'new-guide']));
}}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: If a file read is pending when the example is loaded, its callback can finish later and replace the example with the older file contents.

Assessment: 🟠 Major · 🔁 Occurrence: Rarely · 🏷️ Race condition

Use CodeAnt Skill Fix in Cursor Fix in VSCode Claude

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** frontend/components/marketing/tools/sitemap-tool.tsx
**Line:** 72:75
**Comment:**
	*Race Condition: If a file read is pending when the example is loaded, its callback can finish later and replace the example with the older file contents.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

@codeant-ai

codeant-ai Bot commented Oct 6, 2026

Copy link
Copy Markdown

CodeAnt PR Risk: Medium Risk

  • The PR needs attention before merging: loading an example does not invalidate pending sitemap file reads, so a late read can replace sample input and skew the comparison.

Assessed commit: 23ebd208ad51

@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@abhineetjain13 abhineetjain13 changed the title feat(marketing): add six browser-only website tools feat(marketing): add six local tools and update build tooling Oct 6, 2026
@sonarqubecloud

sonarqubecloud Bot commented Oct 6, 2026

Copy link
Copy Markdown

@abhineetjain13
abhineetjain13 merged commit d3fa4ba into main Oct 6, 2026
20 checks passed
@abhineetjain13
abhineetjain13 deleted the feat/marketing-free-tools branch October 6, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL This PR changes 1000+ lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant