Skip to content

Improve sitemap & remove AI crawler blocking #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
damianlegawiec merged 18 commits into from
Apr 1, 2026
Merged

Improve sitemap & remove AI crawler blocking #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
1c5d73e
init: Sitemap & Robot files
Cichorek Feb 13, 2026
13a9907
FIX sitemap & robots: remove invalid revalidate exports, add URL fall…
Cichorek Feb 13, 2026
6e1045f
FIX robots.txt sitemap URL to match generateSitemaps output
Cichorek Feb 13, 2026
7d4e66c
FIX sitemap: defensive lastModified dates and validate locale mode
Cichorek Feb 13, 2026
32ea4b2
FIX sitemap: guard image URL filter against both null and undefined
Cichorek Feb 16, 2026
608e5ae
FIX coderabbits comments
Cichorek Feb 18, 2026
4f5f032
Apply suggestion from @damianlegawiec
Cichorek Feb 20, 2026
3dd285f
Allow ai robots by default
Cichorek Feb 20, 2026
1e47124
Update README.md
Cichorek Feb 20, 2026
8f2e727
Improve sitemap robustness and fix README markdown issues
Cichorek Feb 20, 2026
60c1793
MOD del unsupported variable
Cichorek Feb 23, 2026
9660e11
Update sitemap and robots to work with new SDK and category routes
Cichorek Mar 11, 2026
b9e8649
Replace env-based sitemap locale config with API-driven market discovery
Cichorek Mar 26, 2026
f615dbb
Fix TypeScript errors for expanded images on Product type
Cichorek Mar 27, 2026
4f283ee
Update sitemap to use Media type from new SDK and sync lockfile
Cichorek Mar 27, 2026
d92850a
Remove AI crawler blocking from robots.txt
Cichorek Mar 30, 2026
dfec569
Fix sitemap TypeScript errors after SDK upgrade
Cichorek Mar 30, 2026
6ba10cd
Remove stale SITEMAP_LOCALE_MODE/SITEMAP_COUNTRIES from README
Cichorek Mar 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,6 @@ SENTRY_AUTH_TOKEN=your-auth-token
# Set to "true" only if you have user consent or a privacy policy covering this.
SENTRY_SEND_DEFAULT_PII=false
NEXT_PUBLIC_SENTRY_SEND_DEFAULT_PII=false

# Public site URL (used as fallback for sitemap/robots if store.url is not set)
# NEXT_PUBLIC_SITE_URL=https://your-store.com
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ SPREE_PUBLISHABLE_KEY=your_publishable_api_key_here

| Variable | Description | Default |
|----------|-------------|---------|
| `NEXT_PUBLIC_DEFAULT_COUNTRY` | Default country ISO code, used for initial redirects before API data loads | `us` |
| `NEXT_PUBLIC_DEFAULT_LOCALE` | Default locale code | `en` |
| `GTM_ID` | Google Tag Manager container ID (e.g. `GTM-XXXXXXX`) | _(disabled)_ |
| `SENTRY_DSN` | Sentry DSN for error tracking (e.g. `https://key@o0.ingest.sentry.io/0`) | _(disabled)_ |
| `SENTRY_ORG` | Sentry organization slug (for source map uploads) | _(none)_ |
Expand Down
Loading
Loading