diff --git a/.claude/commands/review-translations.md b/.claude/commands/review-translations.md index 8c93f2ce809f..4b798a183ae2 100644 --- a/.claude/commands/review-translations.md +++ b/.claude/commands/review-translations.md @@ -240,7 +240,7 @@ curl -sf "$GLOSSARY_HOST/llms.txt" \ ENGLISH_SOURCE=$(cat "$WORKTREE_PATH/public/content/{path}.md") curl -sf -X POST "$GLOSSARY_API_URL/filter" \ -H "Content-Type: application/json" \ - -d "$(jq -n --arg text "$ENGLISH_SOURCE" --arg lang "{LANGUAGE_CODE}" '{text: $text, language: $lang}')" + -d "$(jq -n --arg content "$ENGLISH_SOURCE" --arg lang "{LANGUAGE_CODE}" '{content: $content, language: $lang}')" ``` **Fallback — full language** when filtering per file is impractical or the endpoint is unreachable: diff --git a/.claude/commands/update-llms-txt.md b/.claude/commands/update-llms-txt.md deleted file mode 100644 index 854340da82e0..000000000000 --- a/.claude/commands/update-llms-txt.md +++ /dev/null @@ -1,87 +0,0 @@ -# Update LLMS.txt Command - -This command helps maintain the `public/llms.txt` file by monitoring key navigation files: - -1. **Main Navigation**: `src/components/Nav/useNavigation.ts` -2. **Developer Docs**: `src/data/developer-docs-links.yaml` -3. **Footer Links**: `src/components/Footer.tsx` - -## How it works - -- Adds missing links to appropriate sections -- Preserves existing descriptions and organization -- Follows established llms.txt structure -- **Prefers static markdown files URLs over html URLs** for better LLM comprehension - -## Implementation - -When this command is executed, I will: - -### Step 1: Parse Navigation Files - -**Main Navigation** (`src/components/Nav/useNavigation.ts`): - -```javascript -// Extract linkSections object structure -// Parse learn, use, build, participate sections -// Get href, label, and description for each link -``` - -**Developer Docs** (`src/data/developer-docs-links.yaml`): - -```yaml -# Parse foundational-topics, ethereum-stack, advanced, design-fundamentals -# Extract href and id mappings -# Build hierarchical structure -``` - -**Footer Links** (`src/components/Footer.tsx`): - -```javascript -// Extract linkSections and dipperLinks arrays -// Get all footer navigation items -// Include external links (blog, ESP, Devcon) -``` - -### Step 2: Analyze Current llms.txt - -- Parse existing sections and their links -- Extract current URLs and descriptions -- Identify section organization and hierarchy - -### Step 3: URL to Markdown File Mapping - -**Priority: Static markdown files URLs over web html URLs** - -For each link, I will: - -1. Check if corresponding markdown file exists in `public/content/`. **Ignore translations**: Skip `public/content/translations/` directory (60+ language versions) -2. Use a URL pointing to the markdown file for the page: `https://ethereum.org/content/[page]/index.md` -3. Fall back to web URL only if no markdown file exists -4. Example: `https://ethereum.org/learn/` → `https://ethereum.org/content/learn/index.md` -5. Example2: `https://ethereum.org/guides/how-to-use-a-wallet/` → `https://ethereum.org/content/guides/how-to-use-a-wallet/index.md` - -### Step 4: Smart Link Categorization - -New links are categorized using these rules: - -1. **Learn Section**: `/learn/`, `/what-is-*`, `/guides/`, `/quizzes/`, `/glossary/` -2. **Use Section**: `/get-eth`, `/wallets/`, `/dapps/`, `/staking/`, use cases -3. **Build Section**: `/developers/`, `/enterprise/`, developer tools -4. **Participate Section**: `/community/`, `/contributing/`, `/foundation/` -5. **Research Section**: `/whitepaper`, `/roadmap/`, `/eips/`, `/governance/` - -### Step 5: Validation & Quality Checks - -- Verify all markdown files exist in `public/content/` -- Check for duplicate links within sections -- Validate section organization and hierarchy -- Ensure descriptions are informative and concise - -### Step 6: Execute Action - -Update llms.txt file with improved structure and validated links - ---- - -The command ensures the llms.txt file remains comprehensive and current with minimal manual maintenance. diff --git a/.claude/skills/design-system/SKILL.md b/.claude/skills/design-system/SKILL.md index d394eab3d2e4..2cc82195a474 100644 --- a/.claude/skills/design-system/SKILL.md +++ b/.claude/skills/design-system/SKILL.md @@ -30,7 +30,7 @@ When the existing primitive doesn't quite fit, the answer is usually "add a vari 6. **Logical CSS for direction.** Use `ms-`/`me-`/`ps-`/`pe-`/`inset-s-`/`inset-e-`/`border-s`/`border-e`/`text-start`/`text-end`. The site supports Arabic and Urdu (RTL). Hard-coded `left-`/`right-`/`ml-`/`mr-`/`pl-`/`pr-` breaks RTL. 7. **Locale-aware formatters.** `numberFormat()` from `@/lib/utils/numbers`, `dateTimeFormat()` from `@/lib/utils/date`. Never `toLocaleString` / `Intl.NumberFormat` directly. 8. **`useRtlFlip()` for directional icons** (right-pointing arrows/chevrons). Or use `ChevronNext`/`ChevronPrev` from `@/components/Chevron`. -9. **Markdown content goes through `MdComponents`.** The legacy `@/components/Card` (default export) is reserved for markdown shortcodes -- never import it from app code; use `@/components/ui/card`. +9. **Markdown content goes through `MdComponents`.** The `` markdown shortcode is backed by `@/components/MarkdownCard` (a thin wrapper around the `ui/card` primitives with an MDX-friendly prop shape). For app code, compose the primitives directly from `@/components/ui/card`. 10. **Storybook stories ship with new UI components.** No automated unit tests; Storybook + Chromatic + types are the verification layer. 11. **Don't add new layouts.** There are six canonical layouts (`TopicLayout`, `StaticLayout`, `DocsLayout`, `TutorialLayout`, `ContentLayout`, `BaseLayout`). New sectioned content goes in `src/data/topics/.ts` as a `TopicLayout` config -- not a new layout component. See `references/layouts.md`. @@ -40,7 +40,7 @@ These are landmines where the code looks reasonable but the pattern is wrong. Th ### Imports that look right but aren't -- **Cards**: `import { Card } from "@/components/ui/card"` is canonical. **Not** `import Card from "@/components/Card"` (default export of that file is reserved for markdown shortcodes). +- **Cards**: `import { Card } from "@/components/ui/card"` is canonical for app code. The `` markdown shortcode is backed by `@/components/MarkdownCard` — that wrapper is rarely imported from app code, since composing the `ui/card` parts directly is more flexible. - **Tooltips**: `import Tooltip from "@/components/Tooltip"` (mobile-aware, Matomo-tracked, scroll-close). **Not** `import { Tooltip } from "@/components/ui/tooltip"` (that's the bare Radix primitive used internally). - **Modals**: `import Modal from "@/components/ui/dialog-modal"` (default export, the high-level convenience) for typical modal needs. `@/components/ui/dialog` is the vanilla shadcn-style primitive for fine-grained Radix control. Same names exported from both files; **do not mix sources within a feature**. - **Heroes**: import from `@/components/Hero` (`ContentHero`, `SimpleHero`, `HubHero`, `MdxHero`, `HomeHero`). **Not** `@/components/PageHero` (deprecation track). @@ -56,6 +56,7 @@ Used in 5 places. Don't introduce new uses. Use Tailwind `dark:` variant + seman ### Subtle component behaviors - ` + + +
+
+
- - +

+ {t(`page-developers-tools-category-${slug}-title`)} +

+ + {t("page-developers-tools-see-all")} + +
+ {previewsByCategory[slug].map((app) => ( }) => { }) => ( -

{name}

- -
+ +

{name}

+ + + {!!tags.length && (
{tags.map((tag) => ( @@ -430,10 +432,12 @@ const Page = async (props: { params: Promise }) => {

{highlight}

))} -
- - {ctaLabel || categoryCtaLabel} - + + + + {ctaLabel || categoryCtaLabel} + + ) )} diff --git a/app/[locale]/gas/page.tsx b/app/[locale]/gas/page.tsx index 485c687aa171..ec2e207f1aed 100644 --- a/app/[locale]/gas/page.tsx +++ b/app/[locale]/gas/page.tsx @@ -1,4 +1,4 @@ -import { type BaseHTMLAttributes, type ComponentPropsWithRef } from "react" +import { type BaseHTMLAttributes } from "react" import { pick } from "lodash" import { getMessages, @@ -8,8 +8,6 @@ import { import type { Lang, PageParams } from "@/lib/types" -import Callout from "@/components/Callout" -import Card from "@/components/Card" import Emoji from "@/components/Emoji" import ExpandableCard from "@/components/ExpandableCard" import FeedbackCard from "@/components/FeedbackCard" @@ -19,11 +17,13 @@ import HorizontalCard from "@/components/HorizontalCard" import I18nProvider from "@/components/I18nProvider" import { Image } from "@/components/Image" import MainArticle from "@/components/MainArticle" +import MarkdownCard from "@/components/MarkdownCard" import PageHero from "@/components/PageHero" import { StandaloneQuizWidget } from "@/components/Quiz/QuizWidget" import Translation from "@/components/Translation" import { Alert, AlertContent, AlertTitle } from "@/components/ui/alert" import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" import { Divider } from "@/components/ui/divider" import { Flex, type FlexProps } from "@/components/ui/flex" import InlineLink, { BaseLink } from "@/components/ui/Link" @@ -68,13 +68,6 @@ const PageContainer = ({ className, ...props }: FlexProps) => ( ) -const StyledCard = (props: ComponentPropsWithRef) => ( - -) - const H2 = ({ className, ...props @@ -205,32 +198,30 @@ const Page = async (props: { params: Promise }) => { {t("page-gas-how-do-i-pay-less-gas-header")}

{t("page-gas-how-do-i-pay-less-gas-text")}

- - + - - - - {t("page-gas-try-layer-2")} - - + ctaLabel={t("page-gas-try-layer-2")} + href="/layer-2/" + /> @@ -398,38 +389,30 @@ const Page = async (props: { params: Promise }) => { - +
-
- - {t("page-gas-use-layer-2")} - -
+ + {t("page-gas-use-layer-2")} + -
- - {tCommunity("page-community-explore-dapps")} - -
+ + {tCommunity("page-community-explore-dapps")} + - +
diff --git a/app/[locale]/get-eth/page.tsx b/app/[locale]/get-eth/page.tsx index 6264775abeb3..6c9e0146a5cd 100644 --- a/app/[locale]/get-eth/page.tsx +++ b/app/[locale]/get-eth/page.tsx @@ -8,7 +8,6 @@ import type { ReactNode } from "react" import type { ChildOnlyProp, Lang, PageParams } from "@/lib/types" -import CalloutBanner from "@/components/CalloutBanner" import CardList, { type CardProps as CardListCardProps, } from "@/components/CardList" @@ -22,11 +21,14 @@ import MainArticle from "@/components/MainArticle" import Translation from "@/components/Translation" import { Alert, AlertContent, AlertDescription } from "@/components/ui/alert" import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" import { Card, CardContent, + CardEmoji, CardFooter, CardHeader, + CardParagraph, CardTitle, } from "@/components/ui/card" import { Divider } from "@/components/ui/divider" @@ -58,13 +60,13 @@ type CardProps = { } const StyledCard = ({ children, emoji, title, description }: CardProps) => ( - - - - {title} + + + - -

{description}

+ + {title} + {description} {children} @@ -365,20 +367,16 @@ export default async function Page(props: { params: Promise }) { - -
- - {t("page-get-eth-checkout-dapps-btn")} - -
- + + {t("page-get-eth-checkout-dapps-btn")} + + = { + "&": "&", + "<": "<", + ">": ">", + '"': """, + "'": "'", +} + +const escapeXml = (value: string): string => + value.replace(/[&<>"']/g, (c) => XML_ESCAPE[c]) + +export async function GET( + _: NextRequest, + { params }: { params: Promise<{ locale: string }> } +) { + const { locale } = await params + + const t = await getTranslations({ + locale, + namespace: "page-latest", + }) + + const posts = await getBlogPostsData(locale) + + // Strip trailing slash for the .xml file URL (getFullUrl appends one for + // directory-style URLs, but feed.xml is a file). + const feedUrl = getFullUrl(locale, "/latest/feed.xml").replace(/\/$/, "") + const channelLink = getFullUrl(locale, "/latest/") + const channelTitle = t("page-latest-title") + const channelDescription = t("page-latest-subtitle") + + const items = posts + .map((post) => { + const link = getFullUrl(locale, post.href) + const pubDate = new Date(post.published).toUTCString() + const creator = post.author + ? `${escapeXml(post.author)}` + : "" + const categories = (post.tags ?? []) + .map((tag) => `${escapeXml(tag)}`) + .join("") + return [ + "", + `${escapeXml(post.title)}`, + `${link}`, + `${link}`, + `${escapeXml(post.description)}`, + creator, + `${pubDate}`, + categories, + "", + ].join("") + }) + .join("") + + const lastBuildDate = new Date().toUTCString() + const xml = [ + '', + '', + "", + `${escapeXml(channelTitle)}`, + `${channelLink}`, + ``, + `${escapeXml(channelDescription)}`, + `${locale}`, + `${lastBuildDate}`, + items, + "", + "", + ].join("") + + return new Response(xml, { + headers: { + "Content-Type": "application/rss+xml; charset=utf-8", + }, + }) +} diff --git a/app/[locale]/latest/page-jsonld.tsx b/app/[locale]/latest/page-jsonld.tsx new file mode 100644 index 000000000000..52252b4fbc7a --- /dev/null +++ b/app/[locale]/latest/page-jsonld.tsx @@ -0,0 +1,82 @@ +import type { BlogPost, FileContributor } from "@/lib/types" + +import PageJsonLD from "@/components/PageJsonLD" + +import { normalizeUrlForJsonLd } from "@/lib/utils/url" + +import { BASE_GRAPH_NODES } from "@/lib/jsonld/constants" +import { REFERENCE } from "@/lib/jsonld/references" + +export default async function BlogPageJsonLD({ + locale, + blogPosts, + contributors, +}: { + locale: string + blogPosts: BlogPost[] + contributors: FileContributor[] +}) { + const url = normalizeUrlForJsonLd(locale, "/latest") + + const contributorList = contributors.map((contributor) => ({ + "@type": "Person", + name: contributor.login, + url: contributor.html_url, + })) + + const blogPostItems = blogPosts.map((post, index) => ({ + "@type": "ListItem", + position: index + 1, + url: normalizeUrlForJsonLd(locale, post.href), + name: post.title, + })) + + const jsonLd = { + "@context": "https://schema.org", + "@graph": [ + ...BASE_GRAPH_NODES, + { + "@type": "CollectionPage", + "@id": url, + name: "Builder updates", + description: + "Builder resources, tools, and developments from the Ethereum ecosystem.", + url, + inLanguage: locale, + isPartOf: REFERENCE.ETHEREUM_ORG_WEBSITE, + publisher: REFERENCE.ETHEREUM_FOUNDATION, + contributor: contributorList, + breadcrumb: { + "@type": "BreadcrumbList", + itemListElement: [ + { + "@type": "ListItem", + position: 1, + name: "Home", + item: normalizeUrlForJsonLd(locale, "/"), + }, + { + "@type": "ListItem", + position: 2, + name: "Developers", + item: normalizeUrlForJsonLd(locale, "/developers"), + }, + { + "@type": "ListItem", + position: 3, + name: "Builder updates", + item: url, + }, + ], + }, + mainEntity: { + "@type": "ItemList", + numberOfItems: blogPosts.length, + itemListElement: blogPostItems, + }, + }, + ], + } + + return +} diff --git a/app/[locale]/latest/page.tsx b/app/[locale]/latest/page.tsx new file mode 100644 index 000000000000..7bd7af20f815 --- /dev/null +++ b/app/[locale]/latest/page.tsx @@ -0,0 +1,163 @@ +import { pick } from "lodash" +import { + getMessages, + getTranslations, + setRequestLocale, +} from "next-intl/server" + +import type { Lang, PageParams } from "@/lib/types" + +import Emoji from "@/components/Emoji" +import FeedbackCard from "@/components/FeedbackCard" +import ContentHero from "@/components/Hero/ContentHero" +import I18nProvider from "@/components/I18nProvider" +import MainArticle from "@/components/MainArticle" +import { BaseLink } from "@/components/ui/Link" +import { Tag } from "@/components/ui/tag" + +import { getAppPageContributorInfo } from "@/lib/utils/contributors" +import { getBlogPostsData } from "@/lib/utils/md" +import { getMetadata } from "@/lib/utils/metadata" +import { getLocaleTimestamp } from "@/lib/utils/time" +import { getRequiredNamespacesForPage } from "@/lib/utils/translations" +import { getFullUrl } from "@/lib/utils/url" + +import BlogPageJsonLD from "./page-jsonld" + +import heroImg from "@/public/images/developers/blog/latest-landing-hero.png" + +const publishedDate = (locale: Lang, published: string) => { + const localeTimestamp = getLocaleTimestamp(locale, published) + + return localeTimestamp !== "Invalid Date" ? ( + + + {localeTimestamp} + + ) : null +} + +const Page = async (props: { params: Promise }) => { + const params = await props.params + const { locale } = params + + setRequestLocale(locale) + + const t = await getTranslations("page-latest") + + // Get i18n messages + const allMessages = await getMessages({ locale }) + const requiredNamespaces = getRequiredNamespacesForPage("/latest") + const messages = pick(allMessages, requiredNamespaces) + + const blogPosts = await getBlogPostsData(locale) + + const { contributors } = await getAppPageContributorInfo( + "latest", + locale as Lang + ) + + return ( + <> + + + + +
+ {blogPosts.length === 0 ? ( +

+ {t("page-latest-no-posts")} +

+ ) : ( + blogPosts.map((post) => ( + +

+ {post.title} +

+

+ + {post.author} + {post.team ? <> • {post.team} : null} + {post.published ? ( + <> •{publishedDate(locale as Lang, post.published)} + ) : null} + {post.timeToRead ? ( + <> + {" "} + • + + {t("page-latest-minute-read", { + minutes: post.timeToRead, + })} + + ) : null} +

+

{post.description}

+
+ {post.tags?.map((tag) => ( + + {tag} + + ))} +
+ + )) + )} +
+ + + + + + ) +} + +export async function generateMetadata(props: { + params: Promise<{ locale: string }> +}) { + const params = await props.params + const { locale } = params + + const t = await getTranslations("page-latest") + + const metadata = await getMetadata({ + locale, + slug: ["latest"], + title: t("page-latest-title"), + description: t("page-latest-meta-description"), + }) + + return { + ...metadata, + alternates: { + ...(metadata.alternates ?? {}), + types: { + "application/rss+xml": getFullUrl(locale, "/latest/feed.xml").replace( + /\/$/, + "" + ), + }, + }, + } +} + +export default Page diff --git a/app/[locale]/layer-2/learn/page.tsx b/app/[locale]/layer-2/learn/page.tsx index df0ac16bd4a3..c0f46267413e 100644 --- a/app/[locale]/layer-2/learn/page.tsx +++ b/app/[locale]/layer-2/learn/page.tsx @@ -7,16 +7,16 @@ import { import type { Lang, PageParams } from "@/lib/types" -import CalloutSSR from "@/components/CalloutSSR" -import Card from "@/components/Card" import FileContributors from "@/components/FileContributors" import { ContentHero, type ContentHeroProps } from "@/components/Hero" import I18nProvider from "@/components/I18nProvider" import { Image } from "@/components/Image" import MainArticle from "@/components/MainArticle" +import MarkdownCard from "@/components/MarkdownCard" import { StandaloneQuizWidget } from "@/components/Quiz/QuizWidget" import Translation from "@/components/Translation" import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" import { getAppPageContributorInfo } from "@/lib/utils/contributors" import { getMetadata } from "@/lib/utils/metadata" @@ -219,19 +219,18 @@ const Page = async (props: { params: Promise }) => { -
-
- {layer2Cards.map((card, idx) => ( -
- -
- ))} -
+
+ {layer2Cards.map((card, idx) => ( + + ))}
}) => { />
-
-
- + + -
- - {t("page-layer-2-learn-learn-more")} - -
- - + + + -
- - {t("page-layer-2-learn-explore-networks")} - -
- -
+ {t("page-layer-2-learn-explore-networks")} + +
diff --git a/app/[locale]/layer-2/networks/page.tsx b/app/[locale]/layer-2/networks/page.tsx index 95b03a22235d..1469879020ff 100644 --- a/app/[locale]/layer-2/networks/page.tsx +++ b/app/[locale]/layer-2/networks/page.tsx @@ -7,13 +7,13 @@ import { import type { ExtendedRollup, Lang, PageParams } from "@/lib/types" -import CalloutSSR from "@/components/CalloutSSR" import { ContentHero, type ContentHeroProps } from "@/components/Hero" import I18nProvider from "@/components/I18nProvider" import Layer2NetworksTable from "@/components/Layer2NetworksTable" import MainArticle from "@/components/MainArticle" import NetworkMaturity from "@/components/NetworkMaturity" import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" import { getAppPageContributorInfo } from "@/lib/utils/contributors" import { getMetadata } from "@/lib/utils/metadata" @@ -193,44 +193,40 @@ const Page = async (props: { params: Promise }) => {
- -
- - {tCommon("learn-more")} - -
- - + {tCommon("learn-more")} + + + -
- - {tCommon("learn-more")} - -
- + + {tCommon("learn-more")} + +
diff --git a/app/[locale]/layer-2/page.tsx b/app/[locale]/layer-2/page.tsx index ed6d2d84f32b..3aff97cd4b99 100644 --- a/app/[locale]/layer-2/page.tsx +++ b/app/[locale]/layer-2/page.tsx @@ -7,15 +7,15 @@ import { import type { Lang, PageParams } from "@/lib/types" -import CalloutSSR from "@/components/CalloutSSR" -import Card from "@/components/Card" import ExpandableCard from "@/components/ExpandableCard" import HubHero, { type HubHeroProps } from "@/components/Hero/HubHero" import I18nProvider from "@/components/I18nProvider" import { Image } from "@/components/Image" import MainArticle from "@/components/MainArticle" +import MarkdownCard from "@/components/MarkdownCard" import Translation from "@/components/Translation" import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" import InlineLink from "@/components/ui/Link" import { getAppPageContributorInfo } from "@/lib/utils/contributors" @@ -107,9 +107,9 @@ const Page = async (props: { params: Promise }) => { }, { content: t("page-layer-2-hero-button-2-content"), - href: "#layer-2-powered-by-ethereum", + href: "/layer-2/learn", matomo: { - eventCategory: "l2_hub", + eventCategory: "l2_learn_page", eventAction: "button_click", eventName: "hero_get_started", }, @@ -323,19 +323,18 @@ const Page = async (props: { params: Promise }) => {
-
-
- {calloutCards.map((card, idx) => ( -
- -
- ))} -
+
+ {calloutCards.map((card, idx) => ( + + ))}
@@ -505,48 +504,42 @@ const Page = async (props: { params: Promise }) => {
-
- -
- - {tCommon("nav-networks-explore-networks-label")} - -
- - + {tCommon("nav-networks-explore-networks-label")} + + + -
- - {tCommon("learn-more")} - -
- + + {tCommon("learn-more")} + +
diff --git a/app/[locale]/layout.tsx b/app/[locale]/layout.tsx index de23551fc14b..5cd502f1d3a1 100644 --- a/app/[locale]/layout.tsx +++ b/app/[locale]/layout.tsx @@ -10,6 +10,7 @@ import Matomo from "@/components/Matomo" import { getLastDeployDate } from "@/lib/utils/getLastDeployDate" import { getLocaleTimestamp } from "@/lib/utils/time" +import { toLanguageTag } from "@/lib/utils/url" import Providers from "./providers" @@ -66,7 +67,7 @@ export default async function LocaleLayout(props: { return ( diff --git a/app/[locale]/learn/page.tsx b/app/[locale]/learn/page.tsx index 0d12aebb9368..65e776d859da 100644 --- a/app/[locale]/learn/page.tsx +++ b/app/[locale]/learn/page.tsx @@ -4,16 +4,18 @@ import { getTranslations } from "next-intl/server" import type { PageParams, ToCItem } from "@/lib/types" import type { Lang } from "@/lib/types" -import CalloutBannerSSR from "@/components/CalloutBannerSSR" import DocLink, { type DocLinkProps } from "@/components/DocLink" import { HubHero } from "@/components/Hero" import type { HubHeroProps } from "@/components/Hero/HubHero" import { Image, type ImageProps } from "@/components/Image" import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" import { Card, CardBanner, CardContent, + CardFooter, + CardHeader, CardParagraph, CardTitle, } from "@/components/ui/card" @@ -70,19 +72,23 @@ const LearnCard = ({ description: string ctaLabel: string }) => ( - - - - - - {title} - {description} + + + + + + + + {title} + {description} - {ctaLabel} + + {ctaLabel} + ) @@ -367,18 +373,18 @@ export default async function Page(props: { params: Promise }) {

{tocItems[2].title}

{t("what-is-ethereum-used-for-1")}

- - + {t("explore-use-cases-cta")} - + diff --git a/app/[locale]/reports/data.ts b/app/[locale]/reports/data.ts new file mode 100644 index 000000000000..1a177d71c866 --- /dev/null +++ b/app/[locale]/reports/data.ts @@ -0,0 +1,185 @@ +import type { StaticImageData } from "next/image" + +import a16zCover from "@/public/images/reports/a16z-state-of-crypto-2025.webp" +import bisCover from "@/public/images/reports/bis-papers-156.webp" +import quantumCover from "@/public/images/reports/coinbase-quantum-blockchain.webp" +import consensysCover from "@/public/images/reports/consensys-trustware.webp" +// import ethereumBasicsCover from "@/public/images/reports/ethereum-basics-governments-institutions.webp" +import fidelityCoinCover from "@/public/images/reports/fidelity-coin-report-ethereum.webp" +import l2LandscapeCover from "@/public/images/reports/l2-landscape.webp" +import mckinseyCover from "@/public/images/reports/mckinsey-ripples-to-waves.webp" +import openzeppelinCover from "@/public/images/reports/openzeppelin-risk-assessment.webp" +import tdsCover from "@/public/images/reports/trillion-dollar-security-card.webp" +import twinstakeCover from "@/public/images/reports/twinstake-pectra.webp" +import whiteHouseCover from "@/public/images/reports/white-house-crypto.webp" + +export type ReportCategory = + | "ef-original" + | "regulator" + | "central-bank" + | "bank-research" + | "big-four" + | "crypto-native" + | "academic" + +export type Report = { + /** Stable slug used for keys and (when internal) the subpage path */ + slug: string + /** Full title as published */ + title: string + /** Publisher (shown as paragraph text under the title) */ + publisher: string + /** + * Publication date in ISO 8601 (YYYY, YYYY-MM, or YYYY-MM-DD). Used as + * `datePublished` in the Report JSON-LD and rendered as short date on card. + */ + dateIso: string + /** Publisher category, used for grouping or filtering. */ + category: ReportCategory + /** Destination link. Internal entries point to /reports//. */ + href: string + /** Internal entries open a subpage; external entries open the publisher URL in a new tab. */ + internal?: boolean + /** Cover image. Publisher OG image or first-page render of the source PDF. */ + imgSrc: StaticImageData + /** + * If the link points directly to a PDF, the file size in bytes (from the + * publisher's Content-Length header at the time of authoring). Surfaced in + * the card so readers know what they are about to download. + */ + fileSizeBytes?: number +} + +/** + * Initial curated set of reports surfaced on /reports. + * + * Ordering: internal EF reports first, then external by date desc. + * + * Every external entry has been independently verified. URL fetched, title, + * author, and date confirmed on the publisher site, Ethereum content + * confirmed substantive. See PR description for the full audit. + */ +export const reports: Report[] = [ + // TODO: PDF to be uploaded to /public/reports/ — confirm filename + add fileSizeBytes. + // { + // slug: "ethereum-basics-governments-institutions", + // title: "Ethereum Basics for Governments and Institutions", + // publisher: "Ethereum Foundation", + // dateIso: "2026", + // category: "ef-original", + // href: "/reports/ethereum-basics-for-governments-and-institutions.pdf", + // imgSrc: ethereumBasicsCover, + // }, + { + slug: "trillion-dollar-security", + title: "Trillion Dollar Security", + publisher: "Ethereum Foundation", + dateIso: "2025-05", + category: "ef-original", + href: "/reports/trillion-dollar-security/", + internal: true, + imgSrc: tdsCover, + }, + { + slug: "openzeppelin-blockchain-network-risk-assessment", + title: "Technical Risk Assessment on Blockchain Networks", + publisher: "OpenZeppelin", + dateIso: "2026-04-30", + category: "crypto-native", + href: "https://openzeppelin.com/hubfs/OpenZeppelin%20%7C%20Technical%20Risk%20Assessment%20on%20Blockchain%20Networks.pdf", + imgSrc: openzeppelinCover, + fileSizeBytes: 841405, + }, + { + slug: "coinbase-iab-quantum-computing-blockchain", + title: "Quantum Computing & Blockchain", + publisher: + "Coinbase Independent Advisory Board on Quantum Computing and Blockchain", + dateIso: "2026-04-21", + category: "academic", + href: "https://assets.ctfassets.net/sygt3q11s4a9/6EjYavuGdtJDYCqaJrASj9/9f464a8bf26f44bd6c85710fe7e4a29f/Quantum_Computing_and_Blockchain_v10.3_15April2026.pdf", + imgSrc: quantumCover, + fileSizeBytes: 434102, + }, + { + slug: "a16z-state-of-crypto-2025", + title: "State of Crypto Report 2025", + publisher: "a16z crypto", + dateIso: "2025-10-22", + category: "crypto-native", + href: "https://dwt2zme5yrom6.cloudfront.net/uploads/2025/10/State-of-Crypto-2025-a16z-crypto.pdf", + imgSrc: a16zCover, + fileSizeBytes: 17291230, + }, + { + slug: "etherealize-nethermind-l2beat-l2-landscape", + title: + "The Future of Financial Infrastructure: Ethereum's Layer 2 Landscape", + publisher: "Etherealize, Nethermind and L2BEAT", + dateIso: "2025-12-04", + category: "crypto-native", + href: "https://cdn.prod.website-files.com/6728e9076a3b5a8ca8ec4816/6931c20f55129e498a8da223_%5BCompressed%5D%20L2s%20Report.pdf", + imgSrc: l2LandscapeCover, + fileSizeBytes: 6953384, + }, + { + slug: "fidelity-coin-report-ethereum", + title: "Coin Report: Ethereum (ETH)", + publisher: "Fidelity Digital Assets", + dateIso: "2025-08-21", + category: "bank-research", + href: "https://www.fidelitydigitalassets.com/research-and-insights/coin-report-ethereum-eth", + imgSrc: fidelityCoinCover, + }, + { + slug: "consensys-ethereum-is-trustware", + title: "Ethereum is Trustware: core trust infrastructure for the world", + publisher: "Consensys", + dateIso: "2025-08-04", + category: "crypto-native", + href: "https://consensys.io/ethereum/trust", + imgSrc: consensysCover, + }, + { + slug: "twinstake-ethereum-pectra-institutional-staking", + title: "Ethereum Pectra Upgrade: The Impact on Institutional Staking", + publisher: "Twinstake", + dateIso: "2025", + category: "crypto-native", + href: "https://cdn.prod.website-files.com/658498cb3744de71ad789ca8/67cee06a2f54159204b600ea_Pectra%20Report.pdf", + imgSrc: twinstakeCover, + fileSizeBytes: 1258985, + }, + { + slug: "white-house-pwg-crypto", + title: "Strengthening American Leadership in Digital Financial Technology", + publisher: + "The White House (President's Working Group on Digital Asset Markets)", + dateIso: "2025-07-30", + category: "regulator", + href: "https://www.whitehouse.gov/wp-content/uploads/2025/07/Digital-Assets-Report-EO14178.pdf", + imgSrc: whiteHouseCover, + fileSizeBytes: 5864568, + }, + { + slug: "bis-papers-156-defi-functions-stability", + title: + "Cryptocurrencies and decentralised finance: functions and financial stability implications", + publisher: "Bank for International Settlements", + dateIso: "2025-04-15", + category: "central-bank", + href: "https://www.bis.org/publ/bppdf/bispap156.pdf", + imgSrc: bisCover, + fileSizeBytes: 751136, + }, + { + slug: "mckinsey-from-ripples-to-waves", + title: + "From ripples to waves: The transformational power of tokenizing assets", + publisher: "McKinsey & Company", + dateIso: "2024-06-20", + category: "big-four", + href: "https://www.mckinsey.com/industries/financial-services/our-insights/from-ripples-to-waves-the-transformational-power-of-tokenizing-assets", + imgSrc: mckinseyCover, + }, +] diff --git a/app/[locale]/reports/page-jsonld.tsx b/app/[locale]/reports/page-jsonld.tsx new file mode 100644 index 000000000000..9b81ce4bfb15 --- /dev/null +++ b/app/[locale]/reports/page-jsonld.tsx @@ -0,0 +1,122 @@ +import { getTranslations } from "next-intl/server" + +import { FileContributor } from "@/lib/types" + +import PageJsonLD from "@/components/PageJsonLD" + +import { isExternal, isPdf, normalizeUrlForJsonLd } from "@/lib/utils/url" + +import { SITE_URL } from "@/lib/constants" + +import type { Report } from "./data" + +import { BASE_GRAPH_NODES } from "@/lib/jsonld/constants" +import { REFERENCE } from "@/lib/jsonld/references" + +const reportSchema = ( + report: Report, + index: number, + locale: string, + pageUrl: string +) => { + const itemUrl = isExternal(report.href) + ? report.href + : normalizeUrlForJsonLd(locale, report.href) + const imageUrl = `${SITE_URL}${report.imgSrc.src}` + + return { + "@type": "ListItem", + position: index + 1, + item: { + "@type": "Report", + "@id": `${pageUrl}#${report.slug}`, + name: report.title, + url: itemUrl, + image: imageUrl, + datePublished: report.dateIso, + inLanguage: "en", + publisher: { + "@type": "Organization", + name: report.publisher, + }, + ...(isPdf(report.href) && { + encodingFormat: "application/pdf", + }), + ...(typeof report.fileSizeBytes === "number" && { + contentSize: `${(report.fileSizeBytes / 1048576).toFixed(1)} MB`, + }), + }, + } +} + +export default async function ReportsPageJsonLD({ + locale, + contributors, + reports, +}: { + locale: string + contributors: FileContributor[] + reports: Report[] +}) { + const t = await getTranslations("page-reports") + + const url = normalizeUrlForJsonLd(locale, "/reports/") + const itemListId = `${url}#reports-list` + + const contributorList = contributors.map((contributor) => ({ + "@type": "Person", + name: contributor.login, + url: contributor.html_url, + })) + + const jsonLd = { + "@context": "https://schema.org", + "@graph": [ + ...BASE_GRAPH_NODES, + { + "@type": "CollectionPage", + "@id": url, + name: t("page-reports-metadata-title"), + description: t("page-reports-metadata-description"), + url, + inLanguage: locale, + contributor: contributorList, + author: [REFERENCE.ETHEREUM_COMMUNITY], + isPartOf: REFERENCE.ETHEREUM_ORG_WEBSITE, + breadcrumb: { + "@type": "BreadcrumbList", + itemListElement: [ + { + "@type": "ListItem", + position: 1, + name: "Home", + item: normalizeUrlForJsonLd(locale, "/"), + }, + { + "@type": "ListItem", + position: 2, + name: t("page-reports-metadata-title"), + item: normalizeUrlForJsonLd(locale, "/reports/"), + }, + ], + }, + publisher: REFERENCE.ETHEREUM_FOUNDATION, + reviewedBy: REFERENCE.ETHEREUM_FOUNDATION, + mainEntity: { "@id": itemListId }, + }, + { + "@type": "ItemList", + "@id": itemListId, + name: t("page-reports-metadata-title"), + description: t("page-reports-metadata-description"), + numberOfItems: reports.length, + itemListOrder: "https://schema.org/ItemListOrderDescending", + itemListElement: reports.map((report, index) => + reportSchema(report, index, locale, url) + ), + }, + ], + } + + return +} diff --git a/app/[locale]/reports/page.tsx b/app/[locale]/reports/page.tsx new file mode 100644 index 000000000000..693e929382a9 --- /dev/null +++ b/app/[locale]/reports/page.tsx @@ -0,0 +1,163 @@ +import { getTranslations, setRequestLocale } from "next-intl/server" + +import type { Lang, PageParams } from "@/lib/types" + +import { ContentHero } from "@/components/Hero" +import { Image } from "@/components/Image" +import MainArticle from "@/components/MainArticle" +import { ButtonLink } from "@/components/ui/buttons/Button" +import { + Card, + CardBanner, + CardContent, + CardHeader, + CardParagraph, + CardTitle, +} from "@/components/ui/card" +import { ExternalLinkIcon } from "@/components/ui/Link" +import { Section } from "@/components/ui/section" +import { Tag } from "@/components/ui/tag" + +import { getAppPageContributorInfo } from "@/lib/utils/contributors" +import { formatDate } from "@/lib/utils/date" +import { getMetadata } from "@/lib/utils/metadata" +import { numberFormat } from "@/lib/utils/numbers" +import { isExternal, isFile } from "@/lib/utils/url" + +import { reports } from "./data" +import ReportsPageJsonLD from "./page-jsonld" + +import heroImg from "@/public/images/reports/reports-hero.webp" + +const Page = async (props: { params: Promise }) => { + const params = await props.params + const { locale } = params + + setRequestLocale(locale) + + const t = await getTranslations("page-reports") + + const { contributors } = await getAppPageContributorInfo( + "reports", + locale as Lang + ) + + return ( + <> + + + + +
+
+

{t("page-reports-heading")}

+

{t("page-reports-intro")}

+
+ +
+ {reports.map( + ({ + slug, + title, + publisher, + dateIso, + href, + internal, + imgSrc, + fileSizeBytes, + }) => ( + + + + + + + +
+ + {formatDate(dateIso, locale, { + year: "numeric", + month: "short", + timeZone: "UTC", + day: undefined, + })} + + {fileSizeBytes ? ( + + {t("page-reports-pdf-size", { + size: numberFormat(locale, { + style: "unit", + unit: "megabyte", + unitDisplay: "short", + maximumFractionDigits: 1, + }).format(fileSizeBytes / 2 ** 20), + })} + + ) : !internal ? ( + + {t("page-reports-web-article")} + + ) : null} +
+ + {title} + {(isExternal(href) || isFile(href)) && ( + + )} + + {publisher} + + + ) + )} +
+
+ +
+
+

{t("page-reports-suggest-heading")}

+

{t("page-reports-suggest-body")}

+
+ + {t("page-reports-suggest-cta")} + +
+ + + ) +} + +export async function generateMetadata(props: { + params: Promise<{ locale: string }> +}) { + const params = await props.params + const { locale } = params + + const t = await getTranslations("page-reports") + + return await getMetadata({ + locale, + slug: ["reports"], + title: t("page-reports-metadata-title"), + description: t("page-reports-metadata-description"), + image: "/images/reports/reports-hero.webp", + }) +} + +export default Page diff --git a/app/[locale]/trillion-dollar-security/page-jsonld.tsx b/app/[locale]/reports/trillion-dollar-security/page-jsonld.tsx similarity index 96% rename from app/[locale]/trillion-dollar-security/page-jsonld.tsx rename to app/[locale]/reports/trillion-dollar-security/page-jsonld.tsx index 8ca265a15ac3..b8b68793e065 100644 --- a/app/[locale]/trillion-dollar-security/page-jsonld.tsx +++ b/app/[locale]/reports/trillion-dollar-security/page-jsonld.tsx @@ -20,7 +20,10 @@ export default async function TrillionDollarSecurityPageJsonLD({ }) { const t = await getTranslations("page-trillion-dollar-security") - const url = normalizeUrlForJsonLd(locale, `/trillion-dollar-security/`) + const url = normalizeUrlForJsonLd( + locale, + `/reports/trillion-dollar-security/` + ) const contributorList = contributors.map((contributor) => ({ "@type": "Person", diff --git a/app/[locale]/trillion-dollar-security/page.tsx b/app/[locale]/reports/trillion-dollar-security/page.tsx similarity index 98% rename from app/[locale]/trillion-dollar-security/page.tsx rename to app/[locale]/reports/trillion-dollar-security/page.tsx index 07f8867337b2..af4c56294edd 100644 --- a/app/[locale]/trillion-dollar-security/page.tsx +++ b/app/[locale]/reports/trillion-dollar-security/page.tsx @@ -6,12 +6,7 @@ import type { Lang, PageParams } from "@/lib/types" import MainArticle from "@/components/MainArticle" import { ButtonLink } from "@/components/ui/buttons/Button" -import { - Card, - CardContent, - CardFooter, - CardParagraph, -} from "@/components/ui/card" +import { Card, CardBanner, CardFooter, CardHeader } from "@/components/ui/card" import InlineLink, { BaseLink as Link } from "@/components/ui/Link" import { getAppPageContributorInfo } from "@/lib/utils/contributors" @@ -22,27 +17,28 @@ import TrillionDollarSecurityPageJsonLD from "./page-jsonld" import TdsHero from "@/public/images/trillion-dollar-security/hero.png" import TdsReport from "@/public/images/trillion-dollar-security/report.png" -const ReportCard = ({ cta, altText }: { cta: string; altText: string }) => { - return ( - - - - {altText} - - - - - {cta} - - - - ) -} +const ReportCard = ({ cta, altText }: { cta: string; altText: string }) => ( + + + + {altText} + + + + + {cta} + + + +) const TdsPage = async (props: { params: Promise }) => { const params = await props.params @@ -53,7 +49,7 @@ const TdsPage = async (props: { params: Promise }) => { const t = await getTranslations("page-trillion-dollar-security") const { contributors } = await getAppPageContributorInfo( - "trillion-dollar-security", + "reports/trillion-dollar-security", locale as Lang ) @@ -1098,7 +1094,7 @@ export async function generateMetadata(props: { return await getMetadata({ locale, - slug: ["trillion-dollar-security"], + slug: ["reports", "trillion-dollar-security"], title: t("page-trillion-dollar-security-meta-title"), description: t("page-trillion-dollar-security-meta-description"), image: "/images/trillion-dollar-security/og-image.png", diff --git a/app/[locale]/roadmap/_vision/page.tsx b/app/[locale]/roadmap/_vision/page.tsx index 98d3db537fa6..2b95798d9a44 100644 --- a/app/[locale]/roadmap/_vision/page.tsx +++ b/app/[locale]/roadmap/_vision/page.tsx @@ -9,12 +9,12 @@ import { import type { ChildOnlyProp, Lang, PageParams } from "@/lib/types" import Breadcrumbs from "@/components/Breadcrumbs" -import Card from "@/components/Card" import Emoji from "@/components/Emoji" import FeedbackCard from "@/components/FeedbackCard" import FileContributors from "@/components/FileContributors" import I18nProvider from "@/components/I18nProvider" import MainArticle from "@/components/MainArticle" +import MarkdownCard from "@/components/MarkdownCard" import PageHero, { type ContentType as PageHeroContent, } from "@/components/PageHero" @@ -80,8 +80,8 @@ const ProblemCardContainer = (props: ChildOnlyProp) => ( ) -const CentreCard = (props: ComponentPropsWithRef) => ( - ) => ( + diff --git a/app/[locale]/stablecoins/page.tsx b/app/[locale]/stablecoins/page.tsx index 81493928afad..965cb8eca28a 100644 --- a/app/[locale]/stablecoins/page.tsx +++ b/app/[locale]/stablecoins/page.tsx @@ -8,7 +8,6 @@ import { import type { Lang, PageParams } from "@/lib/types" -import CalloutBannerSSR from "@/components/CalloutBannerSSR" import DataProductCard from "@/components/DataProductCard" import Emoji from "@/components/Emoji" import FeedbackCard from "@/components/FeedbackCard" @@ -25,6 +24,7 @@ import StablecoinsTable from "@/components/StablecoinsTable" import Tooltip from "@/components/Tooltip" import Translation from "@/components/Translation" import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" import { Divider } from "@/components/ui/divider" import { Flex } from "@/components/ui/flex" import InlineLink from "@/components/ui/Link" @@ -595,30 +595,21 @@ async function Page(props: { params: Promise }) {
- -
- - {t("page-stablecoins-explore-dapps")} - - - {t("page-stablecoins-more-defi-button")} - -
- + + {t("page-stablecoins-explore-dapps")} + + + {t("page-stablecoins-more-defi-button")} + +

{t("page-stablecoins-save-stablecoins")}

diff --git a/app/[locale]/staking/_components/StakingCommunityCallout.tsx b/app/[locale]/staking/_components/StakingCommunityCallout.tsx new file mode 100644 index 000000000000..c9fbf5ce5d25 --- /dev/null +++ b/app/[locale]/staking/_components/StakingCommunityCallout.tsx @@ -0,0 +1,56 @@ +import React from "react" +import { getTranslations } from "next-intl/server" + +import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" + +import image from "@/public/images/enterprise-eth.png" + +const StakingCommunityCallout = async ( + props: React.HTMLAttributes +) => { + const t = await getTranslations("page-staking") + const tCommon = await getTranslations("common") + + return ( + + + Discord + + + Reddit + + + {tCommon("rollup-component-website")} + + + ) +} + +export default StakingCommunityCallout diff --git a/app/[locale]/staking/deposit-contract/_components/deposit-contract.tsx b/app/[locale]/staking/deposit-contract/_components/deposit-contract.tsx index 4b81989de420..14394507cd49 100644 --- a/app/[locale]/staking/deposit-contract/_components/deposit-contract.tsx +++ b/app/[locale]/staking/deposit-contract/_components/deposit-contract.tsx @@ -26,6 +26,14 @@ import { type ButtonLinkProps, type ButtonProps, } from "@/components/ui/buttons/Button" +import { + Card, + CardContent, + CardFooter, + CardHeader, + CardParagraph, + CardTitle, +} from "@/components/ui/card" import Checkbox from "@/components/ui/checkbox" import { Flex } from "@/components/ui/flex" import InlineLink from "@/components/ui/Link" @@ -61,13 +69,6 @@ const Subtitle = (props: ChildOnlyProp) => (

) -const ButtonRow = (props: ChildOnlyProp) => ( - -) - const H2 = (props: ChildOnlyProp) => (

) @@ -81,29 +82,6 @@ const StyledButton = ({ ) -const CardTag = (props: ChildOnlyProp) => ( - -) - -const AddressCard = (props: ChildOnlyProp) => { - return ( -
- ) -} - -const Address = (props: ChildOnlyProp) => ( -
-) - const CopyButton = (props: ButtonProps) => (

{benefits.map( - ({ title, description, emoji, linkText, href }, idx) => ( - ( + - {href && linkText && ( - {linkText} + {href && ctaLabel && ( + {ctaLabel} )} - + ) )} diff --git a/app/[locale]/use-cases/page.tsx b/app/[locale]/use-cases/page.tsx index ba5b9fa36d8d..a2a15f2c161a 100644 --- a/app/[locale]/use-cases/page.tsx +++ b/app/[locale]/use-cases/page.tsx @@ -3,16 +3,18 @@ import { getTranslations } from "next-intl/server" import type { PageParams, ToCItem } from "@/lib/types" import type { Lang } from "@/lib/types" -import CalloutBannerSSR from "@/components/CalloutBannerSSR" import DocLink from "@/components/DocLink" import { HubHero } from "@/components/Hero" import type { HubHeroProps } from "@/components/Hero/HubHero" import { Image, ImageProps } from "@/components/Image" import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" import { Card, CardBanner, CardContent, + CardFooter, + CardHeader, CardParagraph, CardTitle, } from "@/components/ui/card" @@ -54,21 +56,25 @@ const UseCaseCard = ({ description: string ctaLabel: string }) => ( - - - - - - {title} - {description} + + + + + + + + {title} + {description} - - {ctaLabel} - + + + {ctaLabel} + + ) @@ -194,17 +200,15 @@ export default async function Page(props: { params: Promise }) {
{/* AI agents banner */} - - - {t("ai-agents-cta")} - - + {t("ai-agents-cta")} + {/* Digital ownership and gaming */}
@@ -283,21 +287,19 @@ export default async function Page(props: { params: Promise }) {
{/* Ready to start? */} - -
- - {t("ready-to-start-wallet-cta")} - - - {t("ready-to-start-eth-cta")} - -
- + + {t("ready-to-start-wallet-cta")} + + + {t("ready-to-start-eth-cta")} + + diff --git a/app/[locale]/videos/_components/VideoGalleryFilter.tsx b/app/[locale]/videos/_components/VideoGalleryFilter.tsx index 06ffe2734343..e2c038621b25 100644 --- a/app/[locale]/videos/_components/VideoGalleryFilter.tsx +++ b/app/[locale]/videos/_components/VideoGalleryFilter.tsx @@ -262,7 +262,12 @@ const VideoGalleryFilter = ({ ) : (
{sortedVideos.map((video) => ( - + - {video.title} - + {video.title} + {video.description} - + {video.duration} diff --git a/app/[locale]/wallets/page.tsx b/app/[locale]/wallets/page.tsx index b52efeef75b9..d78b82a83cd7 100644 --- a/app/[locale]/wallets/page.tsx +++ b/app/[locale]/wallets/page.tsx @@ -1,4 +1,4 @@ -import { ComponentPropsWithRef, Suspense } from "react" +import { Suspense } from "react" import { pick } from "lodash" import { getMessages, @@ -8,8 +8,6 @@ import { import type { Lang, PageParams } from "@/lib/types" -import Callout from "@/components/Callout" -import Card from "@/components/Card" import CardList from "@/components/CardList" import FeedbackCard from "@/components/FeedbackCard" import FileContributors from "@/components/FileContributors" @@ -18,11 +16,13 @@ import I18nProvider from "@/components/I18nProvider" import { Image } from "@/components/Image" import ListenToPlayer from "@/components/ListenToPlayer" import MainArticle from "@/components/MainArticle" +import MarkdownCard from "@/components/MarkdownCard" import PageHero from "@/components/PageHero" import { StandaloneQuizWidget } from "@/components/Quiz/QuizWidget" import { SIMULATOR_ID } from "@/components/Simulator/constants" import Translation from "@/components/Translation" import { ButtonLink } from "@/components/ui/buttons/Button" +import Callout from "@/components/ui/callout" import { Divider } from "@/components/ui/divider" import { getAppPageContributorInfo } from "@/lib/utils/contributors" @@ -37,13 +37,6 @@ import ETHImage from "@/public/images/eth-logo.png" import FindWalletImage from "@/public/images/wallets/find-wallet.png" import HeroImage from "@/public/images/wallets/wallet-hero.png" -const StyledCard = (props: ComponentPropsWithRef) => ( - -) - const Page = async (props: { params: Promise }) => { const params = await props.params const { locale } = params @@ -233,17 +226,17 @@ const Page = async (props: { params: Promise }) => {

-
-
- {cards.map((card, idx) => ( - - ))} -
+
+ {cards.map((card, idx) => ( + + ))}
@@ -415,32 +408,26 @@ const Page = async (props: { params: Promise }) => {

{t("page-wallets-explore")}

-
+
-
- - {t("page-wallets-get-some-btn")} - -
+ + {t("page-wallets-get-some-btn")} + -
- - {t("page-wallets-more-on-dapps-btn")} - -
+ + {t("page-wallets-more-on-dapps-btn")} +
diff --git a/app/[locale]/what-is-ether/page.tsx b/app/[locale]/what-is-ether/page.tsx index c303d9ad09dc..e6dc6d6366cf 100644 --- a/app/[locale]/what-is-ether/page.tsx +++ b/app/[locale]/what-is-ether/page.tsx @@ -7,7 +7,6 @@ import FileContributors from "@/components/FileContributors" import ContentHero, { ContentHeroProps } from "@/components/Hero/ContentHero" import { HighlightCard, - HighlightCardContent, HighlightStack, IconBox, } from "@/components/HighlightCard" @@ -16,7 +15,7 @@ import { Strong } from "@/components/IntlStringElements" import MainArticle from "@/components/MainArticle" import TableOfContents from "@/components/TableOfContents" import { Alert, AlertContent, AlertEmoji } from "@/components/ui/alert" -import { CardTitle } from "@/components/ui/card" +import { CardParagraph, CardTitle } from "@/components/ui/card" import Link, { LinkWithArrow } from "@/components/ui/Link" import { ListItem, OrderedList, UnorderedList } from "@/components/ui/list" import { Section } from "@/components/ui/section" @@ -165,45 +164,39 @@ const Page = async (props: { params: Promise<{ locale: Lang }> }) => { -
- +
+ {t("page-what-is-ether-what-is-ether-description-6")} - -

- {t("page-what-is-ether-what-is-ether-description-7")} -

- + + {t("page-what-is-ether-what-is-ether-description-7")} +
-
- +
+ {t("page-what-is-ether-what-is-ether-description-8")} - -

- {t("page-what-is-ether-what-is-ether-description-9")} -

- + + {t("page-what-is-ether-what-is-ether-description-9")} +
-
- +
+ {t("page-what-is-ether-what-is-ether-description-10")} - -

- {t("page-what-is-ether-what-is-ether-description-11")} -

- + + {t("page-what-is-ether-what-is-ether-description-11")} +
diff --git a/app/[locale]/what-is-ethereum/page.tsx b/app/[locale]/what-is-ethereum/page.tsx index f1a5c3ff1511..d6789b22e791 100644 --- a/app/[locale]/what-is-ethereum/page.tsx +++ b/app/[locale]/what-is-ethereum/page.tsx @@ -16,7 +16,6 @@ import FileContributors from "@/components/FileContributors" import ContentHero, { ContentHeroProps } from "@/components/Hero/ContentHero" import { HighlightCard, - HighlightCardContent, HighlightStack, IconBox, } from "@/components/HighlightCard" @@ -26,7 +25,14 @@ import ListenToPlayer from "@/components/ListenToPlayer/lazy" import MainArticle from "@/components/MainArticle" import TableOfContents from "@/components/TableOfContents" import { ButtonLink } from "@/components/ui/buttons/Button" -import { Card, CardContent, CardTitle } from "@/components/ui/card" +import { + Card, + CardContent, + CardFooter, + CardHeader, + CardParagraph, + CardTitle, +} from "@/components/ui/card" import Link, { LinkWithArrow } from "@/components/ui/Link" import { ListItem, OrderedList, UnorderedList } from "@/components/ui/list" import { Section } from "@/components/ui/section" @@ -210,60 +216,54 @@ const Page = async (props: { params: Promise }) => { -
- + + {t("page-what-is-ethereum-network-censorship-title")} - -

- {t("page-what-is-ethereum-network-censorship-desc-1")} -

-

- {t("page-what-is-ethereum-network-censorship-desc-2")} -

- -
+ + {t("page-what-is-ethereum-network-censorship-desc-1")} + + + {t("page-what-is-ethereum-network-censorship-desc-2")} + + -
- + + {t("page-what-is-ethereum-network-security-title")} - -

- {t("page-what-is-ethereum-network-security-desc-1")} -

-

- {t("page-what-is-ethereum-network-security-desc-2")} -

- -
+ + {t("page-what-is-ethereum-network-security-desc-1")} + + + {t("page-what-is-ethereum-network-security-desc-2")} + + -
- + + {t("page-what-is-ethereum-network-reliability-title")} - -

- {t.rich( - "page-what-is-ethereum-network-reliability-desc-1", - { - a: (chunks) => {chunks}, - } - )} -

-

- {t("page-what-is-ethereum-network-reliability-desc-2")} -

- -
+ + {t.rich( + "page-what-is-ethereum-network-reliability-desc-1", + { + a: (chunks) => {chunks}, + } + )} + + + {t("page-what-is-ethereum-network-reliability-desc-2")} + + @@ -478,138 +478,135 @@ const Page = async (props: { params: Promise }) => { -
- + + {t("page-what-is-ethereum-what-consumers-title")} - -

{t("page-what-is-ethereum-what-consumers-desc-1")}

-

{t("page-what-is-ethereum-what-consumers-desc-2")}

- - - {t.rich( - "page-what-is-ethereum-what-consumers-benefit-1", - { - strong: Strong, - } - )} - - - {t.rich( - "page-what-is-ethereum-what-consumers-benefit-2", - { - strong: Strong, - } - )} - - - {t.rich( - "page-what-is-ethereum-what-consumers-benefit-3", - { - strong: Strong, - } - )} - - - {t("page-what-is-ethereum-what-consumers-benefit-4")} - - - -
+ + {t("page-what-is-ethereum-what-consumers-desc-1")} + + + {t("page-what-is-ethereum-what-consumers-desc-2")} + + + + {t.rich( + "page-what-is-ethereum-what-consumers-benefit-1", + { + strong: Strong, + } + )} + + + {t.rich( + "page-what-is-ethereum-what-consumers-benefit-2", + { + strong: Strong, + } + )} + + + {t.rich( + "page-what-is-ethereum-what-consumers-benefit-3", + { + strong: Strong, + } + )} + + + {t("page-what-is-ethereum-what-consumers-benefit-4")} + + + -
- + + {t("page-what-is-ethereum-what-businesses-title")} - - - - {t("page-what-is-ethereum-what-businesses-benefit-1")} - - - {t.rich( - "page-what-is-ethereum-what-businesses-benefit-2", - { - strong: Strong, - } - )} - - - {t("page-what-is-ethereum-what-businesses-benefit-3")} - - -

+ + + {t("page-what-is-ethereum-what-businesses-benefit-1")} + + {t.rich( - "page-what-is-ethereum-what-businesses-example", + "page-what-is-ethereum-what-businesses-benefit-2", { - a: (chunks) => ( - - {chunks} - - ), + strong: Strong, } )} -

- -
+ + + {t("page-what-is-ethereum-what-businesses-benefit-3")} + + + + {t.rich("page-what-is-ethereum-what-businesses-example", { + a: (chunks) => ( + + {chunks} + + ), + })} + + -
- + + {t("page-what-is-ethereum-what-governments-title")} - -

{t("page-what-is-ethereum-what-governments-intro")}

- - - {t.rich( - "page-what-is-ethereum-what-governments-benefit-1", - { - strong: Strong, - } - )} - - - {t.rich( - "page-what-is-ethereum-what-governments-benefit-2", - { - strong: Strong, - } - )} - - - {t.rich( - "page-what-is-ethereum-what-governments-benefit-3", - { - strong: Strong, - } - )} - - -

+ + {t("page-what-is-ethereum-what-governments-intro")} + + + {t.rich( - "page-what-is-ethereum-what-governments-example-1", + "page-what-is-ethereum-what-governments-benefit-1", { - a: (chunks) => ( - - {chunks} - - ), + strong: Strong, } )} -

-

- {t("page-what-is-ethereum-what-governments-example-2")} -

- -
+ + + {t.rich( + "page-what-is-ethereum-what-governments-benefit-2", + { + strong: Strong, + } + )} + + + {t.rich( + "page-what-is-ethereum-what-governments-benefit-3", + { + strong: Strong, + } + )} + + + + {t.rich( + "page-what-is-ethereum-what-governments-example-1", + { + a: (chunks) => ( + + {chunks} + + ), + } + )} + + + {t("page-what-is-ethereum-what-governments-example-2")} + + @@ -635,71 +632,71 @@ const Page = async (props: { params: Promise }) => {
- - + + - {t("page-what-is-ethereum-start-individuals-title")} - - -
-

- - {t("page-what-is-ethereum-start-individuals-desc-1")} - -

+ + {t("page-what-is-ethereum-start-individuals-title")} + + + + + + {t("page-what-is-ethereum-start-individuals-desc-1")} + + -

+ + {t.rich("page-what-is-ethereum-start-individuals-desc-3", { + zerion: (chunks) => ( + {chunks} + ), + rainbow: (chunks) => ( + {chunks} + ), + coinbase: (chunks) => ( + + {chunks} + + ), + })} + + + + + {t("page-what-is-ethereum-start-individuals-step-1")} + + + {t("page-what-is-ethereum-start-individuals-step-2")} + + {t.rich( - "page-what-is-ethereum-start-individuals-desc-3", + "page-what-is-ethereum-start-individuals-step-3", { - zerion: (chunks) => ( - {chunks} - ), - rainbow: (chunks) => ( - {chunks} + zora: (chunks) => ( + {chunks} ), - coinbase: (chunks) => ( - + uniswap: (chunks) => ( + {chunks} ), + farcaster: (chunks) => ( + {chunks} + ), } )} -

- - - - {t("page-what-is-ethereum-start-individuals-step-1")} - - - {t("page-what-is-ethereum-start-individuals-step-2")} - - - {t.rich( - "page-what-is-ethereum-start-individuals-step-3", - { - zora: (chunks) => ( - {chunks} - ), - uniswap: (chunks) => ( - - {chunks} - - ), - farcaster: (chunks) => ( - - {chunks} - - ), - } - )} - - - -

{t("page-what-is-ethereum-start-individuals-desc-4")}

-

{t("page-what-is-ethereum-start-individuals-desc-5")}

-
+ + + + {t("page-what-is-ethereum-start-individuals-desc-4")} + + + {t("page-what-is-ethereum-start-individuals-desc-5")} + + +
{t("page-what-is-ethereum-start-individuals-cta-1")} @@ -708,86 +705,100 @@ const Page = async (props: { params: Promise }) => { {t("page-what-is-ethereum-start-individuals-cta-2")}
- + - - + + - {t("page-what-is-ethereum-start-developers-title")} - - -
-

{t("page-what-is-ethereum-start-developers-desc-1")}

-

- {t.rich("page-what-is-ethereum-start-developers-desc-2", { - a: (chunks) => ( - {chunks} - ), - })} -

-

- {t.rich("page-what-is-ethereum-start-developers-desc-3", { - hardhat: (chunks) => ( - {chunks} - ), - foundry: (chunks) => ( - {chunks} - ), - ethers: (chunks) => ( - {chunks} - ), - thirdweb: (chunks) => ( - {chunks} - ), - moralis: (chunks) => ( - {chunks} - ), - })} -

-

{t("page-what-is-ethereum-start-developers-desc-4")}

-
+ + {t("page-what-is-ethereum-start-developers-title")} + + + + + {t("page-what-is-ethereum-start-developers-desc-1")} + + + {t.rich("page-what-is-ethereum-start-developers-desc-2", { + a: (chunks) => ( + {chunks} + ), + })} + + + {t.rich("page-what-is-ethereum-start-developers-desc-3", { + hardhat: (chunks) => ( + {chunks} + ), + foundry: (chunks) => ( + {chunks} + ), + ethers: (chunks) => ( + {chunks} + ), + thirdweb: (chunks) => ( + {chunks} + ), + moralis: (chunks) => ( + {chunks} + ), + })} + + + {t("page-what-is-ethereum-start-developers-desc-4")} + + + {t("page-what-is-ethereum-start-developers-cta")} - + - - + + - {t("page-what-is-ethereum-start-business-title")} - - -
-

{t("page-what-is-ethereum-start-business-desc-1")}

-

{t("page-what-is-ethereum-start-business-desc-2")}

-

{t("page-what-is-ethereum-start-business-desc-3")}

- - - {t("page-what-is-ethereum-start-business-benefit-1")} - - - {t("page-what-is-ethereum-start-business-benefit-2")} - - - {t("page-what-is-ethereum-start-business-benefit-3")} - - -

- {t.rich("page-what-is-ethereum-start-business-example", { - a: (chunks) => ( - - {chunks} - - ), - })} -

-
+ + {t("page-what-is-ethereum-start-business-title")} + + + + + {t("page-what-is-ethereum-start-business-desc-1")} + + + {t("page-what-is-ethereum-start-business-desc-2")} + + + {t("page-what-is-ethereum-start-business-desc-3")} + + + + {t("page-what-is-ethereum-start-business-benefit-1")} + + + {t("page-what-is-ethereum-start-business-benefit-2")} + + + {t("page-what-is-ethereum-start-business-benefit-3")} + + + + {t.rich("page-what-is-ethereum-start-business-example", { + a: (chunks) => ( + + {chunks} + + ), + })} + + + {t("page-what-is-ethereum-start-business-cta")} - + diff --git a/app/developers/docs/llms.txt/route.ts b/app/developers/docs/llms.txt/route.ts new file mode 100644 index 000000000000..03d9aba20767 --- /dev/null +++ b/app/developers/docs/llms.txt/route.ts @@ -0,0 +1,48 @@ +import { getTranslations } from "next-intl/server" + +import docLinks from "@/data/developer-docs-links.yaml" + +import { SITE_URL } from "@/lib/constants" + +import { + type DocLink, + renderDocsNode, + type Translator, +} from "@/lib/llms-txt/render" + +export const dynamic = "force-static" + +const INTRO = `# Ethereum Developer Documentation + +> Technical reference for building on Ethereum: protocol concepts, the Ethereum stack, smart contracts, scaling solutions, and developer tooling. + +This file indexes the developer documentation under ${SITE_URL}/developers/docs/. For the full ethereum.org index including learner content, guides, and community resources, see ${SITE_URL}/llms.txt.` + +const links = docLinks as unknown as DocLink[] + +const renderTopGroup = (group: DocLink, t: Translator): string => { + const lines = [`## ${t(group.id)}`, ""] + for (const item of group.items ?? []) { + lines.push(...renderDocsNode(item, 0, t)) + } + return lines.join("\n") +} + +export const GET = async () => { + const t = await getTranslations({ + locale: "en", + namespace: "page-developers-docs", + }) + + const sections = links.map((entry) => + entry.items + ? renderTopGroup(entry, t) + : renderDocsNode(entry, 0, t).join("\n") + ) + + const body = [INTRO, ...sections, ""].join("\n\n") + + return new Response(body, { + headers: { "content-type": "text/plain; charset=utf-8" }, + }) +} diff --git a/app/llms.txt/route.ts b/app/llms.txt/route.ts new file mode 100644 index 000000000000..713365f0569f --- /dev/null +++ b/app/llms.txt/route.ts @@ -0,0 +1,44 @@ +import { getTranslations } from "next-intl/server" + +import { SITE_URL } from "@/lib/constants" + +import { renderLegalSection, renderNavSection } from "@/lib/llms-txt/render" +import { buildNavigation } from "@/lib/nav/buildNavigation" +import { + buildFooterDipperLinks, + buildFooterLinkSections, +} from "@/lib/nav/footerLinks" + +export const dynamic = "force-static" + +const INTRO = `# Ethereum.org + +> The official Ethereum website providing comprehensive education, resources, and community information about Ethereum — the decentralized world computer that enables smart contracts and decentralized applications. + +Ethereum.org is the primary educational hub for Ethereum, offering beginner-friendly explanations alongside advanced technical documentation. The site covers everything from basic concepts like "What is Ethereum?" to detailed developer guides, staking information, and protocol research. For the developer-documentation-only index, see ${SITE_URL}/developers/docs/llms.txt.` + +export const GET = async () => { + const t = await getTranslations({ locale: "en", namespace: "common" }) + + const nav = buildNavigation(t) + const footerSections = buildFooterLinkSections(t) + const dipperLinks = buildFooterDipperLinks(t) + + const findFooter = (title: string) => + footerSections.find((s) => s.title === title) + + const body = [ + INTRO, + renderNavSection(nav.learn, findFooter(nav.learn.label)), + renderNavSection(nav.use, findFooter(nav.use.label)), + renderNavSection(nav.build, findFooter(nav.build.label)), + renderNavSection(nav.participate, findFooter(nav.participate.label)), + renderNavSection(nav.research, findFooter(nav.research.label)), + renderLegalSection(dipperLinks), + "", + ].join("\n\n") + + return new Response(body, { + headers: { "content-type": "text/plain; charset=utf-8" }, + }) +} diff --git a/app/sitemap.ts b/app/sitemap.ts index 614a0b25cc87..94fcfcfe171c 100644 --- a/app/sitemap.ts +++ b/app/sitemap.ts @@ -1,6 +1,6 @@ import type { MetadataRoute } from "next" -import { getFullUrl } from "@/lib/utils/url" +import { getFullUrl, toLanguageTag } from "@/lib/utils/url" import { DEFAULT_LOCALE } from "@/lib/constants" @@ -21,7 +21,7 @@ export default async function sitemap(): Promise { "x-default": getFullUrl(DEFAULT_LOCALE, normalizedSlug), ...Object.fromEntries( translatedLocales.map((locale) => [ - locale, + toLanguageTag(locale), getFullUrl(locale, normalizedSlug), ]) ), diff --git a/docs/gemini-translation-roadmap.md b/docs/gemini-translation-roadmap.md deleted file mode 100644 index a6f2ea171164..000000000000 --- a/docs/gemini-translation-roadmap.md +++ /dev/null @@ -1,365 +0,0 @@ -# Gemini Translation Pipeline -- Roadmap - -Status: Active plan -Last updated: 2026-03-27 - ---- - -## Current state - -The initial full-repo translation pass is ~97-99% complete across 24 non-English -languages. The Gemini translation pipeline (`gemini-translations.yml`) works well -for full-file translation but has limitations as we shift to ongoing maintenance. - -### What works today - -- Full-file translation with glossary enforcement -- Code block extraction/restoration (`` placeholders) -- Comment translation within code blocks -- Incremental commit per language (no work lost on partial failure) -- Progress tracking and run resumption -- Post-import sanitization and transliteration -- Configurable concurrency, include/exclude paths, per-language targeting -- 100% custom header ID coverage (`{#custom-id}`) across all markdown files, - preserved identically in translations (verified 2026-03-27) - -### Gaps (being addressed) - -1. ~42 file/language pairs failed during the initial pass (see "Failed files") -2. No drift detection (no way to know which translations are stale) -3. No incremental translation (every run retranslates from scratch) -4. Manual triggering (no automation for ongoing maintenance) -5. Limited error logging from the `@google/gen-ai` SDK -6. Some existing translations done with Gemini 2.5 Pro before current sanitizer - improvements, transliteration banks, and glossary enhancements - -### Cost context - -- Initial full-repo pass: ~$1,500 (via Crowdin + Gemini 2.5 Pro) -- Current pipeline (direct Gemini, bypassing Crowdin): ~80% cheaper -- Estimated full sweep with current pipeline: ~$300-500 -- Gemini Pro pricing (approximate): - - Input: ~$1.25 / 1M tokens - - Output: ~$10.00 / 1M tokens (output dominates cost) - ---- - -## Priority 1: Fix failed files (branch: `gemini-v3`) - -Close the initial pass from ~97-99% to ~100%. This is the most urgent work item. - -### Failed file inventory - -42 file/language pairs failed. Full list: - -``` -ar: glossary.json -bn: json-rpc/index.md, ethash/index.md, ethereum-forks/index.md, - fusaka/peerdas/index.md, glossary.json, learn-quizzes.json, - page-resources.json, page-trillion-dollar-security.json -de: ethereum-forks/index.md, whitepaper/index.md, glossary.json, - learn-quizzes.json -id: nodes-and-clients/index.md, glamsterdam/index.md, merge/index.md, - glossary.json, learn-quizzes.json -it: hello-world-smart-contract-fullstack/index.md, glossary.json, - learn-quizzes.json -sw: glossary.json -ta: translatathon/index.md, json-rpc/index.md, poa/index.md, - pos-vs-pow/index.md, ethash/index.md, web2-vs-web3/index.md, - fusaka/peerdas/index.md, glamsterdam/index.md, glossary.json, - learn-quizzes.json -ur: json-rpc/index.md, dagger-hashimoto/index.md, ethash/index.md, - dapps/index.md, secret-state/index.md, ethereum-forks/index.md, - fusaka/peerdas/index.md, pectra/maxeb/index.md, glossary.json, - page-what-is-the-ethereum-network.json -``` - -### Failure pattern analysis - -| Root cause | Files affected | Details | -|------------------------|----------------|--------------------------------------------| -| Token overload (>15k) | ~7 | whitepaper (90KB), json-rpc (75KB), etc. | -| Code block density | ~5 | json-rpc (172 blocks), hello-world (128) | -| Table/component density| ~5 | ethereum-forks (60 JSX + 33 tables) | -| JSON with embedded HTML| ~3 | glossary.json (317 anchors, 540 escapes) | - -**Repeat offenders:** -- `glossary.json` -- fails for 8 languages (ar, bn, de, id, it, sw, ta, ur) -- `learn-quizzes.json` -- fails for 5 languages (bn, de, id, ta, ur) - -**Languages with most failures:** Tamil (10), Urdu (10), Bengali (8) - -### Fixes to implement on `gemini-v3` - -#### A. Markdown: header ID-based chunking - -Replace token-count-based chunking (which failed) with structure-aware chunking -using the `{#custom-id}` header anchors. - -- Split at heading boundaries, grouping sections up to a token budget per chunk -- Each chunk carries its header IDs for deterministic reassembly -- Header IDs are 100% consistent across the repo and preserved in translations -- Intro content before the first heading gets a synthetic `_intro` key - -#### B. JSON: namespace batching with HTML placeholder pre-parsing - -Two improvements for large/complex JSON files: - -1. **Batch by top-level keys**: Send ~100 key-value pairs per request (with a - ~20 key buffer to avoid wasteful tiny final batches -- e.g., a file with 110 - keys sends one batch of 110, not 100 + 10) - -2. **HTML placeholder pre-parsing**: Before translation, replace embedded HTML - in JSON values with numbered placeholders (similar to Crowdin's `<0>` - pattern but more descriptive). Restore after translation. - - ``` - Before: "A DAO is..." - After: "A DAO is..." - (with restoration map stored separately) - ``` - - Validation: after restoration, verify all placeholders were preserved. Flag - chunks with missing/duplicated placeholders for retry. - -#### C. Code fence extraction audit - -The `` extraction works on successful files. Investigate why -it fails on code-dense files: -- Run the extractor in isolation on failing files, inspect output -- Check for edge cases: nested fences, non-standard fence syntax, very high - placeholder counts (>100 per file) -- May be interaction between chunking failure + code blocks (if chunking fails, - the entire code-heavy file hits Gemini as one blob) - -#### D. Error logging improvements - -Add structured error logging from the `@google/gen-ai` SDK: -- Capture failure reason, response status, partial output if available -- Log per-file/per-language so failures can be triaged without re-running -- Distinguish error types: rate limit vs. content filter vs. malformed output - vs. timeout (each needs different retry strategy) - -#### E. Validation - -- Retranslate the ~42 failed file/language pairs as the test case -- Compare output quality against successfully translated files of similar size - ---- - -## Priority 2: Section hash manifest (branch: `gemini-v4`) - -Build per-section content hashing infrastructure. This is the foundation for both -drift detection and incremental translation. - -### Markdown: header ID-keyed section hashes - -Parse each English markdown file into a tree of sections keyed by `{#custom-id}`. -Hash each section's content. Structure: - -```json -{ - "public/content/roadmap/index.md": { - "fileHash": "abc123", - "sections": { - "_intro": "def456", - "what-is-the-roadmap": "ghi789", - "why-does-ethereum-need-a-roadmap": "jkl012", - ... - } - } -} -``` - -**Possible future optimization**: merkle trie structure where leaf hashes bubble -up to parent sections. Allows O(1) "has anything changed?" checks at the file -level, with drill-down to find exactly which sections changed. Worth considering -once the flat hash map is working, if performance demands it. - -### JSON: key-level hashes - -For JSON files, hash individual key-value pairs (or namespace groups for deeply -nested files). Structure: - -```json -{ - "src/intl/en/glossary.json": { - "fileHash": "mno345", - "keys": { - "account": "pqr678", - "address": "stu901", - ... - } - } -} -``` - -### Storage: manifest file - -**Decision**: Use a manifest file (`src/intl/translation-manifest.json`). - -- Single file, easy to query, no content file pollution -- Works for both JSON and markdown -- Can include metadata: timestamp, pipeline version, token cost, Gemini model -- Trade-off: potential merge conflicts if multiple translation PRs run - simultaneously (mitigated by per-language PRs or lock-step merging) - ---- - -## Priority 3: Baseline sweep + quality refresh - -**Decision**: "Stamp now" approach (Option B from brainstorming). - -One combined operation (~$300-500) that accomplishes two goals simultaneously: - -1. **Establish baseline**: Record current English source SHAs in the manifest - for every file/language pair. Going forward, drift is detectable by comparing - recorded SHA against current English SHA. - -2. **Quality refresh**: Retranslate everything using current best pipeline: - - Gemini 3.1 Pro (upgraded from 2.5 Pro used in original pass) - - Current sanitizer with all accumulated fixes - - Transliteration banks for non-Latin script languages - - Improved translation glossary (in development separately) - -After this sweep, every translation in the repo is (a) generated by the best -available pipeline and (b) tracked in the manifest with a known English source -SHA. This is the clean foundation for incremental work going forward. - -### Prerequisite: glossary and transliteration improvements - -The quality refresh is most valuable after: -- Translation glossary expansion is complete (in flight) -- Transliteration bank coverage is solid for non-Latin scripts -- All Priority 1 fixes are deployed (so zero files fail) - -### Approach alternatives considered - -**Option A (rejected): Git history bootstrap** -- Analyze commit messages -(pattern: `i18n(pl): Crowdin translations`) to determine when each file was -last truly translated. Feasible since commits are programmatic, but complicated -by cleanup commits that are more recent than actual translation timestamps. - -**Option B (selected): Stamp now, sweep forward** -- Accept that current -translations have unknown-precision freshness. Do one full sweep with current -pipeline, stamping SHAs as we go. After this, the manifest is authoritative. - -**Option C (rejected): Hybrid git + LLM spot-check** -- Use git where clear, -LLM where ambiguous. More accurate bootstrap but more complexity for marginal -benefit given we want a quality refresh anyway. - ---- - -## Priority 4: Incremental translation (branch: `gemini-v4`) - -Once the manifest exists with per-section hashes, incremental translation -becomes straightforward. - -### JSON: key-level diff and translate - -1. Deep-diff current English JSON against manifest's recorded English version -2. Collect added and changed key paths -3. Send only those key-value pairs to Gemini for translation -4. Deep-merge translated pairs into existing translation JSON -5. Update manifest with new SHAs -6. Run sanitizer on the merged file - -**Complexity**: Low. JSON key merging is deterministic and safe. - -### Markdown: section-level diff and translate - -1. Parse current English file into sections keyed by `{#header-id}` -2. Compare section hashes against manifest -3. For each changed section: - a. Extract corresponding section from existing translation - b. Send to Gemini: English section + existing translation + context - c. Receive translated section -4. Reassemble: unchanged sections from existing translation + new translations -5. Update manifest with new SHAs -6. Run sanitizer on reassembled file - -**Complexity**: Medium. The 100% header ID coverage makes this much more -feasible than initially estimated. Splicing by ID is deterministic. Edge case: -intro content before first heading (use synthetic `_intro` key). - -**Fallback**: If >50% of sections changed, fall back to full-file retranslation -(the incremental overhead isn't worth it at that point). - -### "Previous English version" question (resolved) - -The manifest's recorded SHA IS the previous English version. When a translation -is generated, the manifest records the English source SHA. On the next -incremental run, diff current English against that SHA to identify what changed. - ---- - -## Priority 5: Automation (branch: `gemini-v4`) - -### End-state vision - -``` -English content merged to dev - | - v -Drift detection scan (automatic or cron) - | - v -Stale file list (per language) - | - v -Batching logic (group by language, thresholds, cooldown) - | - v -Incremental translation dispatch (Gemini 3.1 Pro) - | - v -Sanitizer + transliteration + review agents - | - v -PR(s) created, ready for human merge -``` - -### Graduation plan - -**Phase 1 (near-term): Manual + tooling** -- Drift scan script runs manually or on cron, outputs report -- Human reviews report and manually dispatches translation -- Existing sanitizer + review pipeline handles quality - -**Phase 2 (mid-term): Semi-automated** -- Nightly/weekly cron runs drift scan -- When stale count exceeds threshold, auto-dispatches translation -- Human merges resulting PRs - -**Phase 3 (long-term): Full automation** -- Push to dev triggers path-filtered action (`public/content/`, `src/intl/en/`) -- Batching logic groups changes (cooldown window during active dev) -- Translation -> sanitizer -> review agents -> PR ready for human merge -- Cron job as safety net catches anything the push trigger missed -- Human stays in the loop at the merge step - -### Batching considerations - -- One PR per language per run (clearest for review) -- Skip whitespace-only or comment-only changes -- Cooldown: don't retranslate files translated in the last N hours -- Size cap: if >50 files stale, split into multiple runs or prioritize by traffic - ---- - -## Branch strategy - -- **`gemini-v3`**: Priority 1 (fix failed files). Patches to the existing - pipeline: chunking, batching, HTML placeholders, error logging. -- **`gemini-v4`**: Priorities 2-5 (new infrastructure). Manifest, drift - detection, incremental translation, automation. - ---- - -## Related workstreams (tracked elsewhere) - -- **Translation glossary expansion** -- in flight, separate task -- **Transliteration bank improvements** -- ongoing per non-Latin locale -- **Full-language retroactive cleanup** -- see `src/scripts/i18n/FUTURE.md` #9 -- **Lowercase ethereum initiative** -- content standardization, tracked in - `docs/lowercase-ethereum-plan.md` diff --git a/docs/i18n-incremental-pipeline.md b/docs/i18n-incremental-pipeline.md deleted file mode 100644 index fc3be340c238..000000000000 --- a/docs/i18n-incremental-pipeline.md +++ /dev/null @@ -1,110 +0,0 @@ -# Incremental Translation Pipeline - -## Overview - -The i18n pipeline translates ethereum.org content (markdown + JSON) to 24 languages using Gemini. It operates in two modes: - -- **Auto (default):** For each file+locale, auto-detects whether to do a full translation (no manifests exist) or an incremental update (manifests exist, only changed content retranslated). -- **Full:** Force retranslation of all targeted files regardless of manifest state. - -The pipeline classifies English changes into two categories: -- **Inert changes** (URLs, image paths, code, component attributes): propagated deterministically without LLM calls. -- **Prose changes** (translatable text): retranslated via Gemini section-by-section, with unchanged sections provided as context for voice/tone consistency. - -## Architecture - -### Translation Branch - -All pipeline runs commit to `intl/pending` by default. This is the single translation branch for the standard `dev`-based workflow. - -- If the branch exists, the pipeline merges the base branch into it first (keeps it current with dev). -- If it doesn't exist, it creates one from the base branch HEAD. -- A GitHub Actions concurrency group ensures only one pipeline run executes at a time (additional runs queue). -- The branch name can be overridden via `translation_branch` workflow input (useful for testing). - -**Design decision:** The pipeline only targets `dev` in production. Hot fixes to `staging` or `master` are English-only until the next release cycle, when `dev` (with translations) flows to `staging` then `master` via the normal prepare-release process. This is a deliberate simplification -- multi-branch translation adds significant complexity for a rare scenario. - -### Manifests - -Two manifest files track translation state per file+locale: - -**Source manifest (`.manifest-source.json`):** A content tree of the English file at the time of last translation. Stores hashes (not content) for each section, element, and attribute. Used to detect what changed in English since last translation. - -**Translation manifest (`.manifest-translation.json`):** Records the inert values (URLs, paths, attribute values) as they existed at translation time. Used to propagate inert changes deterministically without re-reading old English content. - -### Pipeline Phases - -1. **Initialize:** Ensure staging branch exists and is up-to-date with base. -2. **Drift Detection:** For each file+locale, compare current English against stored manifest. Classify changes as inert, translatable, added, or removed. Files without manifests are queued for full translation. -3. **Full Translation:** New files go through `translateFile()` (normalizer + Gemini). Both manifests are generated and committed. -4. **Inert Propagation:** Deterministic replacement of URLs, paths, and attributes in existing translated files. No LLM calls. Handles reordered links (e.g., Japanese SOV word order placing links in different positions than English). -5. **Prose Retranslation:** Changed sections sent to Gemini with unchanged sections as context. Responses are spliced back into the locale file. -6. **Commit:** Updated locale files and refreshed manifests committed to the staging branch. -7. **Sanitize:** Post-import sanitizer runs on all Gemini-produced content (BiDi fixes for RTL languages, code fence alignment, etc.). - -### Removed Content Handling - -When English content is removed (sections deleted, JSON keys removed), the pipeline detects these as `drift.removed` entries and strips the corresponding content from all locale files. This enables safe deprecation of components and content without manual editing of translated files. - -## Workflow - -### GitHub Actions - -```bash -# Default: auto-detect mode, commits to intl/pending -gh workflow run gemini-translations.yml \ - -f target_path="public/content/some-page/index.md" \ - -f target_languages="es,ja,ur" - -# Force full retranslation -gh workflow run gemini-translations.yml \ - -f target_path="public/content/some-page/index.md" \ - -f mode="full" - -# Testing: use a feature branch with a custom translation branch -gh workflow run gemini-translations.yml \ - --ref test-6/gemini-v4 \ - -f base_branch="test-6/gemini-v4" \ - -f translation_branch="intl/test-pending" -``` - -### Content Author Workflow - -1. Author writes/edits English content, merges PR to `dev`. -2. Pipeline dispatches (manually or scheduled), detects changes, translates. -3. Translations appear on `intl/pending` as a PR against `dev`. -4. Reviewer checks the translation PR, merges when satisfied. -5. For component deprecations: remove from English first, let the pipeline strip it from locales (via removed content handling), then a cleanup job can safely delete the component file. - -### Hot Fixes - -Hot fixes to `staging` or `master` are not automatically translated. They go out in English-only. Translations catch up on the next release cycle when `dev` (with translations) merges to `staging` via prepare-release. If a hot fix translation is truly urgent, the pipeline can be manually dispatched with `base_branch=staging` and a custom `translation_branch`, but this is not the standard flow. - -### Recovery - -**Bad translation (not yet merged):** Re-run the pipeline targeting the specific file+locale. New commit overwrites the bad translation on the staging branch. - -**Bad translation (already merged to dev):** Re-run with `mode: full` for that file. Fresh translation + manifest stamped. - -**Corrupted manifests:** Delete the manifest files for the affected locale. Pipeline auto-detects "no manifest" and does full translation with fresh manifest generation. - -**Nuclear recovery:** Delete all manifests for a locale and re-run full. Equivalent to a fresh translation sweep. Expensive but always safe. - -## Key Design Decisions - -- **Manifests are cheap, translations are expensive.** The architecture makes it easy to regenerate manifests and hard to lose good translations. -- **English is the source of truth.** Non-English files should never be edited manually. The pipeline is the exclusive manipulator. -- **Inert propagation avoids unnecessary LLM calls.** URL changes, path updates, and attribute changes are handled deterministically -- no Gemini tokens spent. -- **Section-level granularity.** Only changed sections are retranslated, with unchanged sections provided as context. This preserves voice consistency while minimizing cost. -- **One translation PR at a time.** The `intl/pending` branch ensures there's never more than one open translation PR, avoiding manifest conflicts. - -## File Locations - -- Pipeline entry: `src/scripts/i18n/main-incremental.ts` -- Full pipeline: `src/scripts/i18n/main-gemini.ts` -- Manifest adapter: `src/scripts/i18n/lib/ai/manifest-adapter.ts` -- Inert propagation: `src/scripts/i18n/lib/ai/propagate-inert.ts` -- Incremental translate: `src/scripts/i18n/lib/ai/incremental-translate.ts` -- Branch utilities: `src/scripts/i18n/lib/github/branches.ts` -- Workflow: `.github/workflows/gemini-translations.yml` -- Content tree package: `intl-content-tree` (npm, MPL-2.0) diff --git a/docs/locales-process.md b/docs/locales-process.md deleted file mode 100644 index 049a9ee5b386..000000000000 --- a/docs/locales-process.md +++ /dev/null @@ -1,9 +0,0 @@ -# Locales generation process - -Every time `pnpm build` or `pnpm start` is executed, the following process is -going to be triggered as well: - - - -With this process, we reduce the amount of text we bundle on each page since we -are querying only the necessary translations that each page needs. diff --git a/docs/solutions/architecture/llms-txt-automation.md b/docs/solutions/architecture/llms-txt-automation.md new file mode 100644 index 000000000000..822a1ee6b67b --- /dev/null +++ b/docs/solutions/architecture/llms-txt-automation.md @@ -0,0 +1,73 @@ +--- +title: "Automated llms.txt and developers/docs/llms.txt Generation" +date: 2026-05-20 +category: architecture +module: app/llms.txt, app/developers/docs/llms.txt, src/lib/llms-txt +tags: + - llms-txt + - seo + - automation + - nav +problem_type: "feature, automation, content-pipeline" +--- + +# Automated `llms.txt` Generation + +Two `force-static` App Router routes replace the hand-maintained `public/llms.txt`. They regenerate on every deploy; no manual maintenance. + +- `ethereum.org/llms.txt` — full site index, organized by main-nav top sections. +- `ethereum.org/developers/docs/llms.txt` — developer-docs-only index, organized by the docs sidebar. + +The split mirrors `nextjs.org/llms.txt` + `nextjs.org/docs/llms.txt`: the root file points at the docs file rather than inlining 100+ lines of deeply nested developer docs. + +## Strategy + +### What appears, and where it lives + +| Decision | Choice | +| ------------------ | ---------------------------------------------------------------------------------------------- | +| Which pages appear | Driven by nav files only (main nav, Footer, developer-docs YAML). Never walks `public/content/`. | +| Section structure | Mirrors the main-nav top sections 1:1 (Learn / Use / Build / Participate / Research) + Legal & Policies from Footer's secondary links. | +| Per-item label | The nav file's label (resolved via the existing i18n JSON). | +| Per-item URL | Always the page's pretty URL (`https://ethereum.org/{href}`). Never `/content/*/index.md`. | +| Per-item description | First non-empty of: page's frontmatter `description` → nav's description → label only. | +| Locale | English only at root. Per-locale variants are a later, opt-in extension (no code refactor needed). | + +### Per-section layout + +``` +## {section.label} ← top sections from main nav + +- {top-level leaf items} ← e.g. Overview, Quizzes, Videos + +### {sub-group label} ← e.g. Ethereum Explained, How Ethereum Works +- {sub-group items} + +### More ← Footer items not already in main nav, dedup'd by href +- ... +``` + +### Root file vs docs file + +- **Root file (`/llms.txt`)** treats the developer docs as one pointer (the four top-level Documentation entries from main nav) and links out to `/developers/docs/llms.txt` for the full tree. +- **Docs file (`/developers/docs/llms.txt`)** renders `developer-docs-links.yaml` directly — top groups as `##`, nested items as indented bullets at the depth they sit in the YAML. + +This keeps the root file scannable (~170 lines) and lets crawlers that want depth follow the cross-link. + +## Sources of truth + +| Source | Provides | +| --------------------------------------------- | ------------------------------------------------------------------------------------------ | +| `src/lib/nav/buildNavigation.ts` | Main-nav top sections + sub-groups + items + their nav descriptions. | +| `src/lib/nav/footerLinks.ts` | Footer link sections + dipper links (Legal & Policies). Extracted so Footer and llms.txt share one source. | +| `src/data/developer-docs-links.yaml` | The docs sidebar tree, including nested items. | +| `src/intl/en/common.json` + `page-developers-docs.json` | English labels and descriptions for the i18n keys above (via `getTranslations`). | +| `public/content/{slug}/index.md` frontmatter | The richer per-page `description` used preferentially over the nav description. | + +To change what appears in `llms.txt`, edit one of the sources above. The output regenerates on the next deploy. The generated `.txt` files are not in the repo — there is nothing to hand-edit. + +## Failure handling + +- Missing i18n key → falls back to the key string via the site-wide `getMessageFallback`. Same behavior as anywhere else on the site. +- Frontmatter `description` missing or unreadable → falls back to the nav description, then to label only. Never throws. +- Nav `href` points at a page with no `index.md` (JSX-only landings, external URLs) → no frontmatter lookup, nav description carries the entry. diff --git a/docs/solutions/build-errors/crowdin-translation-sanitizer-mdx-fence-bugs.md b/docs/solutions/build-errors/crowdin-translation-sanitizer-mdx-fence-bugs.md deleted file mode 100644 index e00ccc2cd8cf..000000000000 --- a/docs/solutions/build-errors/crowdin-translation-sanitizer-mdx-fence-bugs.md +++ /dev/null @@ -1,283 +0,0 @@ -# Crowdin Translation Sanitizer: MDX Build Failures from Backslash Injection and Code Fence Drift - -> **Date:** 2026-02-27 -> **PR:** #17125 (French Crowdin import) -> **Component:** post-import translation sanitizer (`src/scripts/i18n/post_import_sanitize.ts`) -> **Problem type:** build-error, translation-artifact -> **Languages affected:** fr (confirmed); potentially any Crowdin-imported language -> **Severity:** Critical -- both patterns cause Netlify build failures (MDX compilation errors) -> **Tags:** crowdin, sanitizer, mdx, build-error, code-fence, backslash, translation - -## Problem Symptom - -The Netlify build for PR #17125 (French translation import, ~300 files) failed with MDX compilation errors in three files: - -1. `public/content/translations/fr/ai-agents/index.md` -- "Unexpected character" at backslash before closing tag -2. `public/content/translations/fr/restaking/index.md` -- same backslash pattern (2 occurrences) -3. `public/content/translations/fr/developers/tutorials/reverse-engineering-a-contract/index.md` -- "Unexpected character `[`" from code fence boundaries being completely inverted - -These are two distinct Crowdin translation artifacts that both manifest as MDX compilation failures. - -## Root Cause Analysis - -### Pattern 12: Backslash Before Closing HTML Tag - -**What happens:** Crowdin's translation memory or post-processing inserts a backslash `\` immediately before closing HTML tags. This produces patterns like: - -``` -Bon a savoir\ -``` - -The backslash is not valid in MDX/JSX context and causes the MDX compiler to choke. - -**Why it happens:** Crowdin's TM system treats ``, ``, ``, `

`, and even JSX fragment closers ``. - -**Real-world occurrences found in FR import:** -- `fr/ai-agents/index.md` line 67: `Bon a savoir\` -- `fr/restaking/index.md` lines 42, 99: same pattern -- `fr/developers/tutorials/creating-a-wagmi-ui-for-your-contract/index.md` line 146: `\` - -### Pattern 13: Catastrophic Code Fence Drift - -**What happens:** Crowdin completely scrambles the boundaries between code fences and prose in translation files containing significant code blocks. The result is: - -- Code that should be inside fences ends up as raw MDX prose (causing compilation errors) -- Prose/comments that should be outside fences get absorbed into code blocks -- Heading lines (`## Title {#anchor-id}`) get merged into adjacent prose paragraphs -- Anchor IDs (`{#custom-id}`) become detached from their heading lines - -**Why it happens:** Crowdin's segmentation algorithm treats code fence markers (`` ``` ``) as segment boundaries. When translating, it can reassemble segments with fence markers in wrong positions. Files with many interleaved code/prose sections (like tutorials) are especially vulnerable. - -**Scale of damage in FR import:** In `reverse-engineering-a-contract/index.md`, approximately 165 lines (581-746) were structurally destroyed: -- 22 code fence markers displaced -- Multiple headings absorbed into prose paragraphs -- Python decompiler output exposed as raw MDX -- Anchor IDs detached from heading lines - -## Investigation Steps Tried - -1. **Direct inspection** -- Read the affected French files and compared against English sources line-by-line -2. **Pattern identification** -- Found the backslash pattern was consistent and regex-matchable; the code fence drift was structural and not auto-fixable -3. **English source comparison** -- Confirmed English sources had correct structure; damage was purely from Crowdin processing -4. **Cross-file check** -- Searched all FR files for both patterns to determine full scope - -## Working Solution - -### Fix 1: `fixBackslashBeforeClosingTag` (Deterministic Auto-Fix) - -Added to `src/scripts/i18n/post_import_sanitize.ts` at line 1625: - -```typescript -function fixBackslashBeforeClosingTag(content: string): { - content: string - fixCount: number -} { - let fixCount = 0 - - // Split content to preserve code blocks (fenced and inline) - const codeBlockPattern = /(```[\s\S]*?```|~~~[\s\S]*?~~~|`[^`]+`)/g - const parts = content.split(codeBlockPattern) - - for (let i = 0; i < parts.length; i++) { - if (i % 2 === 1) continue // Skip code blocks - - // Match backslash immediately before - parts[i] = parts[i].replace(/\\(<\/[a-zA-Z]*>)/g, (_match, tag) => { - fixCount++ - return tag - }) - } - - return { content: parts.join(""), fixCount } -} -``` - -**Wired into pipeline:** Placed before `removeOrphanedClosingTags` in `processMarkdownFile` via `applyFix`. - -**Regex explanation:** `\\(<\/[a-zA-Z]*>)` matches a literal backslash followed by ``. The capture group preserves the valid closing tag while discarding the backslash. - -### Fix 2: `warnCatastrophicCodeFenceDrift` (Detection + Warning) - -Added to `src/scripts/i18n/post_import_sanitize.ts` at line 497. This is a warning-only function because the damage is too structural to auto-fix -- it requires LLM-assisted reconstruction. - -The function performs three checks: - -1. **Prose-in-fences:** Compares each translated fence body against its English counterpart. If English has 3+ lines with code keywords but translated has <=2 lines with no code keywords, flags it. - -2. **Code-outside-fences:** Scans prose sections (outside fences) for programming keywords (`def `, `class `, `if `, `return `, `require `, etc.). If 3+ keyword occurrences found outside fences, flags catastrophic inversion. - -3. **Detached anchor IDs:** Checks that `{#anchor-id}` patterns only appear on heading lines (starting with `#`). Detached anchors indicate heading absorption. - -### Manual Reconstruction for Pattern 13 - -The `reverse-engineering-a-contract/index.md` file required full manual reconstruction of lines 581-746 using the English structural skeleton with French prose reinserted. This was done by an agent that: - -1. Used the English source as the structural template (fence positions, headings, anchor IDs) -2. Extracted translatable prose from the mangled French file -3. Rebuilt the file maintaining English code blocks verbatim -4. Verified: 22 fence markers balanced, all anchor IDs on heading lines - -## Tests Added - -### Standalone Fix Tests (`tests/unit/sanitizer/standalone-fixes.spec.ts`) - -7 new tests for `fixBackslashBeforeClosingTag`: - -| Test | Description | -|------|-------------| -| fixes backslash before closing strong tag | `\` -> `` | -| fixes backslash before closing em tag | `\` -> `` | -| fixes backslash before closing a tag | `\` -> `` | -| fixes multiple occurrences in one string | 2 fixes in single content block | -| leaves correct closing tags unchanged | No false positives on valid HTML | -| does not modify content inside code blocks | Preserves `\` inside backticks | -| fixes JSX fragment closer | `\` -> `` | - -### Warning Tests (`tests/unit/sanitizer/warnings.spec.ts`) - -5 new tests for `warnCatastrophicCodeFenceDrift`: - -| Test | Description | -|------|-------------| -| detects prose inside code fences | Prose replacing code triggers warning | -| detects code keywords outside fences | `def`, `return`, etc. in prose triggers warning | -| no warning on correctly structured files | Clean content produces no warnings | -| detects detached heading anchors | `{#id}` not on `##` line triggers warning | -| no false positive on properly anchored headings | Correct `## Heading {#id}` produces no warning | - -**Test results:** All 111 sanitizer tests pass (56 standalone + 22 warnings + 33 other). - -## Prevention Strategies - -### Short-term - -1. **Sanitizer pipeline catches Pattern 12 automatically** -- The `fixBackslashBeforeClosingTag` function runs on every Crowdin import, catching all occurrences without manual intervention. - -2. **Sanitizer warns on Pattern 13** -- The `warnCatastrophicCodeFenceDrift` function flags files with structural damage so they can be routed to LLM review. - -### Medium-term - -3. **Pre-import Crowdin configuration** -- Investigate Crowdin's "Code" content type settings to prevent code fence boundaries from being treated as translatable segments. - -4. **Expand code keyword detection** -- Add language-specific keywords (Solidity: `pragma`, `mapping`, `event`; JavaScript: `const`, `function`, `async`) to improve detection sensitivity. - -5. **Automated LLM reconstruction pipeline** -- When catastrophic drift is detected, automatically invoke an LLM agent to reconstruct the file from English structure + translated prose, similar to the manual process used here. - -### Long-term - -6. **Crowdin segment locking** -- Work with Crowdin to lock code fence markers and their contents as non-translatable segments, preventing the drift at source. - -7. **Structural hash comparison** -- Generate a structural hash (fence positions, heading levels, anchor IDs) for English source files and compare against translated files post-import to catch any structural divergence. - -## Files Changed - -| File | Change | -|------|--------| -| `src/scripts/i18n/post_import_sanitize.ts` | Added `fixBackslashBeforeClosingTag` (line 1625) and `warnCatastrophicCodeFenceDrift` (line 497); wired both into pipeline; added to `_testOnly` export | -| `tests/unit/sanitizer/standalone-fixes.spec.ts` | 7 new tests for backslash fix | -| `tests/unit/sanitizer/warnings.spec.ts` | 5 new tests for catastrophic drift detection | -| `docs/solutions/integration-issues/sanitizer-test-research.md` | Added patterns 12 and 13 to catalog; moved both to "handled" list | -| `public/content/translations/fr/ai-agents/index.md` | Fixed `\` at line 67 | -| `public/content/translations/fr/restaking/index.md` | Fixed `\` at lines 42 and 99 | -| `public/content/translations/fr/developers/tutorials/creating-a-wagmi-ui-for-your-contract/index.md` | Fixed `\` at line 146 | -| `public/content/translations/fr/developers/tutorials/reverse-engineering-a-contract/index.md` | Full structural reconstruction of lines 581-746 | - ---- - -## Part 2: Follow-Up Build Failures (Patterns 14-15) - -> **Date:** 2026-02-27 (same session, second build attempt) - -After the Part 1 fixes were pushed and built, two more MDX compilation errors surfaced in the same PR. - -### Pattern 14: Translated Word After Bare `<` Breaks MDX Tag Parsing - -**Symptom:** `Unexpected character [ (U+005B) in name` on `/fr/developers/tutorials/reverse-engineering-a-contract` - -**Root cause:** English has `\ { - fixCount++ - return `\\<${after}` -}) -``` - -The pattern `` - -**Symptom:** `Unexpected closing slash / in tag` on `/fr/developers/tutorials/creating-a-wagmi-ui-for-your-contract` - -**Root cause:** The Pattern 12 fix (from Part 1) used regex `[a-zA-Z]*` (zero or more letters), which matched `\` -- a JSX fragment with an empty tag name. Stripping the backslash exposed bare `` to the MDX parser. - -The `\` was actually a legitimate escape (placed by `escapeMdxAngleBrackets`) protecting a bare `` that sat outside inline code backticks due to a Crowdin backtick misplacement. - -**Fix:** Changed quantifier from `*` to `+`: - -```typescript -// Requires at least one letter in tag name -- leaves \ (JSX fragment) intact -parts[i] = parts[i].replace(/\\(<\/[a-zA-Z]+>)/g, (_, tag) => { -``` - -**Cascading effect:** Once `\` was preserved, the existing `repairUnclosedBackticks` function could detect the real problem (odd backtick count on line 146) and fix the misplaced backtick: `` (`<> ...` `) `` became `` (`<> ... `) ``. - -### Key Lesson: Sanitizer Function Interaction - -This was a cascading failure: -1. `fixBackslashBeforeClosingTag` (Pattern 12 fix) stripped `\` -- unmasking a bare `` in prose -2. That bare `` was actually the symptom of a deeper problem: a misplaced backtick -3. `repairUnclosedBackticks` could have fixed the backtick, but only if the escapes were intact -4. Narrowing the regex (Pattern 15 fix) preserved `\`, letting `repairUnclosedBackticks` do its job - -**Takeaway:** Sanitizer functions form a pipeline. Earlier functions that strip escapes can mask problems that later functions would fix. Each function should only strip escapes it's certain about -- use `+` not `*`, enumerate known tag names, and test with JSX fragments. - -### Tests Added (Part 2) - -6 new tests in `standalone-fixes.spec.ts`: - -| Test | Description | -|------|-------------| -| does NOT strip backslash from JSX fragment `\` | Verifies `[a-zA-Z]+` excludes empty tag names | -| escapes bare `<` before word containing `[` | ` `\` left alone | - -**Test results:** All 116 sanitizer tests pass (61 standalone + 22 warnings + 33 other). - -### Additional Files Changed (Part 2) - -| File | Change | -|------|--------| -| `src/scripts/i18n/post_import_sanitize.ts` | Extended `escapeMdxAngleBrackets` (+1 rule); narrowed `fixBackslashBeforeClosingTag` (`*` -> `+`) | -| `tests/unit/sanitizer/standalone-fixes.spec.ts` | 6 new tests | -| `docs/solutions/integration-issues/sanitizer-test-research.md` | Added patterns 14 and 15 | -| `public/content/translations/fr/.../reverse-engineering-a-contract/index.md` | ``, ``) as edge cases for any HTML tag regex -3. **Test idempotency** -- `sanitize(sanitize(x))` should equal `sanitize(x)` -4. **Test function composition** -- run functions in sequence and verify escapes survive -5. **Run `npx tsc --noEmit`** before pushing -- catches unused variables the test runner misses - -## Cross-References - -- [Sanitizer Test Research: Pattern Catalog](../integration-issues/sanitizer-test-research.md) -- Patterns 12-15 -- [Post-Import Sanitizer Bugs Found During Japanese Review](../integration-issues/post-import-sanitizer-bugs-found-japanese-review.md) -- Prior sanitizer bug documentation -- [Crowdin Import Review Agent Calibration](../integration-issues/crowdin-import-review-agent-calibration.md) -- Agent calibration for translation reviews -- [Crowdin File Path Mapping and Review Workflow](../integration-issues/crowdin-file-path-mapping-and-review-workflow.md) -- Full review pipeline documentation -- [French Import Review](../integration-issues/crowdin-french-import-review-pr-17125.md) -- Comprehensive FR import context -- PR #17125 -- French Crowdin import (source of these bugs) -- PR #17654 -- Sanitizer test infrastructure (where test framework was established) diff --git a/docs/solutions/build-errors/mdx-compilation-crowdin-backtick-split.md b/docs/solutions/build-errors/mdx-compilation-crowdin-backtick-split.md deleted file mode 100644 index 7c54bad2d42d..000000000000 --- a/docs/solutions/build-errors/mdx-compilation-crowdin-backtick-split.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -title: MDX Compilation Error from Split Backticks in Translated Angle Bracket Expressions -date: 2026-02-19 -category: build-errors -tags: - - MDX - - translations - - Crowdin - - angle-brackets - - backticks - - i18n - - next-mdx-remote -severity: high -component: next-mdx-remote / Crowdin translations -symptoms: - - "Netlify build fails with: Expected a closing tag for `` before the end of `paragraph`" - - Build failure on translated pages only (not English source) - - Error points to lines where angle brackets appear outside backtick code spans -root_cause: Crowdin translation splits backtick wrapping around code expressions containing angle brackets, leaving bare , , etc. that MDX interprets as unclosed JSX tags -resolution_time: quick ---- - -# MDX Compilation Error from Split Backticks in Translated Angle Bracket Expressions - -## Problem Symptom - -Netlify build fails during static page generation with repeated errors like: - -``` -[next-mdx-remote] error compiling MDX: -Expected a closing tag for `` (436:204-436:207) before the end of `paragraph` -``` - -Affected pages (this instance): -- `/cs/developers/tutorials/ai-trading-agent` -- `/ja/developers/tutorials/ai-trading-agent` -- `/pt-br/developers/tutorials/ai-trading-agent` -- `/ur/developers/tutorials/ai-trading-agent` - -## Root Cause Analysis - -The English source uses backtick-wrapped code expressions containing angle brackets: - -```markdown -which in a C-derived language would be ` ? : `. -``` - -During Crowdin translation, translators inadvertently split the backtick pair. The backtick closes after `?`, leaving `` and `` as bare text: - -```markdown -# Broken (backtick closes too early): -který by v jazyce odvozeném od C byl ` ?` : `. - -# Fixed (single backtick pair wraps all angle brackets): -který by v jazyce odvozeném od C byl ` ? : `. -``` - -MDX (via `next-mdx-remote`) interprets bare `` and `` as opening JSX/HTML tags without corresponding closing tags, causing compilation failure. - -## Solution - -Re-wrap the angle-bracketed expressions inside a single backtick pair in each affected translation file. - -### Files Modified - -| Language | File | Line | -|----------|------|------| -| Czech | `public/content/translations/cs/developers/tutorials/ai-trading-agent/index.md` | 446 | -| Japanese | `public/content/translations/ja/developers/tutorials/ai-trading-agent/index.md` | 445 | -| Portuguese-BR | `public/content/translations/pt-br/developers/tutorials/ai-trading-agent/index.md` | 447 | -| Urdu | `public/content/translations/ur/developers/tutorials/ai-trading-agent/index.md` | 446 | - -### Before/After - -**Czech:** -```diff -- ` ?` : ` -+ ` ? : ` -``` - -**Japanese:** -```diff -- ` ?` : ` -+ ` ? : ` -``` - -**Portuguese-BR:** -```diff -- ` ?` : `.` -+ ` ? : `. -``` -(Also removed stray extra backtick at end) - -**Urdu:** -```diff -- ` ?` ہوگا : ` -+ ` ? : ` ہوگا -``` -(Reordered so angle brackets stay inside backticks) - -## Related Context - -### Prior Occurrences - -- **Commit `76675a5717`** ("fix: escape angle brackets in translated MDX files") — Fixed the same class of issue across 18 languages in an earlier import. This confirms it is a recurring pattern. -- **Commit `3ef2bbb91e`** ("i18n: post-import sanitization") — Ran the existing sanitizer on the same tutorial content. -- **Commit `01fd093439`** ("fix(i18n): post_import_sanitize on ai-trading-agents") — Latest sanitizer run, which did not catch this particular pattern. - -### Existing Sanitizer - -The project has `src/scripts/i18n/post_import_sanitize.ts` with functions like `fixAsciiGuillemets()` that handle `<<`/`>>` conversions. However, it does **not** currently detect split backtick wrapping around single angle bracket expressions like ``, ``, ``. - -## Prevention Strategies - -### 1. Add Sanitizer Rule (Recommended) - -Add a `fixBareAngleBrackets()` function to `post_import_sanitize.ts` that: -- Scans each line for bare `` patterns outside of backtick spans and code fences -- Compares with the English source to find the intended backtick-wrapped version -- Auto-fixes when the match is unambiguous; flags for manual review otherwise - -### 2. Pre-Build Validation - -Add a validation step that checks all translated MDX files for bare angle brackets outside code contexts before the build runs. This would catch issues before they reach Netlify. - -### 3. Crowdin Configuration - -- Lock inline code patterns containing angle brackets as "Do not translate" segments -- Add context notes warning translators about preserving backtick pairs around code expressions - -## Verification - -After applying fixes, confirm with a full build: - -```bash -pnpm build -``` - -Check that the 4 previously-failing pages no longer produce MDX compilation errors. diff --git a/docs/solutions/integration-issues/crowdin-file-path-mapping-and-review-workflow.md b/docs/solutions/integration-issues/crowdin-file-path-mapping-and-review-workflow.md deleted file mode 100644 index ef42edc003b8..000000000000 --- a/docs/solutions/integration-issues/crowdin-file-path-mapping-and-review-workflow.md +++ /dev/null @@ -1,323 +0,0 @@ ---- -title: "Crowdin File Path Mapping Bugs and Translation Review Workflow" -date: 2026-02-21 -category: integration-issues -tags: - - crowdin - - translations - - i18n - - file-path-matching - - sanitizer - - worktree - - automation - - permissions -severity: high -component: crowdin-import-pipeline -symptoms: - - "Translated files placed at incorrect paths (e.g., cs/beacon-chain/ instead of cs/roadmap/beacon-chain/)" - - "Systematic translation errors: AI replaced with UI, semantic inversions" - - "Orphaned translation files with no corresponding English source" - - "Inaccurate fix count logging in sanitizer output" - - "Cross-block href replacements affecting wrong sections" - - "Build failures in worktrees due to missing .env.local" - - "Merge conflicts discovered only at push time" -root_causes: - - "findCrowdinFile() used .endsWith() for path matching, producing false matches on similarly named paths" - - "Crowdin/Gemini translation engine confusing acronyms and inverting meaning" - - "Sanitizer tracked in-memory transforms instead of actual disk changes" - - "Href replacement applied globally instead of per-block" - - "No .env.local in worktrees (USE_MOCK_DATA not set)" - - "PR branches diverged from dev without early merge" -status: solved -related_prs: - - 17547 - - 17553 - - 17556 - - 17182 ---- - -# Crowdin File Path Mapping Bugs and Translation Review Workflow - -## Problem Summary - -During Phase 2 of the translation review pipeline (Czech pilot), we discovered that 12 Czech translation files were placed at incorrect filesystem paths. Investigation revealed the root cause in the Crowdin import workflow's path matching logic. Additionally, we established a reproducible worktree-based workflow for reviewing translation PRs and cataloged all permissions needed for automation. - -## Root Cause Analysis - -### Misplaced Translation Files - -**File:** `src/scripts/i18n/lib/crowdin/files.ts` - -The `findCrowdinFile()` function used `.endsWith()` to match Crowdin file paths against expected content paths: - -```ts -// BROKEN: matches too broadly -const found = crowdinFiles.find(({ path }) => - path.endsWith(targetFile.filePath) -) -``` - -When looking for `public/content/roadmap/beacon-chain/index.md`, this matched `cs/beacon-chain/index.md` because the suffix `beacon-chain/index.md` is valid for both. The `roadmap/` parent directory was silently ignored. - -**Data flow of the bug:** - -``` -GitHub file path findCrowdinFile() processedFileIdToPath Download destination -public/content/roadmap/ --> .endsWith() matches --> Stores wrong Crowdin --> cs/beacon-chain/ - beacon-chain/index.md cs/beacon-chain/ path for this fileId index.md (WRONG) -``` - -**12 Czech files affected:** - -| Wrong Location | Correct Location | -|---|---| -| `cs/account-abstraction/` | `cs/roadmap/account-abstraction/` | -| `cs/beacon-chain/` | `cs/roadmap/beacon-chain/` | -| `cs/danksharding/` | `cs/roadmap/danksharding/` | -| `cs/future-proofing/` | `cs/roadmap/future-proofing/` | -| `cs/scaling/` | `cs/roadmap/scaling/` | -| `cs/statelessness/` | `cs/roadmap/statelessness/` | -| `cs/user-experience/` | `cs/roadmap/user-experience/` | -| `cs/withdrawals/` | `cs/staking/withdrawals/` | -| `cs/dvt/` | `cs/staking/dvt/` | -| `cs/support/` | `cs/community/support/` | -| `cs/code-of-conduct/` | `cs/community/code-of-conduct/` | -| `cs/developers/docs/wrapped-eth/` | `cs/wrapped-eth/` | - -### Translation Quality Issues - -Crowdin/Gemini translation engine produced two categories of critical error: - -1. **Acronym confusion**: "AI" systematically replaced with "UI" (5 instances in `cs/ai-agents/index.md`) -2. **Semantic inversion**: "malicious intent" translated as "good intentions" in `cs/bridges/index.md` - -### Sanitizer Logging Inaccuracy - -Individual fix functions returned `fixCount` based on in-memory transforms, not actual disk changes. Reported "22 files modified" when no bytes were written. - -### Cross-Block Href Interference - -`fixTranslatedHrefs()` applied replacements globally. When `/developers/docs/evm` appeared in both an EVM block and an Oracles block, the global replacement changed the correct href in the EVM block. - -## Solutions Implemented - -### Fix 1: Stricter Path Matching in findCrowdinFile() - -**File:** `src/scripts/i18n/lib/crowdin/files.ts` -**Branch:** `fix-i18n-workflow` - -```ts -// 1. Exact match first (after normalizing leading slashes) -const exactMatch = crowdinFiles.find( - ({ path }) => path.replace(/^\/+/, "") === normalizedTarget -) -if (exactMatch) return exactMatch - -// 2. Suffix match with "/" boundary guard -const suffixMatches = crowdinFiles.filter(({ path }) => { - const normalized = path.replace(/^\/+/, "") - if (!normalized.endsWith(normalizedTarget)) return false - const prefixLength = normalized.length - normalizedTarget.length - if (prefixLength === 0) return true - return normalized[prefixLength - 1] === "/" -}) - -// 3. Prefer longest (most specific) match -suffixMatches.sort((a, b) => b.path.length - a.path.length) -return suffixMatches[0] ?? null -``` - -### Fix 2: Orphan Detection in Sanitizer - -**File:** `src/scripts/i18n/post_import_sanitize.ts` -**Branch:** `fix-review-translations` - -For each translated file, derives the expected English source path and checks existence. If missing, searches by filename to suggest the correct location: - -```ts -const englishPath = path.join(CONTENT_ROOT, relPathWithinLang) - -if (!fs.existsSync(englishPath)) { - // Search for matching parent/file pattern in English content - const englishContentFiles = listFiles(CONTENT_ROOT, (f) => { - if (f.includes(`${path.sep}translations${path.sep}`)) return false - return f.endsWith(`${path.sep}${parentDir}${path.sep}${basename}`) - }) - - if (englishContentFiles.length === 1) { - suggestion = `Likely belongs at: ${correctTranslationPath}` - } else if (englishContentFiles.length > 1) { - suggestion = `Ambiguous: ${englishContentFiles.length} candidates` - } -} -``` - -### Fix 3: Accurate Disk-Write Tracking - -**File:** `src/scripts/i18n/post_import_sanitize.ts` - -Added `applyFix()` helper that snapshots content before/after each transform, plus `originalOnDisk` comparison: - -```ts -function applyFix( - fn: () => { content: string; fixCount: number }, - label: (count: number) => string -) { - const snapshot = content - const result = fn() - content = result.content - if (content !== snapshot) { - issues.push(label(result.fixCount)) - } -} -``` - -### Fix 4: Block-Scoped Href Replacement - -**File:** `src/scripts/i18n/post_import_sanitize.ts` - -Track `blockIdx` from detection phase, apply replacements only within the specific block: - -```ts -// Detection phase -blockFixes.push({ blockIdx: i, wrong: translatedHref, correct: expectedHref }) - -// Replacement phase - scoped to block -for (const { blockIdx, wrong, correct } of blockFixes) { - const originalBlock = translatedBlocks[blockIdx] - let fixedBlock = originalBlock.replace(markdownRe, `$1${correct}$2`) - if (fixedBlock !== originalBlock) { - result = result.replace(originalBlock, fixedBlock) - } -} -``` - -## Worktree Workflow for Translation Review - -Reproducible 8-step sequence for reviewing a translation PR: - -```bash -# 1. Create worktree from PR branch -git worktree add .worktrees/ -cd .worktrees/ - -# 2. Provide environment variables (USE_MOCK_DATA=true avoids network calls) -cp .env.example .env.local - -# 3. Merge latest dev to catch conflicts early -git fetch origin dev && git merge origin/dev -# Resolve conflicts — typically modify/delete for misplaced files - -# 4. Copy sanitizer scripts from canonical branch (until merged to dev) -# Also add franc-min to package.json devDependencies -cp /src/scripts/i18n/post_import_sanitize.ts ./src/scripts/i18n/ -cp /src/scripts/i18n/lib/workflows/sanitization.ts ./src/scripts/i18n/lib/workflows/ - -# 5. Install dependencies -pnpm install - -# 6. Run sanitizer for orphan detection -TARGET_LANGUAGES= npx ts-node -O '{"module":"commonjs"}' ./src/scripts/i18n/post_import_sanitize.ts - -# 7. Run review (critical issues only — no soft suggestions) -# Use /review-translations-local --pr= --language= - -# 8. Validate build -npx tsc --noEmit # TypeScript check FIRST -NEXT_PUBLIC_BUILD_LOCALES=en, pnpm build # Scoped build -``` - -### Key Notes - -- **Always run `npx tsc --noEmit` before `pnpm build`** — catches type errors cheaply -- **`.env.local` is mandatory** — without it, build attempts real API connections and fails -- **Merge dev early** — resolving conflicts before review prevents wasted work -- **Merge conflicts are expected** — misplaced files from prior imports cause modify/delete conflicts; accept the deletion -- **`franc-min` is required** — ESM-only package, needs devDependency until sanitizer changes reach dev - -### Tool Reliability: `diff` Command - -During cs-part-07 review, `diff` returned empty output comparing two files that were verifiably different (confirmed by reading both files and re-running `diff` with identical arguments, which then returned correct output). Root cause unknown — not conclusively a sandbox issue since the second run succeeded with the same arguments. - -**For automation, do not trust empty `diff` output as proof of file equality.** Mitigations: -- Check `diff` exit code explicitly (`0` = identical, `1` = different, `2` = error) -- Use `diff --brief` for a quick same/different check before assuming equality -- When comparing files for migration decisions (orphan dedup), read and verify content directly if `diff` returns empty - -## Automation Permissions Required - -All sandbox-restricted operations needed for this workflow: - -### Git Operations - -| Command | Purpose | -|---|---| -| `git worktree add/remove` | Create/destroy isolated review environments | -| `git fetch origin` | Retrieve latest upstream branches | -| `git merge origin/dev` | Integrate dev into PR branch | -| `git stash push/pop` | Temporarily shelve local edits | -| `git rm` | Remove orphaned/misplaced files | -| `git add` / `git commit` | Stage and commit fixes | -| `git push` | Push corrected branch to remote | - -### Package Management - -| Command | Purpose | -|---|---| -| `pnpm install` | Install dependencies (network + node_modules writes) | - -### Script Execution - -| Command | Purpose | -|---|---| -| `npx tsc --noEmit` | TypeScript check without emitting | -| `npx ts-node