Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config/infra.json
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@
"id": "markdown-compressed",
"target": "markdown-twin",
"assert": { "compressionPrefers": { "offer": "gzip, deflate, br, zstd", "expect": "br" } },
"why": "Cloudflare's default content-type list carries text/x-markdown, the pre-RFC name, and NOT text/markdown, the RFC 7763 type this site serves. So every Markdown surface shipped completely uncompressed until the Compression Rule went in: /index.md, /auth.md, /whoareyou.md, /bot.md and the www/md/ twins, 12,302 bytes that brotli takes to 5,943 — a 52% loss, measured 2026-07-28. It landed on exactly the surfaces built for agents and LLM crawlers, which fetch the .md representation by preference, and nothing in the tree noticed because no check had ever looked at a content type other than the homepage's. Serving text/x-markdown instead would win compression from the default list and is the wrong fix: it is deprecated, and agents expect the registered type. This check is the tripwire for the whole class, so if a new content type ever appears here, give it a target and a line.",
"why": "Cloudflare's default content-type list carries text/x-markdown, the pre-RFC name, and NOT text/markdown, the RFC 7763 type this site serves. So every Markdown surface shipped completely uncompressed until the Compression Rule went in: /index.md, /auth.md, /whoareyou.md, /bot.md and the src/content/md/ twins, 12,302 bytes that brotli takes to 5,943 — a 52% loss, measured 2026-07-28. It landed on exactly the surfaces built for agents and LLM crawlers, which fetch the .md representation by preference, and nothing in the tree noticed because no check had ever looked at a content type other than the homepage's. Serving text/x-markdown instead would win compression from the default list and is the wrong fix: it is deprecated, and agents expect the registered type. This check is the tripwire for the whole class, so if a new content type ever appears here, give it a target and a line.",
"pairs_with": "compression-prefers-brotli. That one asserts WHICH encoding is chosen; this one asserts that any is chosen at all. A rule scoped to Cloudflare's default content types would satisfy the first and fail this one."
},
{
Expand Down
2 changes: 1 addition & 1 deletion config/site-manifest.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"version": 1,
"note": "The single registry of aadhar.sh's public routable surfaces. Authored by hand. tools/gen-manifest.mjs derives the shell/sitemap/agent projections from it (pnpm run gen:manifest), and build.mjs invariant #8 fails the deploy on drift. Leaf content that already has its own registry is deliberately NOT duplicated here: writing posts live in www/writing/posts.json, photos in www/images/manifest.json, resume artifacts are static files. Flags map to one surface each so drift is checkable: run=nav.js PAGES (Run palette), taskbar=nav.js SUBPAGES (pinned apps), sitemap=sitemap.xml <loc>, gallery=www/garage/index.html shelf, agents=MCP resources/list projection, webmention=accepts an inbound Webmention (a section so flagged also vouches for its leaf posts in posts.json); searchIndex=generate-search-index.mjs manual injection (for worker-rendered pages the source-tree walk cannot see; static pages are auto-indexed regardless).",
"note": "The single registry of aadhar.sh's public routable surfaces. Authored by hand. tools/gen-manifest.mjs derives the shell/sitemap/agent projections from it (pnpm run gen:manifest), and build.mjs invariant #8 fails the deploy on drift. Leaf content that already has its own registry is deliberately NOT duplicated here: writing posts live in src/content/writing/posts.json, photos in www/images/manifest.json, resume artifacts are static files. Flags map to one surface each so drift is checkable: run=nav.js PAGES (Run palette), taskbar=nav.js SUBPAGES (pinned apps), sitemap=sitemap.xml <loc>, gallery=www/garage/index.html shelf, agents=MCP resources/list projection, webmention=accepts an inbound Webmention (a section so flagged also vouches for its leaf posts in posts.json); searchIndex=generate-search-index.mjs manual injection (for worker-rendered pages the source-tree walk cannot see; static pages are auto-indexed regardless).",
"surfaces": [
{ "path": "/", "title": "Home", "section": "home", "kind": "page",
"description": "The aadhar.sh homepage: now-playing tracks, a photo grid, and the XP desktop.",
Expand Down
4 changes: 2 additions & 2 deletions docs/MAINTENANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -507,7 +507,7 @@ render, wider than the effect being measured):
*Hole.* Cloudflare's default content-type list carries `text/x-markdown`, the
pre-RFC name, and **not** `text/markdown`, the RFC 7763 type this site correctly
serves. So every Markdown surface shipped uncompressed — `/index.md`, `/auth.md`,
`/whoareyou.md`, `/bot.md`, and the `www/md/` twins. 12,302 bytes that brotli
`/whoareyou.md`, `/bot.md`, and the `src/content/md/` twins. 12,302 bytes that brotli
takes to 5,943, a 52% loss on exactly the surfaces built for agents and LLM
crawlers, which prefer the `.md` representation. Serving `text/x-markdown` to win
compression is the wrong fix: that is a deprecated type, and agents expect the
Expand Down Expand Up @@ -1713,7 +1713,7 @@ curl -s https://aadhar.sh/garage/llms.txt | head

The first two must both answer `text/markdown`; a browser `Accept` header and a
bare `*/*` must both still get `text/html`. `/bot` and `/whoareyou` are the two
twins written BY HAND (in `www/md/`), because those pages render from Worker
twins written BY HAND (in `src/content/md/`), because those pages render from Worker
template literals no build step can read. Edit either page and the deploy fails
until its twin agrees: `checkTwinFacts()` recomposes the User-Agent from
`botauth.js`'s own constants and requires it verbatim in `bot.md`, so a
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion src/worker/bot.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ export async function handleBotPage(request, env, ctx) {
// This is the page a stranger reads after finding AadharshBot in their logs,
// and that stranger is increasingly an agent. Answer in Markdown when asked;
// /bot.md is the stable, cacheable URL for the same bytes. The twin is
// hand-authored (www/md/bot.md) because this page renders from a template
// hand-authored (src/content/md/bot.md) because this page renders from a template
// literal, and build.mjs fails the deploy if it drifts from the constants here.
if (wantsMarkdown(request)) {
const md = await serveMarkdownTwin(request, env, "/bot.md");
Expand Down
2 changes: 1 addition & 1 deletion src/worker/rn.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ export const RN_FALLBACK = "https://open.spotify.com/playlist/4IRq9W1N2tOWHhH0O3
//
// So /rn answers Markdown instead of redirecting, and the answer is RENDERED
// from the same live payload /rn/tracks serves rather than described in a twin
// under www/md/. There is nothing fixed here to hand-author: the playlist
// under src/content/md/. There is nothing fixed here to hand-author: the playlist
// changes, and a file claiming otherwise would be wrong within a rollover. This
// also cannot drift by construction, which is the property the twin machinery
// buys with checkTwinFacts.
Expand Down
2 changes: 1 addition & 1 deletion src/worker/security.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import { esc, wantsMarkdown } from "./lib/http.js";
export async function handleSecurityCenter(request, env) {
// The prose here is static; only the three connection values below are
// per-request, and the twin says so and points at /whoareyou.json for them.
// Hand-authored (www/md/security.md) because this page renders from a
// Hand-authored (src/content/md/security.md) because this page renders from a
// template literal, and build.mjs fails the deploy if the twin drifts from
// lib/security.js, which is where the header values it quotes actually live.
// x-robots-tag rides along because the HTML sets it: a Markdown rendering of a
Expand Down
7 changes: 7 additions & 0 deletions tools/build.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -637,6 +637,13 @@ await cp("src/worker", `${OUT}/src/worker`, { recursive: true });
// back to the ROOT of the served tree because their public URLs are /nav.js and
// /luna.css. Source layout and URL layout are different questions; only the first
// one moved, so every served byte and every /a/ content hash is untouched.
// Authored PROSE and the registries beside it: the writing posts and their
// posts.json, the hand-written Markdown twins for the Worker-rendered pages, and
// the four root .md documents. They stage back into the served tree at the SAME
// paths, so /writing/<slug>.txt, /index.md and the rest answer exactly where they
// did. Source layout and URL layout are different questions, and only the first
// one moved.
await cp("src/content", `${OUT}/www`, { recursive: true });
await cp("src/client", `${OUT}/www`, { recursive: true });
await cp("src/styles", `${OUT}/www`, { recursive: true });
await mkdir(`${OUT}/cal`, { recursive: true });
Expand Down
2 changes: 1 addition & 1 deletion tools/check-infra.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -1170,7 +1170,7 @@ async function checkCodeScanning(repo, slug, token) {

// Which pages answer an agent in Markdown, measured on the wire rather than inferred
// from filenames. The twins arrive by three different conventions — /index.md for the
// homepage, www/md/<name>.md for /whoareyou and /bot, build-generated twins for
// homepage, src/content/md/<name>.md for /whoareyou and /bot, build-generated twins for
// /garage/* and /lwe/* — so a local file check would have to know all three and would
// still be guessing about production. One request per page settles it.
//
Expand Down
4 changes: 2 additions & 2 deletions tools/contract-tests.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -2676,7 +2676,7 @@ test("robots.txt never forbids a path the site advertises to agents", async () =
"_headers": await read("www/_headers"),
".well-known/api-catalog": await read("www/.well-known/api-catalog"),
"agent-card.json": await read("www/.well-known/agent-card.json"),
"auth.md": await read("www/auth.md"),
"auth.md": await read("src/content/auth.md"),
"llms.txt": await read("www/llms.txt"),
};

Expand All @@ -2702,7 +2702,7 @@ test("browser RUM and its ledger proxy stay fully removed", async () => {
const wranglerDev = await readFile(new URL("wrangler.dev.jsonc", ROOT), "utf8");
const headers = await readFile(new URL("www/_headers", ROOT), "utf8");
const whoareyou = await readFile(new URL("src/worker/whoareyou.js", ROOT), "utf8");
const whoareyouMd = await readFile(new URL("www/md/whoareyou.md", ROOT), "utf8");
const whoareyouMd = await readFile(new URL("src/content/md/whoareyou.md", ROOT), "utf8");
const securityPage = await readFile(new URL("src/worker/security.js", ROOT), "utf8");

// The loader, both route legs, both asset allowlists, and the dedicated module
Expand Down
2 changes: 1 addition & 1 deletion tools/gen-feeds.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ export const FEEDS = [
export function buildFeeds(root = ".") {
const manifest = JSON.parse(readFileSync(`${root}/config/site-manifest.json`, "utf8"));
const dates = sitemapDates(readFileSync(`${root}/www/sitemap.xml`, "utf8"));
const posts = JSON.parse(readFileSync(`${root}/www/writing/posts.json`, "utf8"));
const posts = JSON.parse(readFileSync(`${root}/src/content/writing/posts.json`, "utf8"));

// /writing entries are not registered surfaces (a post is a row in posts.json,
// not a manifest page), so that section reads its own registry and carries its
Expand Down
19 changes: 12 additions & 7 deletions tools/gen-md-twins.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
// WHAT IS AUTHORED BY HAND
// /bot, /whoareyou and /security render from the Worker, so their prose lives in
// template literals rather than a file this script can read. Their twins are
// authored in www/md/, and checkTwinFacts() below pins the load-bearing
// authored in src/content/md/, and checkTwinFacts() below pins the load-bearing
// strings so the twin cannot quietly disagree with the page.
//
// Every exported function is PURE so build.mjs can re-run this in-memory; only
Expand All @@ -28,7 +28,7 @@ import { readDocument } from "./lib/html-to-md.mjs";
export const ORIGIN = "https://aadhar.sh";
const MANIFEST = "config/site-manifest.json";
const SITEMAP = "www/sitemap.xml";
const HAND_DIR = "www/md";
const HAND_DIR = "src/content/md";

// the sections that earn their own index. A section index is only worth a fetch
// when the section is big enough that the root llms.txt is the wrong grain.
Expand Down Expand Up @@ -165,7 +165,7 @@ export function renderSectionIndex(section, surfaces, { descriptions = {} } = {}
* strip.
*/
export function renderWritingIndex(root = ".") {
const posts = JSON.parse(readFileSync(join(root, "www/writing/posts.json"), "utf8"));
const posts = JSON.parse(readFileSync(join(root, "src/content/writing/posts.json"), "utf8"));
const rows = [...posts]
.sort((a, b) => String(b.date || "").localeCompare(String(a.date || "")))
.map((p) => `- **${p.date}** [${p.title}](${ORIGIN}/writing/${p.slug}) ([plain text](${ORIGIN}/writing/${p.slug}.txt))`)
Expand Down Expand Up @@ -197,7 +197,12 @@ export function buildTwins(root = ".", opts = {}) {
// www/index.html would; it also already ships as a static asset and
// already answers `Accept: text/markdown`. Generating over it would be a
// regression dressed up as consistency.
if (existsSync(join(root, "www", twinPath(surface.path).replace(/^\//, "")))) {
// Looked in www/ until the prose moved: the homepage twin is authored at
// src/content/index.md now and STAGES to /index.md, so the check follows the
// source rather than the served path. Reading the old location silently
// stopped finding it, and the symptom was the hand-written twin being
// overwritten by a generated one that is 472 bytes shorter.
if (existsSync(join(root, "src/content", twinPath(surface.path).replace(/^\//, "")))) {
descriptions[surface.path] = surface.description || "";
continue;
}
Expand Down Expand Up @@ -266,7 +271,7 @@ export function buildTwins(root = ".", opts = {}) {
*/
export const TWIN_FACTS = [
{
twin: "www/md/bot.md",
twin: "src/content/md/bot.md",
facts: [
{
label: "User-Agent",
Expand All @@ -285,7 +290,7 @@ export const TWIN_FACTS = [
],
},
{
twin: "www/md/whoareyou.md",
twin: "src/content/md/whoareyou.md",
facts: [
{ label: "JSON endpoint", source: "src/worker/whoareyou.js", string: "/whoareyou.json" },
{ label: "no-storage claim", source: "src/worker/whoareyou.js", string: "none of it is stored" },
Expand All @@ -302,7 +307,7 @@ export const TWIN_FACTS = [
// those headers would only prove the twin agrees with prose that is itself
// free to rot. Every fact below is read from lib/security.js, the module that
// actually sends them, except the JWKS path, which is the page's own claim.
twin: "www/md/security.md",
twin: "src/content/md/security.md",
facts: [
{ label: "frame-ancestors", source: "src/worker/lib/security.js", string: "frame-ancestors 'none'" },
{ label: "object-src", source: "src/worker/lib/security.js", string: "object-src 'none'" },
Expand Down
2 changes: 1 addition & 1 deletion tools/lib/dependency-docs.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
// exemptions carrying their reason; and a pure core with negative tests.
//
// Same shape as checkTwinFacts() in gen-md-twins.mjs, which pins the
// load-bearing strings in www/md/*.md against the Worker in both directions.
// load-bearing strings in src/content/md/*.md against the Worker in both directions.
// Same shape as check-tools.mjs #375 in intent, with one difference worth
// keeping in mind: tools.json had to DECLARE what no manifest recorded, while
// every number here already lives in package.json or requirements.txt. So this
Expand Down
2 changes: 1 addition & 1 deletion tools/verify-routes.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ const ROUTES = [
{ path: "/whoareyou.json", status: 200, ct: "application/json" },
{ path: "/security", status: 200, ct: "text/html" },
// /security is static prose about the headers, so it earns a hand twin
// (www/md/security.md). Both halves asserted for the same reason the
// (src/content/md/security.md). Both halves asserted for the same reason the
// generated ones below are: the .md URL proves the build staged it, the
// negotiated form proves handleSecurityCenter reaches it.
{ path: "/security.md", status: 200, ct: "text/markdown", marker: "Security Center" },
Expand Down
Loading