Repository navigation
fix(social): endpoint discovery was taking three of the decoys - #132
Conversation
Ran the site's sender against the IndieWeb conformance suite at webmention.rocks, which plants deliberate traps in its 23 discovery pages. Three of them caught us. All three were the same shape: the parser was matching on text rather than on structure. #12 rel="not-webmention" took the decoy #13 decoy inside <!-- --> took the decoy #20 <link rel="webmention"> gave up instead of reading on 12 is the sharp one. The old test was /\bwebmention\b/, and "-" is a word boundary, so "not-webmention" matched. rel is a space-separated token list, so the check is now tokenize-and-compare, in the header path too. 13 needed comments stripped before the scan (escaped HTML in #14 was already safe, since these patterns want a literal "<"). 20 was treating the first rel=webmention tag as the answer even with no href, so a bare <link> shadowed the real <a> further down; it now skips candidates without an href and keeps looking. href="" is deliberately NOT that case: it is a legal self-reference, and #15 asserts it. Kept the single document-order scan. Tempting to look at every <link> before any <a>, but #16 exists precisely to punish that: it puts the real endpoint in an <a> and a decoy in a later <link>. 23/23 against the live suite now, up from 20. Frozen as fixtures rather than live fetches, because this file is deliberately network-free. Confirmed the fixtures fail against the old parser. discoverEndpoint is exported to make the suite runnable, matching the other helpers here that are already exported for tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Understanding-first reviewThis is a prompt, not a score or merge gate. Before approving, reconstruct the change:
For docs and dependency changes, a compact answer is enough. For behavior or production changes, include the causal path and one concrete counterexample. The author’s claim card belongs in the PR description; this prompt is for the reviewer’s independent reconstruction. |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
aadhar-sh | 7ed3bfd | Jul 27 2026, 06:21 PM |
Drove an actual webmention through the deployed receiver from a public source page. It verified and stored correctly, but the moderation email was 280 characters of SVG path geometry: 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018…"> A throwaway source page. It exists to drive one real w Two causes, either of which is enough on its own. <svg> was not stripped alongside <script> and <style>, and a page like a GitHub gist is full of inline icons. On its own that is survivable, since the tag stripper removes complete tags with their attributes. What made it escape is the excerpt window. It is a fixed 400 characters either side of the target, cut at a byte offset, and the target lives inside an href — so centring on a link GUARANTEES the cut lands mid-tag, and often mid-attribute. A leading partial tag is not a complete tag, so the stripper leaves it as text. That is the stray "> in the middle. Now strips <svg>/<noscript> and drops the partial tags at both edges, detecting a mid-tag cut by whether > precedes < in the slice. Same page now yields the actual sentence around the link. Also fixes two things found alongside it: 202 no longer carries a Location header. The spec ties Location to 201, where it names a status URL the sender can poll; webmention.rocks receiver test #1 fails a 202 that sends one. /inbox was never a status URL for a pending mention anyway, since a pending mention deliberately does not appear there. Citations now skip anchors that exist to open a hover card: interestfor on the garage pages, class="car-link" on the homepage. Every one of them points at a Google SEARCH url, so telling google.com/search?q=Singer it had been cited is noise, not credit. Live citation targets drop 56 -> 53, and all three removed are search pages. Verified the excerpt test fails with both fixes removed, reproducing the exact garbage above. Each fix alone is sufficient, so it takes removing both to break it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ran both halves against real receivers. Four more bugs.Outbound is now proven end to endDrove the site's own
#13 and #20 are the two this PR fixes. Before it, #13 would have POSTed to Inbound is proven end to endFull lifecycle against the deployed worker plus a local run for the signed-approval half:
The three new fixes in the latest commitThe excerpt was 280 characters of SVG path data. A real mention produced 202 carried a Location header. The spec ties Location to 201, where it names a pollable status URL. Receiver test #1 fails a 202 that sends one, and Hover-card anchors counted as citations. Verification
Still open, and it is the real "looks like spam" problemOur mention rendered on their page as "No Name / Comment text not found", because the source had no microformats. Every page we send from has the same gap:
So every webmention this site sends will show up on other people's sites as an anonymous, textless link. Fixing it means |
1 participant
Ran the outbound sender against webmention.rocks, the IndieWeb conformance suite, whose 23 discovery pages plant deliberate traps. Three caught us. All three were the same shape: matching on text instead of on structure.
rel="not-webmention"<!-- --><link rel="webmention">with no href#12 is the sharp one. The check was
/\bwebmention\b/, and-is a word boundary, sonot-webmentionmatched.relis a space-separated token list, so it is now tokenize-and-compare, in the header path too.#13 needed comments stripped before the scan. Escaped HTML (#14) was already safe, since these patterns want a literal
<.#20 treated the first
rel=webmentiontag as the answer even with nohref, so a bare<link>shadowed the real<a>below it. It now skips href-less candidates and keeps looking.href=""is deliberately not that case: it is a legal self-reference, and #15 asserts it still resolves to the page.The refactor I didn't do
Scanning every
<link>before any<a>reads cleaner and is wrong. #16 exists to punish exactly that: the real endpoint is in an<a>, with a decoy in a later<link>, and the spec says first-in-document-order. The single-pass scan stays.Verification
contract-tests.mjsis deliberately network-free. Each fixture is lifted from the real page and labelled with its test number.#12asserts first) and pass against the new one.node build.mjsexit 0,perf-budgetgreen.discoverEndpointis now exported so the suite can drive it, matchingcitationsIn/findEndpointIn/contentOf, already exported for tests.Worth knowing
None of the site's 56 current citation targets advertise an endpoint, so this code path has never actually fired in production and tomorrow's cron will log
0 sent. That is correct behaviour, but it means the bug would have sat here silently until the first time the site cited a webmention-aware page.🤖 Generated with Claude Code
Claim card
Intended model. Two independent things are changing, and the boundary is "the code that talks to strangers". Outbound: endpoint discovery parses
relas a token list, ignores HTML comments, and skips href-less candidates. Inbound: the excerpt stops leaking markup, and the 202 stops carrying aLocation. They ride together because one testing session against real receivers surfaced all four, and separating them would mean re-running the same live suite twice.Prediction. A sender pointed at any of the 23 webmention.rocks discovery pages resolves the endpoint those pages call correct, including the three that plant decoys. A mention received from a page full of inline SVG stores the sentence around the link rather than path geometry.
POST /webmentionanswers202with noLocation. The daily cron's target count drops from 56 to 53, and the three that disappear are allgoogle.com/searchURLs.Likely misconception or failure mode. The tempting refactor is to scan every
<link>before any<a>, since the spec lists them in that order. That is wrong, and discovery test #16 is built to punish it: the real endpoint sits in an<a>with a decoy in a later<link>, and the rule is first-in-document-order. The single-pass scan is deliberate, not accidental.The second misconception is that stripping
<svg>was the excerpt fix. It is the smaller half. A complete<svg>tag is already removed by the tag stripper, attributes and all. The leak needs the 400-character window to cut into thed="…"attribute, which centring on anhrefmakes near-certain. Remove only the svg strip and the bug still hides; the fixture demonstrates this.Evidence. 23/23 against the live suite, up from 20/23, then frozen as fixtures because this file is network-free. Four real POSTs to webmention.rocks returned 200 and the comments appear on their pages, which is acceptance rather than a queued 200, since the receiver verifies the backlink first. Inbound walked the full lifecycle against the deployed worker: 202, verified, pending, absent from
/inbox, forged sig 403, signed approve 200, rendered, then retracted to an empty inbox. 35/35 tests; the excerpt test reproduces the exact production garbage when both fixes are removed.Remaining uncertainty. Three things.
The excerpt fix is heuristic. A mid-tag cut is detected by whether
>precedes<, which mis-fires on prose containing a bare>before any tag. The cost is a slightly short excerpt, not corruption, andEXCERPT_MAXalready truncates.The citation filter now encodes two markup conventions (
interestfor,class="car-link"). A third hover-card idiom would silently count as a citation again. Nothing tests that the site has not invented one.Largest: this PR does not make our own mentions render well on other sites. The live send showed up as "No Name / Comment text not found", because every page we send from carries no microformats. That is the actual "looks like spam" problem and it is untouched here.