Skip to content
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
cdf407e
chore(telemetry-worker): add telemetry ingest worker
allxsmith Aug 22, 2026
408e6b4
ci: harden the skills-roster check and derive skill discovery once
allxsmith Aug 22, 2026
e59c0cd
docs: repair skills-roster provenance pointers and bundling claims
allxsmith Aug 22, 2026
805f90a
docs: add telemetry disclosure guide
allxsmith Aug 22, 2026
1e5b488
feat(create-bestax): add opt-in anonymous telemetry
allxsmith Aug 22, 2026
20c5492
feat(bestax-migrate): add opt-in anonymous telemetry
allxsmith Aug 22, 2026
b6b1a1f
feat(bestax-mcp): attribute emitted docs links with utm_source
allxsmith Aug 22, 2026
453e11b
refactor(create-bestax): extract shared telemetry kernel and await pa…
allxsmith Aug 22, 2026
c4c7426
refactor(bestax-migrate): share telemetry kernel with create-bestax
allxsmith Aug 22, 2026
1ef43ce
fix(bestax-mcp): keep URL fragments when attributing docs links
allxsmith Aug 22, 2026
7f350fc
docs: note prop: slugs in the telemetry disclosure
allxsmith Aug 22, 2026
5029bd2
chore: align telemetry ingest with CLI payloads and gate it in CI
allxsmith Aug 22, 2026
decba35
Merge branch 'fix/541-skills-roster-hardening' into feat/telemetry
allxsmith Aug 23, 2026
8def8da
fix(create-bestax): gate consent on both TTYs, honor DNT and Ctrl-C
allxsmith Aug 23, 2026
271ba4c
fix(bestax-migrate): stop sending changedBucket, honor Ctrl-C at consent
allxsmith Aug 23, 2026
493aad2
chore(telemetry-worker): cap todo counts, derive buckets, stream the …
allxsmith Aug 23, 2026
b3b2a31
docs: true up the telemetry disclosure
allxsmith Aug 23, 2026
9cf0c23
chore: normalize utm_source out of eval docs metrics; list telemetry-…
allxsmith Aug 23, 2026
da27d67
chore: add telemetry-worker to the pnpm workspace
allxsmith Aug 23, 2026
1ecb5ee
ci: deploy the telemetry worker from main
allxsmith Aug 23, 2026
a6db102
fix(create-bestax): never send telemetry from the e2e scaffold harness
allxsmith Aug 23, 2026
3a430bc
ci: make the skills vetting gate repo-aware and file-granular
allxsmith Aug 23, 2026
d2e568b
ci: import telemetry producer enums instead of scraping, close scope …
allxsmith Aug 23, 2026
7bf859f
ci: harden-runner on the worker deploy job
allxsmith Aug 23, 2026
71b18ba
chore: guard the ts resolver hook and the eval utm strip
allxsmith Aug 23, 2026
06b3653
fix(bestax-migrate): name the wire caps and pin them to the worker
allxsmith Aug 23, 2026
00545b4
docs: true up deploy, backstop, and utm-coverage claims
allxsmith Aug 23, 2026
750b53e
fix(create-bestax): review-thread fixes across the telemetry surface
allxsmith Aug 23, 2026
81df919
fix(create-bestax): fail closed on foreign consent records, gate igno…
allxsmith Aug 23, 2026
ec0aae3
ci: deploy wrangler from the workspace lockfile; dedupe worker tasks
allxsmith Aug 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions .github/workflows/deploy-worker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Deploys telemetry-worker to Cloudflare Workers on every main-branch change
# under telemetry-worker/**, mirroring the Pages deploy in deploy.yml: build
# happens here (wrangler bundles src/index.ts), Cloudflare only receives the
# artifact. workflow_dispatch allows a manual redeploy from the Actions tab.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
#
# The only credential in this job is CLOUDFLARE_WORKERS_DEPLOY_TOKEN, an API
# token scoped to Workers Scripts:Edit + Workers Routes:Edit on bestax.io —
# deliberately separate from CLOUDFLARE_API_TOKEN (Pages) so each pipeline
# holds only what it deploys.
name: Deploy telemetry worker

on:
push:
branches:
- main
paths:
- 'telemetry-worker/**'
workflow_dispatch:
Comment thread
allxsmith marked this conversation as resolved.
Resolved

permissions:
contents: read

# Serialize deploys: two quick main pushes must not race — an older run
# finishing last would ship stale worker code. No cancel-in-progress: a
# deploy already uploading should finish, then the newer one runs.
concurrency:
group: deploy-worker
cancel-in-progress: false

jobs:
deploy:
name: Deploy to Cloudflare Workers
runs-on: ubuntu-latest
steps:
# Rule 10 (.github/CLAUDE.md): new jobs ship with egress-policy: block —
# this is the job holding the Cloudflare deploy token. Endpoints:
# checkout (github.com trio), wrangler install (npm registry), and the
# upload itself (api.cloudflare.com). Wrangler's own metrics beacon is
# disabled below instead of allow-listing its host.
- name: Harden runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
api.cloudflare.com:443
api.github.com:443
github.com:443
objects.githubusercontent.com:443
registry.npmjs.org:443

- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
Comment thread
allxsmith marked this conversation as resolved.
Resolved
Comment thread
allxsmith marked this conversation as resolved.
Resolved
with:
# No later step needs authenticated git; don't leave the token in
# the local git config for the rest of the job.
persist-credentials: false

# No pnpm/Node setup: the worker is stdlib-only TypeScript with no build
# step of its own — wrangler-action installs the pinned wrangler and its
# bundler does the rest.
- name: Deploy with wrangler
uses: cloudflare/wrangler-action@ebbaa1584979971c8614a24965b4405ff95890e0 # v4
Comment thread
allxsmith marked this conversation as resolved.
Resolved
env:
WRANGLER_SEND_METRICS: 'false'
with:
apiToken: ${{ secrets.CLOUDFLARE_WORKERS_DEPLOY_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
workingDirectory: telemetry-worker
wranglerVersion: '4.124.0'
command: deploy
3 changes: 3 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ React component library for **Bulma v1** in TypeScript. pnpm monorepo orchestrat
- `bestax-mcp/` — the `bestax-mcp` MCP server; its `data/` index is **generated**
(has its own CLAUDE.md)
- `skills/` — Agent Skills, a **shipped product** bundled into create-bestax (has its own CLAUDE.md)
- `telemetry-worker/` — Cloudflare Worker ingesting the CLIs' opt-in telemetry
(deployed from CI by `deploy-worker.yml` — a merged change under it ships to
production immediately)
- `eval/agent-loop/` — cold-start eval harness: does an unassisted agent build correctly when
given one of our guidance channels (the skills, the MCP server, both)? Frozen rubric, its
own README
Expand Down
6 changes: 4 additions & 2 deletions bestax-mcp/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,10 @@ the API pages and the skill catalog.
spends the context the client needs for the actual task.
- **No hardcoded roster.** Prompts, resources and skill lookups are all derived
from `data/skills.json`, which the generator builds by reading the `skills/`
directory. `skills/CLAUDE.md` already counts five places the roster is
duplicated; do not add a sixth.
directory. The rule repo-wide is derive, don't list (#540); the prose copies
that cannot be derived are held to the directory by the `skills-roster`
conformance check (`pnpm check:conformance --only=skills-roster`). Do not add
a hand-maintained copy here for that check to chase.
- **Nothing on stdout but JSON-RPC.** stdout _is_ the transport. Diagnostics go
to stderr — a stray `console.log` surfaces to the user as an unexplained
disconnect.
Expand Down
27 changes: 21 additions & 6 deletions bestax-mcp/scripts/sync-skills.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,10 @@ import { createHash } from 'node:crypto';
import { setTimeout as sleep } from 'node:timers/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import {
readSkillNames,
untrackedSkillPaths,
} from '../../scripts/lib/skills.mjs';

const here = path.dirname(fileURLToPath(import.meta.url));
const pkgRoot = path.resolve(here, '..');
Expand All @@ -68,18 +72,26 @@ if (!existsSync(skillsSrc)) {
process.exit(1);
}

const names = (await readdir(skillsSrc, { withFileTypes: true }))
.filter(
e => e.isDirectory() && existsSync(path.join(skillsSrc, e.name, 'SKILL.md'))
)
.map(e => e.name)
.sort();
// The shared predicate (scripts/lib/skills.mjs): a directory holding a
// SKILL.md, sorted deterministically — one definition for all four consumers.
const names = await readSkillNames(skillsSrc);

if (!names.length) {
console.error(`[sync-skills] no skills found in ${skillsSrc}`);
process.exit(1);
}

// Same vetting gate as create-bestax's sync: data/skills ships in the npm
// tarball, so an untracked scratch skill must not reach a manual publish.
const untrackedHere = untrackedSkillPaths(skillsSrc, names);
if (untrackedHere.length) {
console.error(
`[sync-skills] refusing to bundle untracked file(s) under skills/: ` +
`${untrackedHere.join(', ')}. \`git add\` them to vet them, or remove them.`
);
process.exit(1);
}

// Hash of every source path and its bytes. Content rather than mtime on
// purpose: a fresh clone or a branch switch rewrites mtimes without changing
// what should be copied, and re-copying is the cheap half — it is the
Expand Down Expand Up @@ -205,6 +217,9 @@ try {
for (const name of names) {
await cp(path.join(skillsSrc, name), path.join(skillsDest, name), {
recursive: true,
// .DS_Store is the one path the vetting gate exempts; keep it out of
// the bundle too so the exemption never becomes shipped content.
filter: src => !src.endsWith('.DS_Store'),
});
}
// Stamped last: a run killed mid-copy leaves no stamp, so the next caller
Expand Down
87 changes: 87 additions & 0 deletions bestax-mcp/src/__tests__/format.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
/**
* Unit tests for `attributed`, the link tagger.
*
* The server is offline by design (openWorldHint: false), so the tag on its
* outbound links is the only attribution signal it has — these pin the
* function's contract. The emission paths themselves are asserted where the
* links surface: server.test.ts for the component link line, version.test.ts
* for the drift note.
*/
import { describe, expect, it } from '@jest/globals';

import { loadComponent } from '../data.js';
import { attributed, renderComponent } from '../format.js';

describe('attributed', () => {
it('appends with ? when the URL has no query string', () => {
expect(attributed('https://bestax.io/docs/api/elements/button')).toBe(
'https://bestax.io/docs/api/elements/button?utm_source=bestax-mcp'
);
});

it('appends with & when the URL already has one, as Storybook links do', () => {
expect(
attributed('https://bestax.io/storybook/?path=/story/elements-button')
).toBe(
'https://bestax.io/storybook/?path=/story/elements-button&utm_source=bestax-mcp'
);
});

it('is idempotent', () => {
const once = attributed('https://bestax.io/docs');
expect(attributed(once)).toBe(once);
});

it('returns an already-tagged URL unchanged', () => {
const tagged = 'https://bestax.io/docs?utm_source=bestax-mcp';
expect(attributed(tagged)).toBe(tagged);
});

it('is not fooled by the tag text inside a fragment', () => {
// The analytics request never sees the fragment, so this URL is NOT
// attributed yet — the already-tagged check must look at the query only.
expect(attributed('https://bestax.io/docs#utm_source=bestax-mcp')).toBe(
'https://bestax.io/docs?utm_source=bestax-mcp#utm_source=bestax-mcp'
);
});

it('is not fooled by the tag text inside another parameter value', () => {
const url = 'https://bestax.io/docs?ref=utm_source=bestax-mcp';
expect(attributed(url)).toBe(`${url}&utm_source=bestax-mcp`);
});

it('passes non-http(s) strings through unchanged', () => {
for (const notAUrl of ['bestax://catalog', 'mailto:a@b.c', 'Button', '']) {
expect(attributed(notAUrl)).toBe(notAUrl);
}
});

it('keeps a fragment after the query string', () => {
expect(attributed('https://bestax.io/docs/guides/foo#bar')).toBe(
'https://bestax.io/docs/guides/foo?utm_source=bestax-mcp#bar'
);
});

it('inserts ahead of the fragment when a query string is already present', () => {
expect(
attributed(
'https://bestax.io/storybook/?path=/story/elements-button#anchor'
)
).toBe(
'https://bestax.io/storybook/?path=/story/elements-button&utm_source=bestax-mcp#anchor'
);
});
});

describe('renderComponent link line', () => {
it('tags both the docs and Storybook links', async () => {
const record = await loadComponent('Button');
const out = renderComponent(record, ['props']);
// Against the committed index: the docs URL carries no query string, the
// Storybook URL carries `?path=...` — so this exercises both separators.
expect(out).toContain(`Docs: ${record.docsUrl}?utm_source=bestax-mcp`);
expect(out).toContain(
`Storybook: ${record.storybook}&utm_source=bestax-mcp`
);
});
});
8 changes: 8 additions & 0 deletions bestax-mcp/src/__tests__/server.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,14 @@ describe('get_component', () => {
expect(failed(res)).toBe(true);
expect(text(res)).toContain('Button');
});

// The server itself never goes online; the utm tag on the links it hands out
// is what makes a follow-through visible in the bestax.io analytics.
it('tags the emitted docs and Storybook links for attribution', async () => {
const out = text(await call('get_component', { name: 'Button' }));
expect(out).toMatch(/Docs: https:\/\/\S+\?utm_source=bestax-mcp/);
expect(out).toMatch(/Storybook: https:\/\/\S+&utm_source=bestax-mcp/);
});
});

describe('get_props', () => {
Expand Down
5 changes: 5 additions & 0 deletions bestax-mcp/src/__tests__/version.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,11 @@ describe('versionNote', () => {
expect(note).toContain('4.0.0');
});

it('tags the docs link it points at, like every emitted link', () => {
const note = versionNote(info({ installed: '4.0.0', drift: 'major' }));
expect(note).toContain('https://bestax.io/docs?utm_source=bestax-mcp');
});

it('warns on a minor mismatch, where props can move', () => {
expect(versionNote(info({ installed: '5.2.0', drift: 'minor' }))).toContain(
'⚠'
Expand Down
37 changes: 35 additions & 2 deletions bestax-mcp/src/format.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,37 @@ const cell = (text: string) =>
.replace(/\r?\n/g, ' ')
.trim();

/**
* Tag an outbound link with `utm_source=bestax-mcp` so a docs or Storybook
* visit that started here shows up as such in the site analytics. Render-time
* only — the URLs in `data/` are generated and stay canonical. Idempotent, and
* anything that is not an http(s) URL passes through unchanged.
*
* Coverage is deliberately partial: only links this server composes itself —
* the Docs/Storybook footer on component responses and the version-drift
* notice — go through here. Skill bodies and reference docs are served
* verbatim from the bundled markdown, untagged links included, because
* rewriting URLs inside arbitrary markdown/code examples risks corrupting
* them. Do not "fix" that by running this over served markdown.
*/
export function attributed(url: string): string {
if (!/^https?:\/\//.test(url)) return url;
// Split the fragment FIRST: the already-tagged check below must inspect
// only the query. A `#utm_source=bestax-mcp` inside a fragment (or the
// same text inside another parameter's value) is not attribution — the
// analytics request never sees it — so matching it would skip tagging.
// Do not use URL.searchParams: it re-encodes Storybook `path=/story/...`
// as `%2F`.
const hashIndex = url.indexOf('#');
const hash = hashIndex === -1 ? '' : url.slice(hashIndex);
const withoutHash = hashIndex === -1 ? url : url.slice(0, hashIndex);
const queryIndex = withoutHash.indexOf('?');
const query = queryIndex === -1 ? '' : withoutHash.slice(queryIndex + 1);
if (query.split('&').includes('utm_source=bestax-mcp')) return url;
const sep = queryIndex === -1 ? '?' : '&';
return `${withoutHash}${sep}utm_source=bestax-mcp${hash}`;
}

export function table(headers: string[], rows: string[][]): string {
if (!rows.length) return '';
return [
Expand Down Expand Up @@ -145,8 +176,10 @@ export function renderComponent(
out.push(`**Related:** ${record.related.map(r => `\`${r}\``).join(', ')}.`);
}

const links = [`Docs: ${record.docsUrl}`];
if (record.storybook) links.push(`Storybook: ${record.storybook}`);
const links = [`Docs: ${attributed(record.docsUrl)}`];
if (record.storybook) {
links.push(`Storybook: ${attributed(record.storybook)}`);
}
out.push(links.join(' · '));
return out.join('\n\n');
}
Expand Down
6 changes: 4 additions & 2 deletions bestax-mcp/src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -778,8 +778,10 @@ export async function createServer(

// -- Prompts, one per skill ----------------------------------------------
//
// Generated from the manifest, never listed here. skills/CLAUDE.md counts the
// roster as already duplicated in five places; this would have been a sixth.
// Generated from the manifest, never listed here. The roster is derived, not
// hand-listed (#540); the prose copies that cannot be derived are held to the
// directory by the skills-roster conformance check (`pnpm check:conformance
// --only=skills-roster`). A list here would be one more copy for it to chase.
for (const skill of skills) {
server.registerPrompt(
skill.promptName,
Expand Down
4 changes: 3 additions & 1 deletion bestax-mcp/src/version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
import { readFile } from 'node:fs/promises';
import { dirname, join, parse } from 'node:path';

import { attributed } from './format.js';

const PACKAGE = '@allxsmith/bestax-bulma';

export interface VersionInfo {
Expand Down Expand Up @@ -171,6 +173,6 @@ export function versionNote(info: VersionInfo): string | null {
return (
`⚠ This index documents ${PACKAGE} ${info.indexed}; this project has ` +
`${info.installed}. Props, defaults and CSS variables may differ — check ` +
`https://bestax.io/docs before relying on anything above.`
`${attributed('https://bestax.io/docs')} before relying on anything above.`
);
}
6 changes: 6 additions & 0 deletions bestax-migrate/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ each source library registers in `src/sources/registry.ts`; the first is
(package.json updater; never runs an install), `jsx-utils.ts` (AST helpers).
- Components with no bestax equivalent (Element, Tile) keep a trimmed, TODO-annotated RBC
import so the code still runs during gradual migration.
- `src/telemetry-core.ts` is a byte-for-byte copy of
`create-bestax/src/telemetry-core.ts` — never edit it here; edit the
create-bestax original and copy it over (`check:conformance
--only=telemetry-core` enforces identity). This package's payload builder and
caps live in `src/telemetry.ts`; the caps are pinned to the ingest worker's
bounds by `scripts/telemetry-contract.test.mjs`.

## Testing

Expand Down
13 changes: 13 additions & 0 deletions bestax-migrate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,19 @@ A codemod rewrites your source in place, so how it is built and published matter

Full detail: [`SECURITY.md`](https://github.com/allxsmith/bestax/blob/main/SECURITY.md) · [Security guide](https://bestax.io/docs/guides/security)

## Telemetry

`bestax-migrate` can send one **anonymous** usage event after a successful
run — only if you opt in when asked (once, on an interactive terminal). The
event is just the run's shape — source library, CSS mode, the dry/deps flags, a
capped changed-file count, and per-rule TODO counts — plus the CLI version,
Node major version, and OS platform name. Never file paths, file contents,
code, names, IPs, or machine IDs, and no identifier exists that could link two
events together. Opt out any time with `--no-telemetry`,
`BESTAX_TELEMETRY=0`, or `DO_NOT_TRACK=1`.

Full disclosure of every field and control: [Telemetry guide](https://bestax.io/docs/guides/telemetry)

## License

MIT © Alex Smith
3 changes: 3 additions & 0 deletions bestax-migrate/jest.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ export default {
'!src/**/__tests__/**',
'!src/**/__testfixtures__/**',
'!src/index.ts', // bin entry — exercised via the e2e/CLI tests, not unit-covered
// Byte-identical to create-bestax; covered there. Drift fails
// `check:conformance --only=telemetry-core`.
'!src/telemetry-core.ts',
],
coverageThreshold: {
global: {
Expand Down
Loading
Loading