[docs] Reorder the docs around the reliability loop, and give the cloud an onboarding path - #687
Closed
NiveditJain wants to merge 7 commits into
Closed
[docs] Reorder the docs around the reliability loop, and give the cloud an onboarding path#687NiveditJain wants to merge 7 commits into
NiveditJain wants to merge 7 commits into
Conversation
NiveditJain
added a commit
that referenced
this pull request
Aug 12, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZHcJRck9qdLpGEZeL51js
|
Important Review skippedToo many files! This PR contains 1287 files, which is 1137 over the limit of 150. To get a review, reduce the PR to 150 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. Usage-priced reviews support at most 300 files. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (24)
📒 Files selected for processing (1287)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Hermes
No summary yet. What this changesNo component map for this revision. RoundsNo review has finished on this pull request yet. FindingsNothing raised yet.
|
Contributor
Hermes
Found four documentation correctness issues: the custom-agent path cannot upload events as written, the legacy history page names a nonexistent dashboard route, the 14 localized sidebars omit the new deployment task, and the README understates the policy count. What this changesflowchart LR
n0Documentationnavigation["~ Documentation navigation"]
n1Cloudonboarding["+ Cloud onboarding"]
n2Cloudreference["+ Cloud reference"]
n3Localizeddocumentation["~ Localized documentation"]
n4Docstranslationtooling["~ Docs translation tooling"]
n5CLIhelpandREADME["~ CLI help and README"]
n0Documentationnavigation -- "routes readers to tasks" --> n1Cloudonboarding
n0Documentationnavigation -- "lists localized pages" --> n3Localizeddocumentation
n1Cloudonboarding -- "links to setup details" --> n2Cloudreference
n3Localizeddocumentation -- "uses localized Cloud routes" --> n2Cloudreference
n5CLIhelpandREADME -- "links users into docs" --> n0Documentationnavigation
Rounds
FindingsOpen
Resolved
|
hermes-exosphere
previously approved these changes
Aug 12, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found no blocking issues in this revision.
2 advisory findings
- Low/High Restore localized labels for the new navigation — Every non-English entry now renders the tab as "Documentation" and groups such as "Guardrails", "See what happened", and "Analyze and act" in English (for example docs/docs.json:144-218 for zh). The generator falls back to the English text for any unmapped name; its map only adds "Start here" and "CLI reference" (scripts/translate-docs/mintlify-nav.ts:86-108). Thus regenerating localized navigation will preserve this regression. (
scripts/translate-docs/mintlify-nav.ts:86) - Low/High Document that plain uninstall can retain the daemon — The new reference says
failproofai uninstallremoves the service and lists service removal as an unconditional step (docs/cli/uninstall.mdx:13-14,49-50). In the implementation, an interactive non-purge uninstall asks separately and keeps the service when declined (src/hooks/uninstall-cli.ts:305-313); the CLI help states this explicitly (bin/failproofai.mjs:994-998). (docs/cli/uninstall.mdx:13)
Contributor
|
I could not establish complete review coverage for What the review did establish: The documentation migration’s navigation, redirects, and internal links resolve across all 825 documentation pages. One edited public README entry still reports the wrong built-in-policy count. Full MDX compilation and Vitest verification could not run because dependencies were unavailable in the isolated container. Re-run with |
chhhee10
changed the title
This was referenced Aug 13, 2026
The site had two top-level tabs, "Enforcement" and "Observability", which asked every reader to work out which half of a single product they were in before reading anything. There is now one navigation, ordered as a journey: install and guard a machine, see what happened on it, then govern the fleet from the cloud. The observability half is repositioned as FailproofAI Cloud and moves from the agenteye section to the cloud section, with a redirect for every old URL. Documents the integration surface that had no docs at all. `failproofai config --connect` existed only in --help: new pages cover connecting a machine (both capabilities, what leaves the machine, fleet provisioning, troubleshooting), managed policies (deployments, digest verification, observe-mode rollout), the fleet view, and session capture across all 12 CLIs -- replacing three per-CLI capture pages written for a separate collector that the daemon has since absorbed. Adds the pages a reader kept needing and could not find: How it works (tool call to decision to dashboard, with the failure-mode table), the failproofaid service (fail-closed, supervision, how the binary arrives, upgrades), Supported agents (per-CLI matrix of what a deny actually blocks, derived from the same source the product asserts against), Policies as a hub, one merged Concepts glossary, a files-and-paths reference, and CLI pages for config, harness, backfill, flush and uninstall. Cloud pages are value- and contract-first: internal architecture claims are out, what a user needs to operate the product stays. Per-CLI hook-schema internals move off the Configuration page and into the support matrix. Two tests hardcoded the old docs directory as a fixture path and are updated to the new location. The 14 language trees are moved and relinked; pages with no counterpart are seeded from English so every nav entry resolves, and the scheduled translation job replaces them on its next run (its cache is keyed on English source hashes, all of which changed). Adds one screenshot of the local session viewer, captured through headless Chrome against synthetic fixture data rather than a real machine's sessions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZHcJRck9qdLpGEZeL51js
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZHcJRck9qdLpGEZeL51js
The restructure was sound; its migration and several of its claims were not. Redirects covered English only. All 14 localized trees lost 30 URLs each with no redirect at all, and 13 redirects that were live on main had been rewritten at the source rather than the destination — so the URLs readers actually hold stopped resolving while 27 new entries pointed from paths that had never been valid. Coverage is now complete in all 15 trees, and 1,792 links in translated pages that pointed into English were localized. Anchors are derived from what Mintlify actually renders, which keeps the em dash rather than stripping it. The claims were checked against the code rather than against each other: the built-in count (40, not 39), the Recommended preset (15, not 16), the hook exit codes, credentials.json, audit/cache/, logs/hooks.log, the deny string, that allows are logged, and that state/ holds the undelivered upload spool and must not be deleted. The sanitize policies detect secrets but on ten of twelve CLIs cannot remove them, so the page no longer promises redaction; the turn-end gap on Hermes and Goose is a scope decision, not a platform limit. The files reference now says when each path appears, because a fresh install creates seven entries and the rest are conditional. The agent-skills page documented three skills where four ship. "Guardrails" is now "policies" throughout, including the nav group and the transliterated forms. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…arding path The docs were technically accurate and hard to start from. Three things were wrong, and they compounded. The value proposition was inverted. The landing page opened "FailproofAI is the policy layer for that problem" and treated the cloud as a second half. The product is a loop — observe every run, score it, find the failures nobody wrote a rule for, prevent them, confirm the next audit is clean — and policy is the fix step inside it, not the pitch. introduction.mdx is now "What is FailproofAI" and says that; how-it-works and concepts follow the same order rather than the hook path's. There was no cloud onboarding at all. /cloud/getting-started redirected to an overview page, getting an API key was an off-site link on four pages, and "send your first trace" existed only inside a 447-line SDK reference whose install step is "talk to your contact". A new user could not self-serve. Five sequential pages now run from nothing to a deployed policy: create your workspace, send your first session, bring in the history you already have, find your first failures, prevent it happening again. They carry the minimum commands and link out for depth rather than restating the reference pages. And the sidebar described a product that no longer exists. cloud/overview enumerated an Observe/Analyze/Govern sidebar, four other pages named that grouping in prose, and four page slugs disagreed with what the dashboard calls them. The nav is now two tabs — the journey, and reference — and cloud/overview is one table mapping every real sidebar item to its page. Renames, with redirects in all 15 language trees: /cloud/incidents -> /cloud/issues /cloud/managed-policies -> /cloud/policy-editor /cloud/fleet -> /cloud/deploy-policies /cloud/access -> /cloud/admin 35 existing redirects pointed at two of those old destinations and were repointed; 461 in-body links across all 15 trees were rewritten, a class CI does not catch (mintlify validate resolves nav links only, validate:mdx resolves assets only). Two editorial rules now run through every page. There are exactly two ways to plug in — a supported harness via `failproofai config --connect`, or a custom agent via the Python SDK — and that fork is stated wherever a reader has to choose, instead of living in one buried note. And the product is FailproofAI Cloud in prose everywhere, with `agenteye` appearing only as something you type; both places a reader meets that name now say why. `--help` printed a third spelling, "FAILPROOF CLOUD". Also corrected against the shipped CLI: the cloud CLI's issue group is `issues`, not `incidents`, and the page claiming to list all 18 commands was missing `audits` and `usage`. Both admin surfaces those back — Usage and Settings — had no docs and now do. The local one-shot scan is "Local audit" against the fleet-wide "Audits", which retires the note in concepts.mdx that had been apologising for the collision. Locale page bodies are unchanged and stay in their current wording; the nightly translation job sees the path-keyed cache misses and rewrites them. No API spend here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B4teTZnSkV9KESL6rD74on
NiveditJain
force-pushed
the
luv-legion-687
branch
from
August 14, 2026 11:35
fd19987 to
d266d91
Compare
NiveditJain
changed the title
…nchor `### Policies run, in order` renders an anchor carrying the comma, while the link on cloud/policy-editor asks for `#policies-run-in-order` — so it landed at the top of the page instead of the section. Removing the punctuation makes the anchor unambiguous rather than betting on one slug rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B4teTZnSkV9KESL6rD74on
hermes-exosphere
requested changes
Aug 14, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found blocking issues that should be addressed.
1 advisory finding
- Low/High Localized key link has an invalid English fragment — All 14 non-English
start/sign-up.mdxpages link to/[locale]/cloud/admin#the-three-keys-most-teams-need(for example,docs/ar/start/sign-up.mdx:34). The Arabic destination heading is## المفاتيح الثلاثة التي تحتاجها معظم عمليات النشر(docs/ar/cloud/admin.mdx:8), so it has no English anchor. The fragment therefore lands readers at the page top rather than the machine-key section. (docs/ar/start/sign-up.mdx:34)
hermes-exosphere
previously approved these changes
Aug 14, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found no blocking issues in this revision.
3 advisory findings
- Medium/High Custom-agent onboarding never configures the uploader — The custom-agent branch presents the Python SDK itself as the connection method, then only calls
agenteye.configure()and emits events before stating that “The collector uploads those files” (docs/start/first-trace.mdx:53 and :107). It never installs/configures the collector or supplies the key from step 1. The collector credential/configuration is written byfailproofai config --connect(src/hooks/cloud-enrollment-cli.ts and src/hooks/collector-config.ts), whose header states it is the only writer of the collector credential. A custom-agent user following this new path will leave JSONL files on disk and never see the promised Sessions row. (docs/start/first-trace.mdx:53) - Medium/High History onboarding points to a dashboard location that no longer exists — The new onboarding tells readers that the fleet view is under “analyse → Policies” (docs/start/history.mdx:79). The new dashboard map instead places the fleet surface, Deploy Policies, under the
fixsection (docs/cloud/overview.mdx:44-50);analyse → Policiesonly leads to the policy editor. The same incorrect instruction is replicated in every localized history page. (docs/start/history.mdx:79) - Low/High README advertises 39 policies although the shipped catalog has 40 — This PR changes the README documentation table to say “All 39 policies” (README.md:193). The exported
BUILTIN_POLICIEScatalog contains 40 entries (src/hooks/builtin-policies.ts, beginning at line 2038), and the rewritten policy reference also states 40. (README.md:193)
hermes-exosphere
approved these changes
Aug 14, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found no blocking issues in this revision.
4 advisory findings
- Medium/High Localized sidebars omit the deployment task — The English Get started sequence includes
start/deploy-policy(docs/docs.json:47-53), but the representative Chinese sidebar still lists the old sign-up/history sequence and ends atstart/first-policy(docs/docs.json:175-180). The same missing deployment page occurs in all 14 non-English navigation trees, so localized readers cannot discover the final onboarding step. (docs/docs.json:175) - Medium/High Custom-agent onboarding never configures the uploader — The custom-agent path only calls
agenteye.configure()and then says that “The collector uploads those files” (docs/start/first-trace.mdx:73-107). It never installs/configuresfailproofaidor supplies itsevents:addcredential. The supported-harness branch does this withfailproofai config --connect; the collector configuration source identifies that connection flow as the writer of the ingest credential. A custom-agent user following this branch therefore only creates local JSONL files. (docs/start/first-trace.mdx:106) - Medium/High History onboarding points to a dashboard location that no longer exists — The history page directs readers to “analyse → Policies → the fleet view” (docs/start/history.mdx:79). The new dashboard map places Policy editor under analyse, while fleet assignments are under “fix → Deploy Policies”; there is no fleet view at the stated location. (
docs/start/history.mdx:79) - Low/High README advertises 39 policies although the shipped catalog has 40 — README.md:193 says “All 39 policies,” while
BUILTIN_POLICIEScontains 40 entries and the updated policy documentation consistently states 40. (README.md:193)
SiddarthAA
added a commit
that referenced
this pull request
Aug 18, 2026
One product owned three top-level dotfiles: ~/.fp (this CLI), ~/.failproofai (the Enforcement CLI) and ~/.agenteye (the SDK and collector spool). This collapses the first into the second. ~/.agenteye stays where it is — it is a wire contract the collector reads, not a preference, and renaming it from this side writes events into a directory nothing watches. Resolution is FP_HOME > $FAILPROOFAI_HOME/fpcli > ~/.failproofai/fpcli. FP_HOME is used as-is because that is what it meant before, so an existing export still addresses the same directory; FAILPROOFAI_HOME names the shared root, so the subdirectory is appended. The old file is neither migrated nor deleted. A session lives 24h and one `fp login` reissues it, which is cheaper than a credential-rewriting path that runs once per machine and is never exercised again — and deleting a file the user did not ask us to touch is the only irreversible act available here. `fp` names the stale file in its not-logged-in message so the sign-out does not read as a bug. ## Registered in a layout this repo already governs ~/.failproofai is not a free directory. src/hooks/fp-home.ts declares its shape, crates/failproofaid/src/paths.rs mirrors it for the daemon, and resetHome walks it with rmSync(recursive). So the path is declared there and classified `user-typed` in HOME_CLASSES, which is what actually keeps it: resettablePaths() is a filter over that table and a migration drops only `derived` and `refetchable`. Verified by running the real resetHome(3,4) against a home holding the file — it removed two derived paths and left the credential intact. LAYOUT_VERSION is deliberately NOT bumped. Preservation comes from the classification, not the version, and that file's own rule is that the version moves when a path moves. Nothing moved; a bump would mark every existing home stale and run a reset on machines with nothing to migrate. Not added to paths.rs, following auditSessionFile: the daemon has no reason to open a human credential, and mirroring a path only Python writes would give paths.rs a row nothing there reads. fpcliDir is registered as deliberately unclassified (COVERED_BY_PARENT), because the credential is the thing to classify and a cache may sit beside it later. ## Four bugs the shared directory created, none of which existed in ~/.fp Writing next to another product's secrets is a different problem from writing into a directory we owned outright. Each of these destroys or hangs on a neighbour, and none of them is visible from either side afterwards. 1. A SYMLINK at cli-auth.json wrote through to its target. O_TRUNC follows links, so a link pointing at ../credentials.json made `fp login` truncate the Enforcement CLI's token and write the session over it. Now refused by name — and the link is left in place, because a person put it there. 2. A HARD LINK did the same and O_NOFOLLOW says nothing about it: it is not a link, it is a second name for one inode. Answered structurally by writing a temp file and renaming it into position, which swaps the directory entry and leaves the other name on the old inode. 3. A FIFO in the config position HUNG the CLI. open() on a FIFO blocks until a reader appears, so `fp login` waited with no output — a mutation run without the rename sat there ten minutes before being killed. The rename never opens the FIFO at all. 4. fpcli/ inherited the umask (0775 under a common 0002). The file was always 0600 so nothing was readable, but a group-writable directory lets anyone in the group replace it, which is a session swap. Created 0700 now. The shared parent is left to the umask when we create it and never re-permissioned when it exists — hardening what we own, not what we do not. The rename also makes the write atomic: a reader never sees a half-written credential, and racing processes end with one whole session. mkstemp rather than a pid-derived temp name, because two THREADS share a pid and collided under O_EXCL — caught by the concurrency test, not by review. ## Tests 46 new, covering the resolution order and empty/relative/trailing-slash/unicode env shapes; a populated home surviving intact; every hostile filesystem shape (home or config as a regular file, a directory, a FIFO, a symlink, a hard link, a broken symlink, read-only, untraversable); permissions created and preserved; temp-file cleanup on the failure path; 8-thread and 4-process concurrency; and that the legacy file is read by nothing and deleted by nothing. Each guard was mutation-tested rather than assumed: reverting the wipe protection, the legacy fallback, the precedence order, O_NOFOLLOW, the 0700, and the atomic write each fails exactly the tests that claim to cover it. Also removes test_v1_routing.py's third leg, which read the AgentEye server's router out of a checkout that is never present in CI. It skipped in every run, and a skip renders green — so the only automated check that CLI paths match real server routes was reporting success while verifying nothing. Removed rather than left switched off; the module docstring records that the coupling is now unguarded and surfaces as a 404 at runtime. 773 pass in fp-cli, 53 in the TS layout suite, 11 in Rust including every_mirrored_path_agrees_with_fp_home_ts. Verified live against a running deployment: reads and writes against the new path, the seeded enforcement-home artifacts byte-identical afterwards, 0600 on the file, 0700 on fpcli/, and no temp files left behind. Docs: fp-cli README, its agent skill, and the CHANGELOG. The enterprise CLI doc in FailproofAI/agenteye needs the same edit and is not in this repo. The public docs are untouched on purpose — docs/agenteye/cli.mdx is still entirely pre-rename (it teaches `agenteye login` and AGENTEYE_HOME), and that rewrite is tracked separately, blocked on #687. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
NiveditJain
added a commit
that referenced
this pull request
Aug 24, 2026
…`failproofai-sdk` (#702) * feat(fp-cli): open-source the Cloud CLI as `fp-cli`, command `fp` Moves the observability CLI out of the private AgentEye monorepo and into this repo, renamed end to end. It was PyPI `agenteye` / command `agenteye` / package `agenteye_cli`; it is now PyPI `fp-cli` / command `fp` / package `fp_cli`. The distribution and the command differ on purpose: `fp` was already taken on PyPI. This is also distinct from the `failproofai` CLI this repo already builds from bin/ + src/ — that one enforces inside the agent loop, this one reads back what the loop did. This is a HARD CUT, matching the precedent set when the collector binary was renamed: no `agenteye` alias, no retired env-var fallback, and no migration of the old config file. Scripts calling `agenteye ...` break on upgrade and users run `fp login` once. - env vars the retired namespace -> FP_* (FP_TOKEN, FP_API_KEY, FP_ORG, FP_DASHBOARD_URL, FP_JSON, FP_INSECURE, FP_HOME, FP_ANALYTICS_DISABLED, FP_CLI_DEV) - config ~/.agenteye/cli.json -> ~/.fp/cli.json (still mode 0600) - telemetry PostHog `product` tag agenteye -> fp-cli. Telemetry has been disabled since well before the rename, so nothing was flowing across the boundary and the series split costs nothing. Deliberately NOT renamed — these are a cross-component contract with the dashboard and the Rust server, neither of which is changing: - the X-AgentEye-Org and X-AgentEye-Client request headers - the ae_session cookie - the SDK/collector home dir, which still belongs to the Python SDK and the collector for their event spool Repo plumbing, all of it new — this is the first Python in the repo: - a matrixed `fp-cli` job in ci.yml (3.10 and 3.13) that tests, builds, and smoke-tests the console script from a clean install of the built wheel - publish-fp-cli.yml, a manual PyPI publish over Trusted Publishing. The trusted publisher must be configured on PyPI before the first release; the workflow header documents exactly what to enter. - a uv dependabot ecosystem, fp-cli/uv.lock in the osv-scanner gate, Python artefacts in .gitignore, and the directory registered in CONTRIBUTING.md and CLAUDE.md Also fixes four things found while verifying, three of them pre-existing: - the wheel now ships a py.typed marker it had been claiming via the `Typing :: Typed` classifier without providing - README documented `fp incidents`, renamed to `issues` long ago, and claimed the dashboard URL was required with no default (there is one). Both were about to become a public PyPI landing page. - tests/conftest.py's env clear-list omitted the insecure-TLS variable, so a developer with it exported ran the whole suite with TLS verification off - tests/test_v1_routing.py anchored the monorepo on any AGENTS.md; this repo has one at its root, so it would have resolved to a root with no server/ under it and failed for the wrong reason. It now anchors on the router file itself and skips cleanly when the monorepo is absent. New guards, because each of these could previously rot silently: - test_help_table_coverage.py — `fp help` renders a HAND-MAINTAINED table, so a registered command missing from it is invisible in help forever. Nothing checked this before. - test_readme_matches_reality.py — pins the README's commands, install instructions, default URL, exit codes and env vars to the code. - a tripwire on the click-compat package scan, which walks a path literal and would pass vacuously if that literal ever stopped resolving. 720 tests pass. Verified beyond the suite, which is entirely respx-faked: the built wheel installs into a clean venv, `fp` resolves, and against a real local HTTP server it sends X-AgentEye-Org, the ae_session cookie and x-request-id unchanged, writes only ~/.fp, leaves the old home dir untouched, returns exit codes 0/2/3/4 with the documented --json envelope, honours FP_*, ignores the retired variables, and prints the retired name nowhere. * docs(changelog): record the fp-cli open-sourcing (#702) * feat(fp-cli): re-establish the agent-skill mirror, now as skills/fp-cli The CLI's agent skill was mirrored to FailproofAI/skills as skills/agenteye-cli/ by sync-skill.yml in the private agenteye repo. That workflow is deleted along with the CLI, which would leave the published skill orphaned — still installable, still teaching the retired `agenteye` command, and synced by nothing. sync-fp-cli-skill.yml replaces it here: fp-cli/skill/ -> skills/fp-cli/, same force-push-one-branch, reuse-one-PR shape as the two surviving mirrors in the agenteye repo. Two things it needs from an admin, both documented in the workflow header: - an Actions secret SKILLS_SYNC_PAT on THIS repo. The agenteye repo has one of the same name; secrets do not cross repos, so this needs its own. - deleting the orphaned skills/agenteye-cli/ folder on FailproofAI/skills. Also fixes the skill's own invoke-resolution step 2, which told an agent to look for a `cli/` directory holding the fp_cli package. That directory is `fp-cli/` here, so the dev-build path would never have resolved. * fix(fp-cli): remove a customer identifier, restore publish authorization Findings from an adversarial review panel. Two are consequences of moving code out of a private repo that nobody would notice from the diff alone. A real customer's tenant slug and company name were in the tree — 20 occurrences across fp_cli/output.py and four test files, carried over verbatim from the private monorepo where naming a live tenant in a fixture was harmless. One of them is a source comment that ships inside the wheel. The name appears nowhere else in this repo, so publishing would have been its first public disclosure. Replaced with globex/Globex Corp, matching the acme/example.com vocabulary the rest of the suite already uses, and pinned by tests/test_no_customer_identifiers.py so it cannot return: it scans the package, the tests, the README, the CHANGELOG and the skill for a deny-list of real organisation names and for customer deployment hostnames. publish-fp-cli.yml had no branch check and no actor allowlist. The workflow it replaces (release-cli.yml, in the private repo) carried both, and they were lost in a change described as a like-for-like move. Authentication here is OIDC Trusted Publishing, so there is no token to withhold — repo write access IS publish access, and workflow_dispatch targets an arbitrary ref. One click on an unreviewed branch would have shipped it to public PyPI as an official release, and PyPI versions cannot be reused. Both guards restored. The publish path also now runs the same clean-install smoke test CI does, rather than only inspecting the zip. Also: - `uv sync` is now `uv sync --locked` in both workflows. uv.lock silently re-resolved eight dependencies during the move — certifi (which decides which CAs the CLI trusts against a self-hosted deployment) and posthog among them — inside a commit described as a move. Without --locked the committed lock is decorative, which also makes the osv-scanner gate over it dishonest. - README documented `fp audits update`; the verb is `edit`. The line was new in this migration, so it was a fresh false claim on the PyPI landing page. test_readme_matches_reality now checks one level deeper into each group's registered subcommands, which is why the group-level check missed it. - the Documentation URL pointed at a docs path that does not exist yet — that docs tree lands in a separate PR. Repointed at the page that exists today. - sync-fp-cli-skill.yml told an admin to delete skills/agenteye-cli/. The live public docs still hand that skill out by name, so deleting it first turns a documented install command into a not-found error. The instruction now spells out the required order. 724 tests pass. Every new guard was negative-controlled — deliberately violated to confirm it fails, rather than assumed to work because it is green. * fix(fp-cli): stop a stdin read draining a query, and a tripwire naming its own customer Six findings from the review bots on #702. `fp query update --sql @-` saved an empty query. `@-` is stdin, which drains on the first read, and the command read it twice — once to work out which fields changed, once to build the request body. Change detection compared the real text while the save wrote "", at exit 0 behind a green card. Read once into a local. `fp issues resolve` and `fp issues comment-delete` printed only the human stderr line when a prompt was declined. Both docstrings promise `{"cancelled": true}` under --json and the other ten write commands emit it, so a script reading stdout got an empty document at exit 0. test_no_customer_identifiers.py spelled out the real tenant slug it exists to keep out of a public wheel — in a public repo, in a file that ships in the sdist — and excluded itself from its own scan, so nothing reported it. The customer entries are SHA-256 digests now, matched over token substrings so both the slug and the longer company name built from it still trip, and a failure names the file, the line and the class of identifier, never the identifier. A planted invented name proves the matcher still matches, since an off-by-one in the substring window would otherwise turn the whole opaque deny-list into an assertion that passes by matching nothing. Our own org names stay in the clear: they are in LICENSE, SECURITY.md and package.json already, and a contributor who trips over one needs to see which it was. publish-fp-cli.yml asked for `id-token: write` and nothing else. Naming any scope sets every unnamed one to `none` rather than leaving it at the default, so checkout got a token that cannot read this repository — with a comment two lines up asserting the opposite. It also binds to a `pypi-fp-cli` environment now: every other guard there (the actor allowlist, the `main` check) lives on the ref being dispatched, so a writer could delete them on a branch and click Run, and OIDC mints a publishing token for whatever the workflow then asks for. The environment's branch rule lives in repo settings and its name in PyPI's publisher config — neither reachable from a branch, and deleting the `environment:` line fails the upload on a claim mismatch. Documented as required setup, because GitHub creates a missing environment implicitly and WITHOUT protection rules. sync-fp-cli-skill.yml wrote its PAT into $WORKDIR/.git/config via the clone URL — a token with Contents write and Pull requests write on FailproofAI/skills, left in a workspace where the next step runs validate-skills.py, fetched from that same repo. Clone and push now authenticate through `git -c http.extraheader` (before the subcommand, so it is not persisted into the new repo's config), from `env:` rather than interpolated into the script body. __tests__/ci/fp-cli-workflows.test.ts pins all four workflow invariants: the two that look redundant — `contents: read`, and the environment name matching the header a maintainer reads it off — are the two a cleanup would delete. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL4iJTndrDrgtjsRavqs6f * chore: re-trigger CI after GitHub 429s in Set up job Both failing jobs died before running a step: codeload.github.com answered 429 to the runner's download of oven-sh/setup-bun (rust-quality) and google/osv-scanner-action (OSV-Scanner), through all three of the runner's own retries. Every job that got past setup passed, including both fp-cli matrix legs, the three test configs, build, test-e2e, docs and quality. Empty on purpose: nothing in 29d04e89 is implicated, and `gh run rerun` is blocked by this repo's own hook policy, so a new head SHA is the only way to ask for the two jobs again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL4iJTndrDrgtjsRavqs6f * chore: re-trigger CI again — GitHub Actions still in a partial outage The previous trigger got 9 of 10 jobs green; `build` lost its oven-sh/setup-bun download to a 429/503 in Set up job, before running a step. GitHub has been in a partial system outage since 13:40 UTC (Actions major outage, ~50% failure rate on repository and archive content downloads), so the failing job rotates between runs. Every job has now passed on this exact tree — build and 8 others on 29d04e89, rust-quality and 8 others on 7900b015, Supply Chain on both — and `bun run build` was verified locally besides. Empty on purpose: `gh run rerun` is blocked by this repo's own hook policy, so a new head SHA is the only way to ask for the remaining job. Stacked rather than amended because the previous placeholder is already pushed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL4iJTndrDrgtjsRavqs6f * chore: re-trigger CI (attempt 4) — one job per run still losing setup-bun to a 429 GitHub has been in a partial system outage since 13:40 UTC (Actions major outage, ~50% failure rate on repository and archive content downloads). Its shape here is consistent: all ten CI jobs fetch the same oven-sh/setup-bun archive at once, exactly one loses it to three 429s in Set up job, and which one rotates — rust-quality, then build, then quality. So each run is ~9/10, and a fully green run is a coin flip rather than a dead end. Every job has passed on this exact tree: quality/build/rust-quality each green in at least one of the three runs, everything else green in all of them, Supply Chain green on the current SHA. `bun run build` verified locally too. Empty on purpose: nothing in 29d04e89 is implicated, and `gh run rerun` — which would re-run the single failed job with no download stampede — is blocked by this repo's own hook policy, so a new head SHA is the only lever available. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL4iJTndrDrgtjsRavqs6f * feat(sdk): open-source the telemetry SDK as `failproofai-sdk` in sdk/python The other end of the pipe from fp-cli. The agent calls this to record what it did; the CLI reads that back. Moved out of the private AgentEye monorepo, where it was `python-sdk/`, distribution `agenteye`, licensed Proprietary and shipped as a private GitHub Release asset. It is now MIT + Commons Clause on public PyPI, matching fp-cli. `sdk/` is a directory rather than a flat `failproofai-sdk/` because more languages go beside `python/`, not inside it. ## The rename stops at the import name, deliberately The Python import name and the PyPI distribution name are the ONLY things that changed. `~/.agenteye/`, `AGENTEYE_HOME`, `AGENTEYE_ENVIRONMENT`, `AGENTEYE_SPOOL_TO_FAILPROOFAI`, the `.tmp`->`.jsonl` publish, every event type and every payload key are a contract with two separately-released daemons — `failproofaid` here and the older `agenteye-collector` in the private repo. Renaming any of them from the SDK's side writes events into a directory nothing watches, with no error on either side: batches pile up on disk, and an unread spool looks exactly like an idle one. This is the same call #702 made for `X-AgentEye-Org` and the `ae_session` cookie. `test_server_contract.py` freezes the literals so a later rename sweep cannot take them. ## Two real bugs found while writing the tests Batch files were named from a millisecond timestamp alone, so two batches written inside one millisecond got the same filename and the second `os.replace` silently destroyed the first — no exception, no log, no trace the events existed. It fired three ways: the atexit flush racing the flush thread (exactly when a run's last events are written), `flush_now()` from two threads, and across processes, since nothing in the name identified the writer and several agents sharing one spool root is the ordinary deployment. The stem now carries the pid and a per-process counter, which is what `fpai-collect`'s own batches already do; both daemons only ever required the `.jsonl` suffix. The cross-component spool test gated every assertion on a source path from the private agenteye repo, so all four skipped in every CI run — including three that assert nothing but this SDK's own resolution rule and need no other checkout at all. It now reads `crates/fpai-collect/src/config.rs` and `src/hooks/fp-home.ts` from THIS repo and never skips; the daemon that reads the spool finally lives next to the SDK that writes it. `FAILPROOFAI_SDK_REQUIRE_CONTRACT=1` in CI turns a moved file into a failure rather than a skip, because a guard that can degrade to a skip is not a guard. ## Tests 188 pass, up from 80. The new suites exist because every failure they catch is silent — the SDK returns None from a background thread and the caller moved on long ago: - `test_wire_format.py` freezes the serialized bytes of all 15 event types, including key ORDER, since `dedup.rs` hashes the canonical payload and a cosmetic reorder stops retried batches collapsing into silent duplicates. - `test_server_contract.py` pins the keys ingest promotes to indexed columns. `ps()` cannot tell a missing key from a wrong-typed one — both store NULL at 200 OK — so it checks types too. - `test_durability.py` covers 16-thread emission, concurrent flushes, fork, every exit path including the `os._exit` loss window (documented, not pretended away), ENOSPC/EACCES retry, and a reader that must never see a torn batch. - `test_zero_dependencies.py` makes the stdlib-only promise enforceable: the source is parsed for non-stdlib imports (including inside functions, which is where `_environment` really imports `os`), the manifest for a `dependencies` key, and CI installs the built wheel with `--no-deps`. - `test_no_customer_identifiers.py` is fp-cli's tripwire, ported. It caught a private-release URL in the README and the skill on its first run. Two suites can reach an AgentEye checkout via `FP_AGENTEYE_ROOT` to verify against the real `ingest.rs` and the older collector; both are opt-in and both pass today. ## Registration CI job matrixed across all five Python versions `requires-python` advertises — wider than fp-cli's two, because a package with no dependencies has no third-party floor quietly constraining which interpreters it is really tested on. Trusted-Publishing PyPI workflow, skill mirror, `uv` dependabot ecosystem, osv-scanner lockfile, and `__tests__/ci/failproofai-sdk-workflows.test.ts` guarding all of it — including that the two skill syncs share no force-pushed branch, which would silently overwrite each other's open PR. Needs out-of-band setup before the first publish: the PyPI pending publisher, the `pypi-failproofai-sdk` environment (GitHub creates a missing one WITHOUT protection rules), the `skill-sync-failproofai-sdk` label, and this repo's own `SKILLS_SYNC_PAT`. Each is documented in the workflow that needs it. The docs keep pointing at `skills/agenteye-python-sdk` until the first mirror PR lands on FailproofAI/skills — repointing them first would turn a documented install command into a not-found error, the same ordering fp-cli used. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M3e8zNeqL33PXoucfJcQ9C * fix(sdk): make the zero-dependency check work on Python 3.10 `tests/test_zero_dependencies.py` imported `tomllib` unconditionally, and that is stdlib only from 3.11. `pyproject.toml` advertises `requires-python = ">=3.10"`, so the suite failed to collect on the oldest interpreter we claim to support — caught by the matrix leg added in the same PR, which is what it is for. fp-cli tests two versions and would not have seen this. Fixed by importing `tomli` as a fallback rather than skipping the module. These are the manifest assertions that make "zero dependencies" enforceable rather than aspirational, and a check that quietly stops running on 3.10 is checked where it matters least — the 3.10 user is exactly the one with the most fragile environment. `tomli` is a TEST dependency. `[project.dependencies]` is still empty, which is the thing actually promised, and CI still installs the built wheel with `--no-deps` to prove it against the artifact. The dev-extra assertion had to loosen to allow it, so it is now an explicit allowlist carrying the reason for each entry rather than "everything must start with pytest". That is the stronger form anyway: the failure it prevents is a convenience library drifting in, and a name with no stated reason is the shape that happens in. Verified locally on all five matrix versions: 194 passed on each. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M3e8zNeqL33PXoucfJcQ9C * test(sdk): stop the configure() thread-safety test depending on execution order `test_configure_is_safe_to_call_from_several_threads` asserts an EXACT event count on the process-wide writer singleton, and did not drain it first. Nothing pollutes it today — the only other test that touches the singleton flushes — so this is not a live failure. It is one test away from being one, and the way it would present is an exact-count assertion failing in a test about thread safety, which sends you looking at the locking rather than at the fixture. Drains to a throwaway directory first. Verified the file passes alone, in the suite, and immediately after `test_sdk.py` (the order that would surface it). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M3e8zNeqL33PXoucfJcQ9C * fix(sdk): address four CodeRabbit findings — all four were real Two behaviour bugs, one weak regression test, one stale docstring. ## Tool and hook pairings shared one keyspace `_pending` namespaced its human and pause pairings (`human:`, `pause:`) but not its tool and hook ones, which keyed on the bare `tool_call_id` and `hook_id`. Those collide routinely — both are frequently the harness's own step id — and when they did, `hook_completed` consumed the `tool_use` timestamp and reported the interval between two unrelated events, while the real `tool_result` that followed got no duration at all. Two plausible numbers, no error, nothing downstream able to tell. I had pinned this with a test rather than fixing it, reasoning that changing recorded durations deserved its own change. That was wrong: nobody depends on a fabricated duration, and the non-uniformity WAS the bug — the fix makes the module consistent with the two pairings that were already namespaced. The keys never leave the process, so no wire format changes. ## An unusable flush_interval killed the writer thread `time.sleep()` runs before the flush loop's `try`, deliberately: wrapping it would turn a failing flush into a full-speed retry loop instead of a next-cycle one. The cost was that -1, NaN and inf all raised out of the thread and killed it — and a dead writer thread is this class's worst state, because `submit()` keeps accepting events, the queue keeps growing, nothing is written, and the caller learns none of it until the process exits and takes everything with it. Zero did not raise but busy-looped, pinning a core. Rejected at the boundary now, in `EventWriter.__init__`, `set_flush_interval()` and `configure()`. `configure()` validates FIRST, before `set_base_dir` — a rejected call must not leave a new base_dir applied with the old interval. ## The collision regression tests only usually reproduced the bug `test_two_batches_in_the_same_millisecond...` and its cross-process sibling did not force the two writes into one millisecond. They do on a fast machine, which is exactly the problem: on a loaded runner the clock advances between them, a timestamp-only stem produces two different names, and the test passes against the implementation it exists to reject. A frozen clock fixture makes it deterministic. Verified by reverting the stem — both now fail (4 processes collapse to 1 file); same check run for the correlation fix. The fixture has to reach the module through `sys.modules`, because `failproofai_sdk._writer` is the EventWriter singleton and shadows the submodule of the same name. ## Stale docstring `test_resolver_umbrella.py` still called `collector/src/config.rs` "the collector this repository ships". I corrected that same claim in `_resolver.py` and missed its copy here. 211 tests pass on all five matrix versions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M3e8zNeqL33PXoucfJcQ9C * fix(fp-cli): two docs claims that contradict the shipped code Both from the Hermes review, both verified against the source, both user-facing — the README ships in the wheel to PyPI and the skill is executed by coding agents. **The README said telemetry is on by default. It is off.** `analytics_config.py` sets `TELEMETRY_DISABLED = True`, and has since before the rename, because the send path stalls every command ~5s when the analytics host is unreachable: the shutdown flush is bounded, the client build and first connect attempt are not. The README also claimed the opposite of that specifically — "sending is time-bounded, so it never delays a command". Now says it is disabled, why, and keeps the collection description as a review-it-in-advance section rather than deleting it, since re-enabling is one constant. **The skill stated half the credential ladder.** It said `FP_API_KEY` takes precedence over `FP_TOKEN`, which is true only between the two environment variables. `resolve_auth` checks the explicit `--token` flag *before* the ambient API key (`_context.py:93-103`), so exporting `FP_API_KEY` in CI and also passing `--token` runs as that user's saved session, with their org memberships, instead of under the scoped key you meant to audit. The skill is instructions an agent executes, so a half-stated rule is one an agent acts on. Now spells out all six rungs, and names the one that catches people. 728 fp-cli tests pass; MDX validates. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M3e8zNeqL33PXoucfJcQ9C * fix(fp-cli): two --help examples that fail when you run them Both were found by running the CLI's own documented examples against a live deployment. Neither is caught by anything: help text is a docstring, so a wrong example compiles, ships, and passes the suite. `query create` pointed at a table that does not exist. The example read `FROM fp.events`; the queryable schema is `analytics`, so running it verbatim returns `relation "fp.events" does not exist` and exit 1. This one came from the migration: pre-move the line read `FROM agenteye.events`, and 3566bce5 rewrote the command name (right) and the ClickHouse database name (wrong) in the same sweep. `agenteye.events` is a protected form — the database is not renaming, and the collector still reads it. Exactly one occurrence; verified none remain in fp-cli/, sdk/, docs/ or skills/. The globals epilogue advertised a `-p` flag that no longer exists. `-p` was real as of 0.1.7 (see CHANGELOG) as the permission input for `keys create`, then was replaced by --permission-set / --add / --remove without the epilogue following. It renders at the bottom of every leaf command's --help, so it is the most-read wrong example in the CLI. The replacement is a form actually exercised against a live server, not a guess. The 0.1.7 CHANGELOG entry keeps saying `-p`: it is a historical record of a release where the flag did exist. Verified: both corrected examples run clean against a live deployment; suite is 728 passed / 0 skipped, including test_v1_routing.py's cross-repo route check run with FP_AGENTEYE_ROOT set (32 passed) — it skips silently by default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(sdk): seven ways the SDK lost events without saying so All seven destroy or corrupt telemetry at runtime with no error reaching the caller — event.*() returned None long ago and the application moved on. Each fix has a regression test that was negative-controlled: the pre-fix behaviour was restored and the guard watched to fail. 1. ONE UNSERIALIZABLE PAYLOAD WEDGED THE SPOOL PERMANENTLY. Encoding was a single json.dumps over the whole drained batch, so one bad event took every event beside it down: _flush re-queued the batch and re-raised, _flush_loop retried the identical batch next interval, forever. Nothing emitted afterwards ever reached disk. default=str never helped — it is consulted for values, not keys, so a tuple-keyed cache or an object holding a back-reference both raise. Encoding is per-entry now: strict first (byte-identical for ordinary events), then a sanitised copy, then drop that one event. Encoding failures drop, IO failures still retry. 2. The queue was unbounded, so anything that stopped the spool draining turned a telemetry outage into an OOM kill of the host agent. Capped at 10_000, oldest-first, with a throttled warning. 3. The flush thread did not survive fork(). A prefork worker (gunicorn, celery, multiprocessing on Linux) published nothing at all. An os.register_at_fork handler restarts it and rebuilds the Event and lock, either of which can be inherited held by a thread that no longer exists. The inherited queue is discarded — those events belong to the parent, and publishing from both duplicated every buffered event. 4. Tool and hook durations correlated ACROSS sessions and agents. _pending is process-wide and human/pause pairs were already scoped; these two were not. Two sessions sharing a step id meant one reported the other's interval and the other reported none. 5. duration_ms/input_tokens/output_tokens were accepted at any type. The server reads them with pu32(), which stores NULL on a mismatch at 200 OK. Now refused at the boundary, where the caller still has a stack trace. 6. A new flush_interval did not apply to the cycle already waiting, so configure() was ignored for one full cycle of the old interval. 7. A flush racing interpreter shutdown lost the batch. Entries are drained before they are written, and _flush's emptiness check sat outside the lock — so the atexit flush saw an empty queue, returned, and the dying thread took the events, leaving at most a stray .tmp. The check moved inside _flush_lock. The atexit hook is also registered once at module scope over weak references (atexit.register(self._flush) made every writer immortal) and now logs its own exceptions instead of printing a traceback into the host agent's stderr during shutdown. Tests: 211 -> 266. Verified on the built wheel in clean containers on Python 3.10/3.11/3.12/3.13/3.14, installed with --no-deps. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(fp-cli): move the session into ~/.failproofai/fpcli/cli-auth.json One product owned three top-level dotfiles: ~/.fp (this CLI), ~/.failproofai (the Enforcement CLI) and ~/.agenteye (the SDK and collector spool). This collapses the first into the second. ~/.agenteye stays where it is — it is a wire contract the collector reads, not a preference, and renaming it from this side writes events into a directory nothing watches. Resolution is FP_HOME > $FAILPROOFAI_HOME/fpcli > ~/.failproofai/fpcli. FP_HOME is used as-is because that is what it meant before, so an existing export still addresses the same directory; FAILPROOFAI_HOME names the shared root, so the subdirectory is appended. The old file is neither migrated nor deleted. A session lives 24h and one `fp login` reissues it, which is cheaper than a credential-rewriting path that runs once per machine and is never exercised again — and deleting a file the user did not ask us to touch is the only irreversible act available here. `fp` names the stale file in its not-logged-in message so the sign-out does not read as a bug. ## Registered in a layout this repo already governs ~/.failproofai is not a free directory. src/hooks/fp-home.ts declares its shape, crates/failproofaid/src/paths.rs mirrors it for the daemon, and resetHome walks it with rmSync(recursive). So the path is declared there and classified `user-typed` in HOME_CLASSES, which is what actually keeps it: resettablePaths() is a filter over that table and a migration drops only `derived` and `refetchable`. Verified by running the real resetHome(3,4) against a home holding the file — it removed two derived paths and left the credential intact. LAYOUT_VERSION is deliberately NOT bumped. Preservation comes from the classification, not the version, and that file's own rule is that the version moves when a path moves. Nothing moved; a bump would mark every existing home stale and run a reset on machines with nothing to migrate. Not added to paths.rs, following auditSessionFile: the daemon has no reason to open a human credential, and mirroring a path only Python writes would give paths.rs a row nothing there reads. fpcliDir is registered as deliberately unclassified (COVERED_BY_PARENT), because the credential is the thing to classify and a cache may sit beside it later. ## Four bugs the shared directory created, none of which existed in ~/.fp Writing next to another product's secrets is a different problem from writing into a directory we owned outright. Each of these destroys or hangs on a neighbour, and none of them is visible from either side afterwards. 1. A SYMLINK at cli-auth.json wrote through to its target. O_TRUNC follows links, so a link pointing at ../credentials.json made `fp login` truncate the Enforcement CLI's token and write the session over it. Now refused by name — and the link is left in place, because a person put it there. 2. A HARD LINK did the same and O_NOFOLLOW says nothing about it: it is not a link, it is a second name for one inode. Answered structurally by writing a temp file and renaming it into position, which swaps the directory entry and leaves the other name on the old inode. 3. A FIFO in the config position HUNG the CLI. open() on a FIFO blocks until a reader appears, so `fp login` waited with no output — a mutation run without the rename sat there ten minutes before being killed. The rename never opens the FIFO at all. 4. fpcli/ inherited the umask (0775 under a common 0002). The file was always 0600 so nothing was readable, but a group-writable directory lets anyone in the group replace it, which is a session swap. Created 0700 now. The shared parent is left to the umask when we create it and never re-permissioned when it exists — hardening what we own, not what we do not. The rename also makes the write atomic: a reader never sees a half-written credential, and racing processes end with one whole session. mkstemp rather than a pid-derived temp name, because two THREADS share a pid and collided under O_EXCL — caught by the concurrency test, not by review. ## Tests 46 new, covering the resolution order and empty/relative/trailing-slash/unicode env shapes; a populated home surviving intact; every hostile filesystem shape (home or config as a regular file, a directory, a FIFO, a symlink, a hard link, a broken symlink, read-only, untraversable); permissions created and preserved; temp-file cleanup on the failure path; 8-thread and 4-process concurrency; and that the legacy file is read by nothing and deleted by nothing. Each guard was mutation-tested rather than assumed: reverting the wipe protection, the legacy fallback, the precedence order, O_NOFOLLOW, the 0700, and the atomic write each fails exactly the tests that claim to cover it. Also removes test_v1_routing.py's third leg, which read the AgentEye server's router out of a checkout that is never present in CI. It skipped in every run, and a skip renders green — so the only automated check that CLI paths match real server routes was reporting success while verifying nothing. Removed rather than left switched off; the module docstring records that the coupling is now unguarded and surfaces as a 404 at runtime. 773 pass in fp-cli, 53 in the TS layout suite, 11 in Rust including every_mirrored_path_agrees_with_fp_home_ts. Verified live against a running deployment: reads and writes against the new path, the seeded enforcement-home artifacts byte-identical afterwards, 0600 on the file, 0700 on fpcli/, and no temp files left behind. Docs: fp-cli README, its agent skill, and the CHANGELOG. The enterprise CLI doc in FailproofAI/agenteye needs the same edit and is not in this repo. The public docs are untouched on purpose — docs/agenteye/cli.mdx is still entirely pre-rename (it teaches `agenteye login` and AGENTEYE_HOME), and that rewrite is tracked separately, blocked on #687. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fp-cli): name the right stale session file for FP_HOME users The move changed the FILENAME as well as the directory — `cli.json` became `cli-auth.json` — so somebody who exported `FP_HOME` is logged out exactly like everyone else, with their old session sitting at `$FP_HOME/cli.json`. The stale-file notice only looked at `~/.fp/cli.json`, which those users may not have at all. Two ways that went wrong, both found by running it rather than reading it: * on a machine with no `~/.fp`, an FP_HOME user got a bare "Not logged in" with nothing connecting it to the upgrade — and FP_HOME is the documented way to relocate this config, so the group least able to shrug at an unexplained logout is the group that got no explanation; * on a machine that had both, the notice named `~/.fp/cli.json` — a file unrelated to how that invocation resolved — and told them to delete it. `legacy_config_paths()` now returns both candidates and checks the relocated one FIRST, so the file named is always the one this invocation would have read. Nothing that authenticates without the config file is touched: `--token` / `FP_TOKEN` and `--api-key` / `FP_API_KEY` never opened a file and still do not, verified against a live deployment with no config present anywhere on the machine. Read-only commands still write nothing. The blast radius of the whole move is exactly one thing — a machine whose session came from the config file needs one `fp login`. Also confirmed, since this is the last thing standing between the change and production: creating `~/.failproofai` cannot fool the Enforcement CLI's setup. `isConfigured()` reads `policies-config.json`, a specific file, not the directory's existence — so a machine where `fp login` ran first still reports unconfigured and still gets its wizard. 776 pass. The two new cases are mutation-tested: collapsing the candidate list back to the default alone fails both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fp-cli): three help strings still printed the pre-move config path `login`, `logout` and `orgs switch` all told the user their session lives at `~/.fp/cli.json`. It does not, and all three print it — this is output, not a comment. `orgs_cmds`'s module docstring said the same. The same failure mode as the `fp.events` example fixed earlier on this branch: help text is a docstring, so a wrong path compiles, ships, and passes a green suite. Nothing in the move could have caught these, because nothing reads them. So this adds the check that would have: a test walking every shipped module for the old path. `config.py` is exempt — that is where the legacy location is deliberately named, to recognise a pre-move install and say so. Verified by planting a stale path back into `orgs_cmds` and watching it fail. 777 pass. `fp login --help` now prints the real location. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(fp-cli): adopt a pre-move session instead of signing the machine out Reverses the forced re-login this branch shipped two commits ago. That was a defensible call for a young CLI and the wrong one for a release going to production: `fp login` needs an emailed code, so it cannot be scripted, and the upgrade would have interrupted every human on every machine to buy nothing but a simpler code path here. A session at the old location is now read on the next command, written to the new one, and returned. Nobody is signed out and no command changes behaviour. Three properties, each chosen against a specific way this goes wrong: * It COPIES. The old file stays exactly where it is, so an older `fp` still finds its session and a half-rolled-out fleet is not a one-way door. Moving it would make the upgrade irreversible on the machine, which is not a property to hand a release that is still `Unreleased`. * It is BEST-EFFORT. A read-only home, a full disk or a symlink we refuse leaves the session that was found still returned to the caller. Our own housekeeping must never be the reason someone is logged out. * It does NOT reach past `FP_HOME`. Somebody who exported it said where their config lives; looking in `~/.fp` anyway would adopt a session from a context they deliberately moved away from — a different tenant, or another user's leftovers on a shared box. That was also how the fallback quietly picked up the developer's own login and turned ten unrelated tests red. With this the change is genuinely non-breaking. Verified end to end against a running deployment: a home holding only a pre-move `~/.fp/cli.json` runs `fp whoami` with no login, comes back authenticated, lands the session at the new path with mode 0600, and leaves the old file intact. The credential-file paths were never the whole story and are unchanged either way: `--token` / `FP_TOKEN` and `--api-key` / `FP_API_KEY` never opened a file, so CI that authenticates by environment never enters any of this, and read-only commands still write nothing. 784 pass. Adoption is mutation-tested — deleting it fails four tests, including the unwritable-target case that exists precisely so a machine that cannot be migrated is not punished for it. Docs: CHANGELOG, README and the agent skill all said "you will be asked to log in once" and now say the opposite, because they now describe the opposite. The enterprise CLI doc lives in FailproofAI/agenteye and is updated there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(fp-cli): adoption must not resurrect a session after logout `logout` writes a config with no token rather than deleting the file, so adoption has to key off the file being ABSENT or unparseable — never off "there is no token in it". Keying off the token would make every command after a logout re-adopt `~/.fp/cli.json` and sign the user back in, which is a worse bug than the one adoption fixes. The code already had it right; nothing asserted it, so the next person to simplify that condition would have found out from a user. Two tests: through `clear_token` as a logout really goes, and the invariant stated directly with a tokenless config planted by hand. Found by driving the built wheel rather than the source — the same pass that confirmed the six-process race leaves valid JSON, mode 0600 and no temp files. 786 pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(sdk): stop the test suite writing into the developer's real spool Two tests build an EventWriter with a long flush interval to inspect its queue and deliberately never flush it. But every writer registers itself in _writer._live_writers, and _flush_all_at_exit flushes ALL of them at interpreter exit — which runs after pytest has torn down its fixtures, so whatever redirection a test applied is already undone and get_base_dir() resolves to the real ~/.agenteye again. One run of the queue-cap tests deposited 162,751 synthetic events into a live spool, where a configured collector would have shipped them to a real dashboard as though an agent had emitted them. A per-test fixture cannot fix this, because the write happens after the last fixture is gone. So tests/conftest.py redirects AGENTEYE_HOME at import, straight into os.environ rather than through monkeypatch — pytest undoes monkeypatch at session end, and session end is still earlier than the flush. setdefault, so a developer already pointing at their own scratch spool keeps it; test_resolver_umbrella.py deletes the variable per test, so the resolution rules themselves are still tested against a clean environment. Both tests also take the `spool` fixture now, which makes the intent explicit rather than relying on the conftest alone. Verified: the real spool's file count is identical before and after a full suite run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(sdk)!: default the spool to ~/.failproofai/custom-agents The umbrella root could always have been selected, through an AGENTEYE_SPOOL_TO_FAILPROOFAI opt-in — except that opt-in ALSO required the directory to already exist, and nothing ever created it: not the SDK, not failproofaid, not either installer. customAgentsEventsDir in fp-home.ts is exported and called from nowhere, and the daemon computes the path only to watch it. So the branch never fired once and every shipped SDK wrote to ~/.agenteye regardless of what the operator set. The feature was documented, tested and unreachable. Resolution order is now: 1. set_base_dir() explicit 2. $AGENTEYE_HOME escape hatch 3. ~/.failproofai/custom-agents default WHY THIS IS SAFE ON failproofaid: it watches BOTH roots and always has (spool_dirs in crates/fpai-collect/src/config.rs is built from custom_agents_events_dir() AND agenteye_events_dir(), both kept indefinitely). So this changes which directory the files land in and nothing else. Batches already spooled under ~/.agenteye/events are not orphaned — they stay put and are still collected; that directory simply stops growing. WHAT BREAKS: a host running the older agenteye-collector, which resolves $AGENTEYE_HOME or ~/.agenteye and nothing else (collector/src/config.rs, base_dir(), verified — it has no reference to failproofai at all). There the new default writes where it does not look, silently. That host sets AGENTEYE_HOME=~/.agenteye, which is the documented escape hatch precisely because both daemons honour it and so it cannot itself desynchronise them. demo-agent in the AgentEye repo is exactly this shape and needs the matching ENV line; that change is on the other side. AGENTEYE_SPOOL_TO_FAILPROOFAI is retired rather than kept as a no-op — anyone who exported it was asking for this and now has it. A new test asserts no module reads it, checked over os.environ lookups rather than source text: the frozen-strings guard was passing on a mention of the name in a comment while the variable itself was being deleted, which is the same vacuous-pass class the guard exists to catch. failproofai_custom_agents_dir() returns Path instead of Path | None and no longer checks existence — that check is what made the opt-in dead, since a spool root that must pre-exist can never be where a first batch is written. The writer already mkdirs what it is about to write into. Verified on the built wheel in clean containers (3.10 and 3.14): the default resolves and creates the umbrella on first write, AGENTEYE_HOME still redirects to the legacy root, and the retired variable is inert. The cross-language contract test reads the Rust and the TypeScript directly and passes with FAILPROOFAI_SDK_REQUIRE_CONTRACT=1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(sdk): pin what the SDK may create inside ~/.failproofai The spool root moved into a directory the CLI and the daemon own, so "make the directory work" is no longer the whole requirement: a machine that has only ever run this SDK must be indistinguishable, to every other component, from a machine that has run nothing. detectLayout() in src/hooks/fp-config.ts is why. It reads VERSION, config.json, config.toml and layout 1's seven markers to decide whether a home is absent, current, stale or future — and a `stale` verdict is what authorises resetHome(), which deletes files. Creating any of those landmarks from here would hand the CLI a half-built home it believes it wrote. Verified against the real detectLayout(): a home holding only custom-agents/ returns {kind: "absent"} with isConfigured() false. Pinned from this side so a regression fails in the SDK's own suite rather than in the CLI's, later. The three machine states each assert the EXACT set of paths that appear, not merely that the events directory exists — that weaker assertion passes just as happily when a VERSION file appears beside it: * spool already present -> exactly one new batch file * home present, no spool -> exactly custom-agents/ + events/ + the batch * nothing present -> exactly the home + those two + the batch Plus: an existing configured home comes through byte-identical AND with mtimes unchanged (a rewritten config.json with identical content is still a component writing a file it does not own), directory modes are owner-rwx and not world-writable, an unwritable home raises and keeps the events queued rather than dropping them, and AGENTEYE_HOME still bypasses the umbrella without creating it. Negative-controlled both ways: stamping a VERSION file fails 5 of these, creating a sibling directory fails 6. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(sdk): four review findings — durability, u32 range, NaN, cancellation All four verified against the code before fixing, all four fixed, each negative-controlled by reverting it and watching the new guard fail. HIGH — batches were atomically published but not durably committed. write_text() + os.replace() makes visibility atomic to readers and commits nothing to the platter, so a power loss could leave a correctly-named, zero-length .jsonl. The collector reads it, POSTs it, takes the 200 and then DELETES it (remove_file in crates/fpai-collect/src/uploader.rs) — permanent, silent loss. An asymmetry more than an oversight: this repo's own Rust spool writer has called sync_all() at this exact point from the start, with the same comment. Now fsync before the rename and fsync the parent directory after it; the second half matters because the reverse failure leaves the bytes on disk under a .tmp name the watcher ignores by design. HIGH — a measured duration_ms could violate the server's u32 contract. _validate_promoted_numeric refuses a CALLER anything outside 0..2**32-1 because pu32() stores NULL for the rest at 200 OK, while the SDK's own computation was unbounded — one field, two standards, depending on who produced it. Over the range: 2**32 ms is ~49.7 days, an ordinary lifetime for a human_wait or an agent_pause. Under it: these are wall-clock readings, so an NTP step backwards yields a negative interval that round() preserves. The four inline computations are one helper now, and an out-of-range interval is OMITTED with a warning rather than clamped — a clamped 49.7 days is indistinguishable from a measurement, and the reason this is computed rather than accepted is that a reported duration is unfalsifiable. MEDIUM — non-finite floats produced invalid JSON. json.dumps writes NaN, Infinity and -Infinity by default; they are a Python extension, not JSON. It does not raise on them, so the sanitising fallback never ran and the malformed line went out looking like a success. Both encode paths use allow_nan=False now, which turns a non-finite float into an ordinary encode failure, and _sanitize maps it to null. MEDIUM — the documented tool_call() bracket caught Exception, and asyncio.CancelledError inherits from BaseException. A cancelled async tool emitted tool_use with no tool_result, orphaning the event and its correlation slot. events.md's session bracket had the same gap; run() in the same file already used BaseException, which is what makes these an inconsistency rather than a policy. The except Exception around the emit call itself is unchanged on purpose — catching BaseException there would let telemetry block a Ctrl-C. Tests 277 -> 313, including tests/test_skill_snippets.py, which parses every fenced Python block in the skill and fails a handler that wraps an emit without catching BaseException — a documented snippet is code an agent copies into a real loop, and nothing else exercises it. Verified live against the running local stack, SDK -> daemon -> DASHBOARD (/v1/events, not the server's :8080) -> ClickHouse: a payload carrying NaN, inf, a reference cycle and a tuple key arrives as {"budget": null, "confidence": null, "label": "kept"} and {"cache": {"(1, 2)": "hit"}, "g": {"name": "node", "self": "<circular reference>"}}, with duration_ms matching the real interval. Both spool roots collected: the new ~/.failproofai/custom-agents and the legacy ~/.agenteye. All four fixes re-verified on the built wheel across Python 3.10-3.14. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(sdk): five bugs found by adversarial testing, two of them crashes Hunted rather than re-run: each was found by attacking a specific assumption, reproduced, fixed, and negative-controlled by reverting the fix and watching the new guard fail. 1. event.*() could raise KeyError INTO THE CALLER'S AGENT LOOP. _track_pending did len() -> next(iter()) -> del with nothing serialising the three, so two threads at a full _pending picked the same victim and the second del raised. 24 crashes per 30_000 calls across 10 threads. Only fires once the map is full — i.e. only in the long-running multi-agent process the cap exists for. Tolerant eviction now, and deliberately no lock: a lock held at a fork() is inherited locked by a thread the child does not have. 2. An exploding __repr__ re-opened the permanent spool wedge. _encode_entry caught (TypeError, ValueError, RecursionError), but default=str runs the caller's __repr__, which can raise anything. Those escaped and the batch was retried forever — the same wedge, a different exception type. Catches Exception now; never BaseException, so Ctrl-C still interrupts. 3. A non-string session_id/agent_id was dropped by the server at 200 OK ({"accepted":0,"skipped":1}, verified live). The SDK reported success and the collector deleted the batch. None is the realistic way in. Validated on all 15 methods; blank ids refused too, because those the server ACCEPTS and silently groups every event under one empty id. 4. A stuck write stranded one .tmp per flush cycle — ~170_000/day at the default interval, on the disk already in trouble, invisible because the watcher ignores them by extension. 5. A lone surrogate made the server skip the whole event. os.fsdecode and errors="surrogateescape" produce them and json.dumps escapes them happily, so nothing failed locally. Scrubbed with backslashreplace, reached via one substring scan so clean events keep the fast path. Also corrected two docs that contradicted the shipped resolver after the default moved: configure()'s docstring (the SDK's most-read) and README:46. Tests 313 -> 428. Verified live end to end against the local stack, SDK -> daemon -> DASHBOARD /v1/events -> ClickHouse: 1803 of 1804 events ingested with the one poison event dropped alone, and a payload carrying NaN, -inf, a lone surrogate, a null byte and 2**64 stored intact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(sdk): stop the spool-contract test aborting the suite outside the repo `REPO_ROOT = Path(__file__).resolve().parents[3]` raises IndexError on a shallower tree, and a shallower tree is precisely the packaged-sdist case that `_read_sibling` in the same file is written to handle — its docstring says "in a packaged sdist that is expected". Because it raised at IMPORT, pytest reported a collection error and stopped the entire run rather than skipping the one file that needs the repository. Reproduced by copying sdk/python somewhere on its own: 428 passing tests became `1 error`. So the graceful path was unreachable in exactly the situation it exists for. REPO_ROOT is now resolved defensively and the existing REQUIRE-driven skip/fail logic decides, as designed: 423 passed / 8 skipped outside the repository, 429 passed / 2 skipped inside it. Found by running the full suite on all five supported interpreters in clean containers, which is how the sdist layout got exercised at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(fp-cli): guard the credential path against drifting from the register ~/.failproofai/ is a governed layout. src/hooks/fp-home.ts declares it — "nothing outside this file may join a path onto the failproofai home" — and what actually keeps a reset off the CLI's session is its `user-typed` entry in HOME_CLASSES, because resettablePaths() is a FILTER OVER that table, not a list of things to keep. Nothing checked that the two sides agreed, and config.py said so itself above FPCLI_SUBDIR: "change one, change the other; nothing checks." Confirmed by experiment rather than assumed: renaming fpcliDir to "fp-cli" in the TypeScript and leaving Python untouched left 53 TS tests and 59 Python tests all passing, with the register describing a directory nothing writes and the real credential sitting at a path it had never heard of — safe only by accident, and only until somebody classifies its parent. tests/test_fp_home_contract.py reads fp-home.ts and pins the subdirectory name, the credential filename, the home directory, the FAILPROOFAI_HOME override, the `user-typed` classification, and the deliberate ABSENCE of a class on the directory itself (auditDir's rule: a user-typed parent would protect a cache added later, a derived parent would delete the session). It mirrors the SDK's test_spool_contract.py next door, including the parts that stop a source-reading test passing vacuously: every pattern must match exactly once, the anchors are asserted separately, and CI sets FP_CLI_REQUIRE_CONTRACT=1 so a moved register fails instead of skipping. The REPO_ROOT resolution is guarded too — the SDK's version raised IndexError at import on a shallower tree, which aborts a whole suite instead of skipping one file. Five negative controls, each failing the right test: rename the directory, rename the file, downgrade the class to `derived`, restructure so the regexes match nothing, and classify the directory as a whole. Verified end to end as well: a real `fp` session planted in a populated home survives a real resettablePaths() reset, while audit/cache beside it is removed. Also corrects three comments that described the behaviour before ce2012db added session adoption — fp-home.ts ("did NOT migrate… costs a login"), config.py ("Never read, never written, never deleted") and test_failproofai_home.py's own docstring ("neither read nor deleted", 200 lines above the tests asserting it IS adopted) — and fp-home.ts's citation of home-classification.test.ts, a file that has never existed. The classification guard is real and lives in __tests__/hooks/fp-home.test.ts. fp-cli 786 -> 794. SDK 429, TS fp-home 53, workflow guards 73, tsc clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [fp-cli] fp policies, fp fleet and fp guardrails — the dashboard's enforcement pages from a terminal (#727) * feat(fp-cli): fp policies, fp fleet and fp guardrails Brings the dashboard's three cloud-managed-policy pages to the CLI, so a person or an agent can do from a terminal what previously needed a browser: write a policy, put it on machines, and see what it blocked. Three commands because they are three jobs, split the way the dashboard splits them — `/policies` authors a version, `/enforcement` decides which machines run it, `/guardrails` reports what happened. Folding them into one would merge "what we intended" with "what occurred", which is the distinction the pages exist to keep. ## The dangerous part, and what the CLI does about it `PUT /enforcement/deployments/{id}` REPLACES a machine's whole policy set. No merge, no server-side lock. The dashboard has no deploy form precisely because of this — it edits the machine's own current set, since a form that asks you to re-tick policies silently drops whatever you forget. So `fleet deploy` is a read-modify-write: it reads what the machine runs, applies `--add`/`--remove`, shows the FULL resulting set, and writes that. `--set` is the only way to drop what you did not name, and is refused alongside `--add`. Three further guards, each for a way this loses work silently: * A bare `--add` of a policy the machine already runs keeps its PINNED version rather than moving to the newest. A pin is deliberate; upgrading a fleet on a command whose author was reordering is not. * The diff shows unchanged rows. The write replaces everything, so the set on screen is the set that will exist — hiding untouched rows hides exactly the ones a mistake drops. * The generation read before the write must come back as `base + 1`. Anything else means somebody deployed in between, and a replace does not merge, so their change is already gone. The CLI refuses instead of reporting success. (`lib/enforcementFleet.ts`'s `staleness()` does the same check, after the fact; doing it before is the difference between a warning and a save.) ## Session-only, deliberately Every route here is ROOT-ONLY on the server — absent from `/v1` because `/v1` is internet-facing and these are operator writes. The commands refuse `--api-key` up front via `deny_in_key_mode` rather than translating a path that would 404, and `enforcement` is classified in `_V1_NO_EQUIVALENT` so the anti-drift test that guards that table stays honest. ## Input and output Policy source arrives as a path, `@path`, a pipe, `-`, or an interactive paste when stdin is a terminal — five shapes because that is where people keep a file they are about to publish, and refusing the clipboard means "save it first" for the most common one-off. Every command supports `--json`, in the SERVER's shape plus what the CLI computed (the deploy plan, the drift flag). Model `to_dict()` rather than `vars()`: the latter leaks Python snake_case into a contract that is camelCase everywhere else, which a harness discovers at runtime rather than in review. Tests: 42 covering the planner, the race check and source resolution — the pure logic, because that is where a wrong answer destroys a fleet's policy set. 836 pass overall. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fp-cli): a typo'd machine id minted a machine instead of failing Three findings from driving the commands against a running deployment rather than reading them. **A deploy to an unknown machine silently succeeded.** The server accepts a deploy to ANY id — that is how a machine can be pre-staged before it ever polls — so `fp fleet deploy no-such-box --add x` returned 0 and created `no-such-box`, carrying policies nothing will ever collect. The only trace is an extra row in `fleet list`. The dashboard cannot reach this state because it deploys to a machine picked from a list; a CLI takes free text, so the check belongs here. Unknown ids are now refused with exit 6, and `--create` allows the pre-staging case explicitly. **A bad `--since` exited 1, not 2.** `guardrails` raised a bare `ValueError` where every other bad flag value in the CLI is a usage error. Now `typer.BadParameter`, so it exits 2 like `--since` everywhere else. **Three key-mode refusals read "the versioned API an key authenticates against".** Grammar, but it is the message a CI job gets, so it is the sentence that has to survive being read once at 3am. Also adds the JSON-contract tests that would have caught an earlier slip in this branch: the models emitted `vars()`, which leaked Python snake_case into a contract that is camelCase everywhere else — the kind of difference a harness finds at runtime rather than in review. `to_dict()` now fixes the shape and the test asserts no key contains an underscore. Docs: the README gains a Cloud-managed policies section leading with the full-replace semantics, and the agent skill gains a `policies · fleet · guardrails` reference — the skill matters most here, because an agent reading only `--help` would meet `--set` without meeting what it drops. The enterprise CLI doc lives in FailproofAI/agenteye and is NOT updated here. 838 pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fp-cli): two renderer strings that stated things that were not true Both found by looking at real output rather than at the code. The deploy footer said "1 policies after this change". Pluralisation, but this line is the summary of a destructive full-replace, and a line that rea…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Rebuilds the documentation as one product, then reorders it around the loop that product actually is: observe every run → score it → find the failures nobody wrote a rule for → prevent them → confirm the next audit is clean.
Two rounds of work, one PR:
/cloud/*, and documenting the integration surface that lived only in--help.Why round 2
Three problems compounded, and none of them was a factual error:
/cloud/getting-startedredirected to an overview page, getting an API key was an off-site link on four pages, and "send your first trace" existed only inside a 447-line SDK reference whose install step is "talk to your contact". A new cloud user could not self-serve.cloud/overviewenumerated an Observe/Analyze/Govern sidebar, four other pages named that grouping in prose, and four slugs disagreed with what the dashboard calls them.The new shape
Two tabs — the journey, and reference. That alone takes 28 pages out of the first sidebar a new reader scrolls.
Five sequential onboarding pages, from nothing to a deployed policy in about ten minutes. Each carries the minimum commands and links out for depth rather than restating the reference pages.
start/sign-upevents:add+policies:pull) and why only those twostart/first-tracefailproofai config --connect; custom agent → Python SDK. Ends with "you should see a row in Sessions" and what to do if you don'tstart/historyharness add-path,backfill --since, and what is already gone from diskstart/first-auditnpx -y failproofai auditas the free local on-rampstart/first-policyTwo rules that now run through every page
There are exactly two ways to plug in, and the docs say so wherever a reader has to choose — on
introduction, inhow-it-works, at the top ofstart/first-trace, inconcepts, and in the nav ordering. It previously existed as one buried<Note>. The SDK is Python only; the docs say that rather than implying parity.The product is FailproofAI Cloud in prose;
agenteyeappears only as something you type. Both places a reader meets that name (cloud/cli,cloud/sdk) now say why in one clause.failproofai config --helpprinted a third spelling,FAILPROOF CLOUD.Renames
Four slugs now match what the dashboard calls them, with redirects in all 15 language trees:
/cloud/incidents/cloud/issues/cloud/managed-policies/cloud/policy-editor/cloud/fleet/cloud/deploy-policies/cloud/access/cloud/admin35 existing redirects pointed at two of those old destinations and were repointed. 461 in-body links across all 15 trees were rewritten — a class CI does not catch (
mintlify validateresolves nav links only;validate:mdxresolves assets only).Corrections against the shipped product
issues, notincidents(add_typer(..., name="issues")).auditsandusage. Both are now documented, verified againstaudits_cmds.py/usage_cmds.py.cloud/admin.cloud/performancedescribed Models / Tools / Hooks as standalone pages; they are tabs inside Dashboards.Telemetry,Error tracking,CLI and agents) now match the pages they point at.Name collisions, retired
concepts.mdxended with a<Note>apologising that "audit" meant two features. It is gone, because the titles now disambiguate: Local audit vs Audits, Local dashboard vs Charts and boards.Evaluations→ Online evals andEvaluators→ Build an evaluator, so the benefits page and the build guide no longer share a name.Translations
Locale page bodies are unchanged and stay in their current wording. Files were
git mv'd in lockstep across all 14 locales, the 5 new pages were seeded from English with/<lang>/link rewriting, and locale navs were regenerated withcli.ts --update-nav. The scheduled translation job sees the path-keyed cache misses and rewrites them on its next pass. No LLM spend in this PR.Verification
Read end to end against a local
mintlify dev: all five onboarding pages resolve, every "Next →" lands, and all four old URLs 307 — including their locale forms (/zh/cloud/incidents).Known gap, flagged not fixed
cloud/sdkstates the SDK wheel is private — "talk to your Failproof AI contact" — whilecloud/clisays the CLI is on public PyPI under the same distribution name. So the custom-agent half ofstart/first-tracecannot be completed self-serve. The page leads with the harness path, which can, and marks the SDK path as requiring access. Making the SDK self-serve is a product change.🤖 Generated with Claude Code
https://claude.ai/code/session_01B4teTZnSkV9KESL6rD74on
Concision and sequence pass
/issuesdashboard route while retaining theincidents:*API permission names.Four-task onboarding
cloud/connectandcloud/captureremain available as reference pages but no longer appear as separate onboarding choices.Hermes review
1bb2f2262a42155ec6ca046ae105a5d2e18e7b9b1d8f31d926828f3bae215c58f5b35baa44acbff0gpt-5.6-terraSummary
Found four documentation correctness issues: the custom-agent path cannot upload events as written, the legacy history page names a nonexistent dashboard route, the 14 localized sidebars omit the new deployment task, and the README understates the policy count.
Changes
Validation
Passedjq empty docs/docs.json; static resolver for MDX internal routes and Markdown assets— docs.json parsed successfully; all 773 direct internal MDX routes resolve to an existing page or redirect, and referenced Markdown assets exist. (4s)Skippeddocker run … oven/bun:latest … bun install --frozen-lockfile --ignore-scripts; bun run validate:mdx; bun scripts/translate-docs/cli.ts --validate; focused Vitest files— No centralized validation command was configured. The isolated container dependency installation did not complete in this harness, so repository validations were not executed. (26s)Findings
No blocking findings.
4 advisory findings
start/deploy-policy(docs/docs.json:47-53), but the representative Chinese sidebar still lists the old sign-up/history sequence and ends atstart/first-policy(docs/docs.json:175-180). The same missing deployment page occurs in all 14 non-English navigation trees, so localized readers cannot discover the final onboarding step. (docs/docs.json:175)agenteye.configure()and then says that “The collector uploads those files” (docs/start/first-trace.mdx:73-107). It never installs/configuresfailproofaidor supplies itsevents:addcredential. The supported-harness branch does this withfailproofai config --connect; the collector configuration source identifies that connection flow as the writer of the ingest credential. A custom-agent user following this branch therefore only creates local JSONL files. (docs/start/first-trace.mdx:106)docs/start/history.mdx:79)BUILTIN_POLICIEScontains 40 entries and the updated policy documentation consistently states 40. (README.md:193)Open questions
None.
Policy overrides
None.