Skip to content

fix(notion-web): production-ready labels, multi-workspace, inference, usage (FINAL) - #7768

Merged
diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.49from
artickc:fix/notion-web-friendly-model-ids
Jul 19, 2026
Merged

diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.49from
artickc:fix/notion-web-friendly-model-ids

Conversation

@artickc

@artickc artickc commented Jul 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Final production-ready follow-up for the Notion AI Web (Unofficial/Experimental) cookie provider. Stack this after the merged foundation PRs so chat actually works end-to-end with real picker model names, multi-workspace Fable unlock, and honest token estimates.

What this PR finishes

  1. Friendly model ids — /v1/models shows fable-5 / Fable 5, not food codenames (acai-budino-high). Wire protocol still uses codenames via resolveNotionCodename.
  2. Plan-locked diagnostics — Fable disabled on personal plan is explained (Business/Enterprise required), not silently missing with no reason.
  3. Auto workspace — raw token_v2 only; getSpaces + multi-space getAvailableModels probe when space_id omitted.
  4. Fable workspace pick — score all workspaces; prefer the one where Fable is enabled (Business) over personal spaces where it is plan-locked.
  5. Working inference (live-verified) — createThread: true + threadId + config/context/user transcript + x-notion-space-id / x-notion-active-user-header. Fixes ValidationError 400 Invalid input from the old human-only / no-thread body.
  6. NDJSON parsing — modern patch / record-map agent-inference text; strip <lang …/>.
  7. Honest usage — Notion has no metering. Estimate from text + estimated: true; do not turn zeros into a flat USAGE_TOKEN_BUFFER (2000). Keep estimated through response sanitizer.

Notion PR stack (include all for a full notion-web)

PR Status Role
#7600 MERGED Foundation: add Notion AI Web provider (token_v2 → runInferenceTranscript)
#7696 MERGED Live model discovery via getAvailableModels
#7768 (this) OPEN Labels, multi-workspace/Fable, working inference, token estimates

Optional / related (not required for chat):

PR Status Role
#2959 MERGED Notion MCP context source (different feature: workspace tools, not AI chat)
#5399 MERGED Dead-code cleanup of unused notion client type

Upstream issue: #6758

Test plan

  • node --import tsx --test tests/unit/executor-notion-web.test.ts tests/unit/notion-web-models-discovery.test.ts tests/unit/chatcore-client-usage-buffer.test.ts tests/unit/usage-token-buffer.test.ts → 55/55 pass
  • Live: Business workspace auto-selected; fable-5 listed
  • Live: chat notion-web/fable-5 → 200 + content (e.g. PONG)
  • Live: usage scales (short ~7 tokens, longer ~65) — not constant 2000
  • Reviewer: confirm base release/v3.8.49 (notion-web not on main yet)

Operator notes

  • Paste raw token_v2 (optional: notion_user_id, notion_browser_id; space_id optional if multi-workspace auto-pick is OK).
  • Fable 5 needs a Business/Enterprise workspace where Notion does not return business_or_enterprise_plan_required.
  • Usage is estimated only — not billing truth from Notion.

@artickc
artickc requested a review from diegosouzapw as a code owner July 19, 2026 13:57

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the Notion AI Web provider and executor to use friendly, client-facing model aliases (e.g., fable-5, gpt-5.6-sol) instead of internal food codenames, while mapping them back to the internal codenames for API requests. It also updates the base URL to app.notion.com, supports camelCase spaceId in cookies, and updates the client version. The review feedback suggests importing NOTION_WEB_FALLBACK_MODELS in the executor, simplifying the cookie space ID extraction with a single regex, ensuring client-facing model IDs are normalized to their canonical friendly slugs, and caching the friendly-to-codename map to optimize performance.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread open-sse/executors/notion-web.ts Outdated
@@ -27,13 +27,15 @@
import { randomUUID } from "node:crypto";
import { BaseExecutor, type ExecuteInput } from "./base.ts";
import { makeExecutorErrorResult as makeErrorResult } from "../utils/error.ts";
import { resolveNotionCodename } from "../services/notionWebModels.ts";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Import NOTION_WEB_FALLBACK_MODELS to enable canonical model ID normalization in the executor.

Suggested change
import { resolveNotionCodename } from "../services/notionWebModels.ts";
import { resolveNotionCodename, NOTION_WEB_FALLBACK_MODELS } from "../services/notionWebModels.ts";

Comment on lines 125 to +128
const match = cookie.match(/(?:^|;\s*)space_id=([^;]+)/i);
return match ? match[1].trim() : "";
if (match) return match[1].trim();
const camel = cookie.match(/(?:^|;\s*)spaceId=([^;]+)/);
return camel ? camel[1].trim() : "";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Simplify the cookie space ID extraction by using a single case-insensitive regular expression that matches both space_id and spaceId (with or without underscore). This avoids running two separate regex matches and handles all case variations of both spellings in a single pass.

Suggested change
const match = cookie.match(/(?:^|;\s*)space_id=([^;]+)/i);
return match ? match[1].trim() : "";
if (match) return match[1].trim();
const camel = cookie.match(/(?:^|;\s*)spaceId=([^;]+)/);
return camel ? camel[1].trim() : "";
const match = String(cookie || "").match(/(?:^|;\s*)space_?id=([^;]+)/i);
return match ? match[1].trim() : "";

Comment thread open-sse/executors/notion-web.ts Outdated
Comment on lines +279 to +285
let clientFacingModel = typeof model === "string" ? model.trim() : "";
if (clientFacingModel.startsWith("notion-web/")) {
clientFacingModel = clientFacingModel.slice("notion-web/".length);
} else if (clientFacingModel.startsWith("nw/")) {
clientFacingModel = clientFacingModel.slice(3);
}
const modelId = clientFacingModel || notionCodename || "notion-ai";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Normalize the client-facing model ID to its canonical friendly slug (e.g., gpt-5.6-sol or fable-5) if it matches a known model in the fallback catalog. This ensures that the returned model field in the OpenAI-compatible chat completion response is always the canonical catalog ID, even if the client requested it using a legacy food codename or a display name.

    let clientFacingModel = typeof model === "string" ? model.trim() : "";
    if (clientFacingModel.startsWith("notion-web/")) {
      clientFacingModel = clientFacingModel.slice("notion-web/".length);
    } else if (clientFacingModel.startsWith("nw/")) {
      clientFacingModel = clientFacingModel.slice(3);
    }
    const canonicalModel = NOTION_WEB_FALLBACK_MODELS.find(
      (m) => m.id === clientFacingModel || m.notionCodename === notionCodename || m.id === notionCodename
    );
    const modelId = canonicalModel?.id || clientFacingModel || "notion-ai";

Comment on lines +426 to +444
export function resolveNotionCodename(
model: string | undefined | null,
extraModels: readonly NotionDiscoveredModel[] = []
): string {
let m = typeof model === "string" ? model.trim() : "";
if (!m || m === "notion-ai") return "";
// Strip provider prefixes added by /v1/models catalog.
if (m.startsWith("notion-web/")) m = m.slice("notion-web/".length);
else if (m.startsWith("nw/")) m = m.slice(3);
if (!m || m === "notion-ai") return "";

const map = buildNotionFriendlyToCodenameMap([
...NOTION_WEB_FALLBACK_MODELS,
...extraModels,
]);
// Unknown ids pass through as-is so a freshly discovered codename still works
// before the fallback table is updated.
return map.get(m) || map.get(m.toLowerCase()) || map.get(slugifyNotionDisplayName(m)) || m;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Pre-compute and cache the friendly-to-codename map for the fallback models as a module-level constant. Since resolveNotionCodename is called on every inference request and extraModels is usually empty, this avoids rebuilding the entire map of ~30 models on every single request, improving performance.

const FALLBACK_CODENAME_MAP = buildNotionFriendlyToCodenameMap(NOTION_WEB_FALLBACK_MODELS);

export function resolveNotionCodename(
  model: string | undefined | null,
  extraModels: readonly NotionDiscoveredModel[] = []
): string {
  let m = typeof model === "string" ? model.trim() : "";
  if (!m || m === "notion-ai") return "";
  // Strip provider prefixes added by /v1/models catalog.
  if (m.startsWith("notion-web/")) m = m.slice("notion-web/".length);
  else if (m.startsWith("nw/")) m = m.slice(3);
  if (!m || m === "notion-ai") return "";

  const map = extraModels.length > 0
    ? buildNotionFriendlyToCodenameMap([...NOTION_WEB_FALLBACK_MODELS, ...extraModels])
    : FALLBACK_CODENAME_MAP;

  // Unknown ids pass through as-is so a freshly discovered codename still works
  // before the fallback table is updated.
  return map.get(m) || map.get(m.toLowerCase()) || map.get(slugifyNotionDisplayName(m)) || m;
}

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Reviewed this as a follow-up to #7696. Ran the full test plan in an isolated worktree against your branch: node --import tsx/esm --test tests/unit/executor-notion-web.test.ts tests/unit/notion-web-models-discovery.test.ts → 37/37 pass. Also confirmed a clean merge against the current release/v3.8.49 tip (no conflicts) and that eslint on all 7 touched files is clean.

On the CI red: Fast Quality Gates / No new ESLint warnings / Unit Tests fast-path (1-4) / dast-smoke are failing identically on other unrelated open PRs right now (verified against #7781) — that's a repo-wide base-red on the queue, not something in this diff. Nothing to fix here on your end for that.

The change itself is solid: nice UX win making /v1/models show real picker labels (fable-5, gpt-5.6-sol, ...) instead of Notion's internal food codenames, while keeping the food codenames as the wire protocol and staying backward compatible with old codename requests. The workspace auto-detection + Fable-unlock scoring in selectBestNotionSpaceId is a genuinely useful addition too.

A few small non-blocking nits for a future pass, not required for this PR:

  1. withFriendlyNotionAliases() in notionWebModels.ts is now a pure identity no-op (return models;). Consider just removing the wrapper and calling sites directly rather than keeping a function that does nothing.
  2. buildNotionFriendlyToCodenameMap() rebuilds its ~24-entry reverse map on every resolveNotionCodename() call (i.e. every chat request) — cheap at this size, but worth memoizing if the catalog grows. (gemini-code-assist's automated review flagged the same thing.)
  3. NOTION_SPACE_CACHE has a 30-min TTL but only lazy eviction on next read of the same key — fine for typical usage, but a cap or periodic sweep would harden it for long-running processes juggling many tokens.

None of these block merging. Thanks for the fast follow-up — this closes the loop nicely on the codename UX issue.

artickc added 5 commits July 19, 2026 20:44
Catalog /v1/models now surfaces web-picker names (fable-5, gpt-5.6-sol)
instead of Notion food codenames (acai-budino-high, orange-mousse).
Food codenames stay internal via notionCodename + resolveNotionCodename
for runInferenceTranscript. Legacy codename requests still work; responses
echo the client-facing id.

Also points discovery/inference at app.notion.com (same host as the AI picker).

Follow-up to cheaper-inference#7696.
Notion returns Fable 5 (acai-budino-high) with isDisabled=true and
disabledReason=business_or_enterprise_plan_required. Keep it out of the
enabled catalog (requests would fail) but surface a discovery warning so
operators know why it is missing. Also warn when space_id is resolved via
getSpaces instead of the cookie.
Operators only need the raw token_v2 value. When space_id is omitted:
- getSpaces loads all workspaces (browser-like headers + user id)
- each workspace is probed via getAvailableModels
- the richest AI catalog wins

Also softens auth hints so they no longer demand a cookie blob with =.
Probe ALL workspaces instead of early-exiting on the first catalog with
>=8 models. Prefer spaces where Fable is enabled over personal spaces
where Notion returns isDisabled=business_or_enterprise_plan_required.
Cache the chosen spaceId for inference when cookie has no space_id.
- runInferenceTranscript: createThread+threadId, config/context/user
  transcript, space/user headers (fixes ValidationError 400)
- Parse modern NDJSON patch/record-map; strip lang tags
- Estimate usage from text (Notion has no metering); mark estimated
- Treat all-zero usage as missing; skip USAGE_TOKEN_BUFFER on estimated
- Keep estimated flag through response sanitizer (was stripped -> flat 2000)

Verified live: fable-5/gpt-5.6-sol chat 200; usage 7 / 65 not constant 2000.
@artickc
artickc force-pushed the fix/notion-web-friendly-model-ids branch from 51e7bfc to f8d60b4 Compare July 19, 2026 17:45
@artickc artickc changed the title fix(notion-web): real picker labels as primary model ids (follow-up #7696) fix(notion-web): production-ready labels, multi-workspace, inference, usage (FINAL) Jul 19, 2026
diegosouzapw and others added 2 commits July 19, 2026 16:12
…/execute under the complexity cap

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks @artickc — this is a solid follow-up to #7696: real picker labels, multi-workspace support, inference and usage wiring for notion-web. 🎉

One pre-merge adjustment: the new code pushed the cyclomatic-complexity ratchet from 2056 (tip) to 2064, over the 2059 baseline. I extracted helpers so the new functions clear the cap — discoverNotionModelsForExplicitSpace / discoverNotionModelsViaGetSpaces / collectUserSpaceEntry / collectFallbackSpaceIds in notionWebModels.ts, and resolveExecuteWorkspace / buildNotionCreateThreadRequestBody / buildNotionExecuteHeaders / sendNotionInferenceRequest plus the transcript/record-map/stream splits in notion-web.ts. Pure extraction, no behavior or signature changes.

Verified after the refactor: complexity back to 2056 (tip level), cognitive-complexity 890 (unchanged), and 45/45 tests green across executor-notion-web, notion-web-models-discovery and chatcore-client-usage-buffer.

Validated in local merge-train @ bcd6330 (FAST gates green: static + changed tests + vitest).

@diegosouzapw
diegosouzapw merged commit 7a68a79 into cheaper-inference:release/v3.8.49 Jul 19, 2026
5 checks passed
@artickc

artickc commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up commit (queue 502)

Also included: skip auto rate-limit for web-cookie providers (notion-web, perplexity-web, …).

They are stored as authType=apikey (cookie in the key field), so autoEnableApiKeyProviders was wrapping slow Notion chats in Bottleneck with the default maxWaitMs=15s. A concurrent/queued request then failed with RATE_LIMIT_QUEUE_TIMEOUT before the first call finished.

Explicit rateLimitProtection: true still works. Real API-key providers unchanged.

diegosouzapw added a commit that referenced this pull request Jul 19, 2026
… (owner-approved)

The /fix-prs validation-train sweep surfaced a cluster of otherwise-clean
contributor feature PRs (#6973/#7683/#7662/#7672/#7633/#7767) whose per-PR
+1/+2 own-growth collectively exceeded the tip's 3-unit complexity slack
(2056 vs 2059). This was the 4th such block of the day (#7695/#7747/#7768
each needed helper extraction earlier). Owner approved raising both ceilings
to give new-feature PRs breathing room: complexity to 2072 (combined-cluster
2068 + 4 headroom), cognitive to 900 (combined 896 + 4). Structural shrink
stays debt (#3501); tighten via --update next cycle.
diegosouzapw added a commit that referenced this pull request Jul 19, 2026
…ndpoint move (base-red unblock)

#7768 switched notion-web to app.notion.com without regenerating the golden,
turning unit shard 3/4 red for every PR on release/v3.8.49. Two-line regen,
reflects the deliberate production change.
diegosouzapw added a commit that referenced this pull request Jul 19, 2026
…rift again (#7798)

* docs(free-tiers): correct the headline to the 1.37B the catalog actually computes

The 2026-06-17 honesty correction landed 1.54B, but v3.8.42 reclassified longcat
from a 150M/mo recurring grant to a one-time 10M signup credit and the doc was
never resynced. Verified against computeFreeModelTotals() at every release tag
from 3.8.13 to HEAD: no free provider was lost by mistake.

* feat(quality): gate the free-tier headline against the live catalog

The README headlined ~1.6B free tokens/mo for seven releases after the catalog had
already been corrected down to 1.37B. No gate watched that number, so the drift was
invisible — check:docs-counts only covered providers, locales, executors, strategies,
oauth, a2a skills and cloud agents.

Adds a STRICT check that runs computeFreeModelTotals() (the same function behind
/api/free-tier/summary) and fails the build when README.md or FREE_TIERS.md publish a
headline that no longer rounds to it. Degrades to a skip if tsx is unavailable rather
than going falsely red.

The extractor is a whitelist: the theoretical ceiling (~10B), the historical ~1.94B and
per-model rows (~1.00B) are legitimate figures that must never trip the gate.

Also adds the biweekly-audit note under the README headline, so readers know the number
moves both ways and is what the catalog computes rather than a rounded-up best case.

* feat(quality): extend the counts gate to engines, MCP tools/scopes and CLI tools

The v3.8.49 audit found four more numbers that had silently drifted, all invisible to
CI because check:docs-counts only watched providers/locales/executors/strategies/oauth/
a2a/cloud-agents: 10->11 compression engines, 94->104 MCP tools, 30->31 scopes,
26->33 CLI tools.

Adds a generic makeNumberClaimValidator that reads every fact in ONE tsx subprocess via
the same functions the app serves (ENGINE_IDS, countUniqueMcpTools, the live scope union,
CLI_TOOLS) — never a hardcoded copy — with DATA_DIR redirected to a throwaway dir so
importing the MCP tool modules can't touch the operator's SQLite. Each check declares a
skip pattern so legitimate non-aggregate figures never trip it: per-module tool counts
('Memory tool definitions (3 tools)') and the CLI catalog total sitting next to the MCP
total. Degrades to a skip when tsx is unavailable rather than a false red.

7 new unit tests (all pass) covering the exact stale values this audit found and proving
per-module counts are ignored.

* docs(diagrams): sync the animated cards and mermaid sources to the audited v3.8.49 numbers

The README text was fixed in #7795 but the SVG cards and mermaid sources kept the
old numbers baked in — exactly the drift the readers see first.

- compression-pipeline.svg: 10 -> 11 engine cells (Omniglyph added as #8, matching
  the README alt text), re-spaced 51px cells, highlight cascade re-timed, the
  Caveman kill-dot repositioned inside its cell, default-stack bracket recentered
- free-tier-budget.svg: bar and grid rebuilt from computeFreeModelTotals() — 21 -> 19
  countable pools (LongCat-2.0 moved to one-time credit, Inclusion provider removed),
  huggingchat entry is now ERNIE 4.5 VL, kiro shows Claude Sonnet 4.5, signup credits
  ~616M -> ~626M (+longcat 10M pill), aria said 'about 1.6 billion' -> 1.4/2.0,
  lower sections shifted up 30px (viewBox 872 -> 842)
- promise-pillars.svg: 26 -> 33 coding agents
- mcp-tools-94.mmd -> mcp-tools-104.mmd: real per-collection unique contributions
  (42 base + memory 3 + skill 4 + githubSkill 3 + pool 6 + gamification 8 + plugin 8
  + notion 6 + obsidian 22 + compression 2), exported SVG regenerated, zh-CN ref synced
- request-pipeline.mmd: 17 -> 18 strategies, exported SVG regenerated
- README free-tier alt + docs/diagrams/README.md synced to the same numbers

Both edited cards pass validate-svg.sh and were render-verified at 4 timestamps
(animation runs; first frame is the finished composition).

* docs(env): register the 4 env vars missing from the .env.example contract (base-red unblock)

FREE_PROXY_AUTO_SYNC_ENABLED / FREE_PROXY_AUTO_SYNC_INTERVAL_MS (scheduler.ts) and
MITM_ROOT_CA_ENABLED / MITM_CERT_MODE (mitm manager/server, #6684) landed on
release/v3.8.49 without their .env.example + ENVIRONMENT.md entries, turning the
docs-gates job red for every PR on the branch. Documented with their real defaults
and the set-by-manager caveat for MITM_CERT_MODE.

* fix(dashboard): narrow the Codex session ParseResult with an equality check (base-red unblock)

#7725 landed 'if (!result.ok)' in OAuthModal — under this repo's strict:false,
tsc 6 only narrows a discriminated union on the equality form, so the negation
raised TS2339 (Property 'error' does not exist on ParseResult) and turned the
dashboard-typecheck gate red for every PR on release/v3.8.49. Runtime semantics
are identical (ok is a strict boolean).

Also ratchets the frozen baseline down 260 -> 259: the real fix here plus 3
baselined errors that other merges already fixed (CostOverviewTab TS2304,
SidebarTab TS2322, FreePoolTab TS2304). Baseline diff is deletions-only.

* fix(dashboard): keep OAuthModal within the frozen file-size cap

The narrowing comment pushed the file to 1032 > 1030 frozen; the rationale lives
in the previous commit message and the dashboard-typecheck gate itself guards the
'=== false' form from being refactored back to '!result.ok'.

* test(providers): align the grok-web credential assertion with the #7567 hint (base-red unblock)

#7713 added hintKey/hintFallback (proactive cf_clearance/User-Agent guidance) to the
grok-web web-session metadata without touching this test's deepEqual, turning unit
shard 2/4 red for every PR on release/v3.8.49. Rewritten in the same contract-only
style the file already uses for lmarena: structural fields stay strictly asserted,
the hint asserts key + intent (cf_clearance / User-Agent) without freezing operator
copy. Net stronger than before — the old assertion never checked the hint at all.

* test(golden): regenerate translate-path snapshot for the notion-web endpoint move (base-red unblock)

#7768 switched notion-web to app.notion.com without regenerating the golden,
turning unit shard 3/4 red for every PR on release/v3.8.49. Two-line regen,
reflects the deliberate production change.
@diegosouzapw diegosouzapw mentioned this pull request Jul 23, 2026
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
… usage (FINAL) (cheaper-inference#7768)

* fix(notion-web): use real picker labels as primary model ids

Catalog /v1/models now surfaces web-picker names (fable-5, gpt-5.6-sol)
instead of Notion food codenames (acai-budino-high, orange-mousse).
Food codenames stay internal via notionCodename + resolveNotionCodename
for runInferenceTranscript. Legacy codename requests still work; responses
echo the client-facing id.

Also points discovery/inference at app.notion.com (same host as the AI picker).

Follow-up to cheaper-inference#7696.

* fix(notion-web): explain plan-locked models like Fable 5

Notion returns Fable 5 (acai-budino-high) with isDisabled=true and
disabledReason=business_or_enterprise_plan_required. Keep it out of the
enabled catalog (requests would fail) but surface a discovery warning so
operators know why it is missing. Also warn when space_id is resolved via
getSpaces instead of the cookie.

* feat(notion-web): auto-detect workspace without pasting space_id

Operators only need the raw token_v2 value. When space_id is omitted:
- getSpaces loads all workspaces (browser-like headers + user id)
- each workspace is probed via getAvailableModels
- the richest AI catalog wins

Also softens auth hints so they no longer demand a cookie blob with =.

* fix(notion-web): pick Business workspace so Fable 5 is listed

Probe ALL workspaces instead of early-exiting on the first catalog with
>=8 models. Prefer spaces where Fable is enabled over personal spaces
where Notion returns isDisabled=business_or_enterprise_plan_required.
Cache the chosen spaceId for inference when cookie has no space_id.

* fix(notion-web): working inference + honest token estimates

- runInferenceTranscript: createThread+threadId, config/context/user
  transcript, space/user headers (fixes ValidationError 400)
- Parse modern NDJSON patch/record-map; strip lang tags
- Estimate usage from text (Notion has no metering); mark estimated
- Treat all-zero usage as missing; skip USAGE_TOKEN_BUFFER on estimated
- Keep estimated flag through response sanitizer (was stripped -> flat 2000)

Verified live: fable-5/gpt-5.6-sol chat 200; usage 7 / 65 not constant 2000.

* refactor(notion-web): extract helpers to keep discoverNotionWebModels/execute under the complexity cap

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
Co-authored-by: artickc <artickc@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
… (owner-approved)

The /fix-prs validation-train sweep surfaced a cluster of otherwise-clean
contributor feature PRs (cheaper-inference#6973/cheaper-inference#7683/cheaper-inference#7662/cheaper-inference#7672/cheaper-inference#7633/cheaper-inference#7767) whose per-PR
+1/+2 own-growth collectively exceeded the tip's 3-unit complexity slack
(2056 vs 2059). This was the 4th such block of the day (cheaper-inference#7695/cheaper-inference#7747/cheaper-inference#7768
each needed helper extraction earlier). Owner approved raising both ceilings
to give new-feature PRs breathing room: complexity to 2072 (combined-cluster
2068 + 4 headroom), cognitive to 900 (combined 896 + 4). Structural shrink
stays debt (cheaper-inference#3501); tighten via --update next cycle.
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…rift again (cheaper-inference#7798)

* docs(free-tiers): correct the headline to the 1.37B the catalog actually computes

The 2026-06-17 honesty correction landed 1.54B, but v3.8.42 reclassified longcat
from a 150M/mo recurring grant to a one-time 10M signup credit and the doc was
never resynced. Verified against computeFreeModelTotals() at every release tag
from 3.8.13 to HEAD: no free provider was lost by mistake.

* feat(quality): gate the free-tier headline against the live catalog

The README headlined ~1.6B free tokens/mo for seven releases after the catalog had
already been corrected down to 1.37B. No gate watched that number, so the drift was
invisible — check:docs-counts only covered providers, locales, executors, strategies,
oauth, a2a skills and cloud agents.

Adds a STRICT check that runs computeFreeModelTotals() (the same function behind
/api/free-tier/summary) and fails the build when README.md or FREE_TIERS.md publish a
headline that no longer rounds to it. Degrades to a skip if tsx is unavailable rather
than going falsely red.

The extractor is a whitelist: the theoretical ceiling (~10B), the historical ~1.94B and
per-model rows (~1.00B) are legitimate figures that must never trip the gate.

Also adds the biweekly-audit note under the README headline, so readers know the number
moves both ways and is what the catalog computes rather than a rounded-up best case.

* feat(quality): extend the counts gate to engines, MCP tools/scopes and CLI tools

The v3.8.49 audit found four more numbers that had silently drifted, all invisible to
CI because check:docs-counts only watched providers/locales/executors/strategies/oauth/
a2a/cloud-agents: 10->11 compression engines, 94->104 MCP tools, 30->31 scopes,
26->33 CLI tools.

Adds a generic makeNumberClaimValidator that reads every fact in ONE tsx subprocess via
the same functions the app serves (ENGINE_IDS, countUniqueMcpTools, the live scope union,
CLI_TOOLS) — never a hardcoded copy — with DATA_DIR redirected to a throwaway dir so
importing the MCP tool modules can't touch the operator's SQLite. Each check declares a
skip pattern so legitimate non-aggregate figures never trip it: per-module tool counts
('Memory tool definitions (3 tools)') and the CLI catalog total sitting next to the MCP
total. Degrades to a skip when tsx is unavailable rather than a false red.

7 new unit tests (all pass) covering the exact stale values this audit found and proving
per-module counts are ignored.

* docs(diagrams): sync the animated cards and mermaid sources to the audited v3.8.49 numbers

The README text was fixed in cheaper-inference#7795 but the SVG cards and mermaid sources kept the
old numbers baked in — exactly the drift the readers see first.

- compression-pipeline.svg: 10 -> 11 engine cells (Omniglyph added as cheaper-inference#8, matching
  the README alt text), re-spaced 51px cells, highlight cascade re-timed, the
  Caveman kill-dot repositioned inside its cell, default-stack bracket recentered
- free-tier-budget.svg: bar and grid rebuilt from computeFreeModelTotals() — 21 -> 19
  countable pools (LongCat-2.0 moved to one-time credit, Inclusion provider removed),
  huggingchat entry is now ERNIE 4.5 VL, kiro shows Claude Sonnet 4.5, signup credits
  ~616M -> ~626M (+longcat 10M pill), aria said 'about 1.6 billion' -> 1.4/2.0,
  lower sections shifted up 30px (viewBox 872 -> 842)
- promise-pillars.svg: 26 -> 33 coding agents
- mcp-tools-94.mmd -> mcp-tools-104.mmd: real per-collection unique contributions
  (42 base + memory 3 + skill 4 + githubSkill 3 + pool 6 + gamification 8 + plugin 8
  + notion 6 + obsidian 22 + compression 2), exported SVG regenerated, zh-CN ref synced
- request-pipeline.mmd: 17 -> 18 strategies, exported SVG regenerated
- README free-tier alt + docs/diagrams/README.md synced to the same numbers

Both edited cards pass validate-svg.sh and were render-verified at 4 timestamps
(animation runs; first frame is the finished composition).

* docs(env): register the 4 env vars missing from the .env.example contract (base-red unblock)

FREE_PROXY_AUTO_SYNC_ENABLED / FREE_PROXY_AUTO_SYNC_INTERVAL_MS (scheduler.ts) and
MITM_ROOT_CA_ENABLED / MITM_CERT_MODE (mitm manager/server, cheaper-inference#6684) landed on
release/v3.8.49 without their .env.example + ENVIRONMENT.md entries, turning the
docs-gates job red for every PR on the branch. Documented with their real defaults
and the set-by-manager caveat for MITM_CERT_MODE.

* fix(dashboard): narrow the Codex session ParseResult with an equality check (base-red unblock)

cheaper-inference#7725 landed 'if (!result.ok)' in OAuthModal — under this repo's strict:false,
tsc 6 only narrows a discriminated union on the equality form, so the negation
raised TS2339 (Property 'error' does not exist on ParseResult) and turned the
dashboard-typecheck gate red for every PR on release/v3.8.49. Runtime semantics
are identical (ok is a strict boolean).

Also ratchets the frozen baseline down 260 -> 259: the real fix here plus 3
baselined errors that other merges already fixed (CostOverviewTab TS2304,
SidebarTab TS2322, FreePoolTab TS2304). Baseline diff is deletions-only.

* fix(dashboard): keep OAuthModal within the frozen file-size cap

The narrowing comment pushed the file to 1032 > 1030 frozen; the rationale lives
in the previous commit message and the dashboard-typecheck gate itself guards the
'=== false' form from being refactored back to '!result.ok'.

* test(providers): align the grok-web credential assertion with the cheaper-inference#7567 hint (base-red unblock)

cheaper-inference#7713 added hintKey/hintFallback (proactive cf_clearance/User-Agent guidance) to the
grok-web web-session metadata without touching this test's deepEqual, turning unit
shard 2/4 red for every PR on release/v3.8.49. Rewritten in the same contract-only
style the file already uses for lmarena: structural fields stay strictly asserted,
the hint asserts key + intent (cf_clearance / User-Agent) without freezing operator
copy. Net stronger than before — the old assertion never checked the hint at all.

* test(golden): regenerate translate-path snapshot for the notion-web endpoint move (base-red unblock)

cheaper-inference#7768 switched notion-web to app.notion.com without regenerating the golden,
turning unit shard 3/4 red for every PR on release/v3.8.49. Two-line regen,
reflects the deliberate production change.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
… usage (FINAL) (cheaper-inference#7768)

* fix(notion-web): use real picker labels as primary model ids

Catalog /v1/models now surfaces web-picker names (fable-5, gpt-5.6-sol)
instead of Notion food codenames (acai-budino-high, orange-mousse).
Food codenames stay internal via notionCodename + resolveNotionCodename
for runInferenceTranscript. Legacy codename requests still work; responses
echo the client-facing id.

Also points discovery/inference at app.notion.com (same host as the AI picker).

Follow-up to cheaper-inference#7696.

* fix(notion-web): explain plan-locked models like Fable 5

Notion returns Fable 5 (acai-budino-high) with isDisabled=true and
disabledReason=business_or_enterprise_plan_required. Keep it out of the
enabled catalog (requests would fail) but surface a discovery warning so
operators know why it is missing. Also warn when space_id is resolved via
getSpaces instead of the cookie.

* feat(notion-web): auto-detect workspace without pasting space_id

Operators only need the raw token_v2 value. When space_id is omitted:
- getSpaces loads all workspaces (browser-like headers + user id)
- each workspace is probed via getAvailableModels
- the richest AI catalog wins

Also softens auth hints so they no longer demand a cookie blob with =.

* fix(notion-web): pick Business workspace so Fable 5 is listed

Probe ALL workspaces instead of early-exiting on the first catalog with
>=8 models. Prefer spaces where Fable is enabled over personal spaces
where Notion returns isDisabled=business_or_enterprise_plan_required.
Cache the chosen spaceId for inference when cookie has no space_id.

* fix(notion-web): working inference + honest token estimates

- runInferenceTranscript: createThread+threadId, config/context/user
  transcript, space/user headers (fixes ValidationError 400)
- Parse modern NDJSON patch/record-map; strip lang tags
- Estimate usage from text (Notion has no metering); mark estimated
- Treat all-zero usage as missing; skip USAGE_TOKEN_BUFFER on estimated
- Keep estimated flag through response sanitizer (was stripped -> flat 2000)

Verified live: fable-5/gpt-5.6-sol chat 200; usage 7 / 65 not constant 2000.

* refactor(notion-web): extract helpers to keep discoverNotionWebModels/execute under the complexity cap

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
Co-authored-by: artickc <artickc@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
… (owner-approved)

The /fix-prs validation-train sweep surfaced a cluster of otherwise-clean
contributor feature PRs (cheaper-inference#6973/cheaper-inference#7683/cheaper-inference#7662/cheaper-inference#7672/cheaper-inference#7633/cheaper-inference#7767) whose per-PR
+1/+2 own-growth collectively exceeded the tip's 3-unit complexity slack
(2056 vs 2059). This was the 4th such block of the day (cheaper-inference#7695/cheaper-inference#7747/cheaper-inference#7768
each needed helper extraction earlier). Owner approved raising both ceilings
to give new-feature PRs breathing room: complexity to 2072 (combined-cluster
2068 + 4 headroom), cognitive to 900 (combined 896 + 4). Structural shrink
stays debt (cheaper-inference#3501); tighten via --update next cycle.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…rift again (cheaper-inference#7798)

* docs(free-tiers): correct the headline to the 1.37B the catalog actually computes

The 2026-06-17 honesty correction landed 1.54B, but v3.8.42 reclassified longcat
from a 150M/mo recurring grant to a one-time 10M signup credit and the doc was
never resynced. Verified against computeFreeModelTotals() at every release tag
from 3.8.13 to HEAD: no free provider was lost by mistake.

* feat(quality): gate the free-tier headline against the live catalog

The README headlined ~1.6B free tokens/mo for seven releases after the catalog had
already been corrected down to 1.37B. No gate watched that number, so the drift was
invisible — check:docs-counts only covered providers, locales, executors, strategies,
oauth, a2a skills and cloud agents.

Adds a STRICT check that runs computeFreeModelTotals() (the same function behind
/api/free-tier/summary) and fails the build when README.md or FREE_TIERS.md publish a
headline that no longer rounds to it. Degrades to a skip if tsx is unavailable rather
than going falsely red.

The extractor is a whitelist: the theoretical ceiling (~10B), the historical ~1.94B and
per-model rows (~1.00B) are legitimate figures that must never trip the gate.

Also adds the biweekly-audit note under the README headline, so readers know the number
moves both ways and is what the catalog computes rather than a rounded-up best case.

* feat(quality): extend the counts gate to engines, MCP tools/scopes and CLI tools

The v3.8.49 audit found four more numbers that had silently drifted, all invisible to
CI because check:docs-counts only watched providers/locales/executors/strategies/oauth/
a2a/cloud-agents: 10->11 compression engines, 94->104 MCP tools, 30->31 scopes,
26->33 CLI tools.

Adds a generic makeNumberClaimValidator that reads every fact in ONE tsx subprocess via
the same functions the app serves (ENGINE_IDS, countUniqueMcpTools, the live scope union,
CLI_TOOLS) — never a hardcoded copy — with DATA_DIR redirected to a throwaway dir so
importing the MCP tool modules can't touch the operator's SQLite. Each check declares a
skip pattern so legitimate non-aggregate figures never trip it: per-module tool counts
('Memory tool definitions (3 tools)') and the CLI catalog total sitting next to the MCP
total. Degrades to a skip when tsx is unavailable rather than a false red.

7 new unit tests (all pass) covering the exact stale values this audit found and proving
per-module counts are ignored.

* docs(diagrams): sync the animated cards and mermaid sources to the audited v3.8.49 numbers

The README text was fixed in cheaper-inference#7795 but the SVG cards and mermaid sources kept the
old numbers baked in — exactly the drift the readers see first.

- compression-pipeline.svg: 10 -> 11 engine cells (Omniglyph added as cheaper-inference#8, matching
  the README alt text), re-spaced 51px cells, highlight cascade re-timed, the
  Caveman kill-dot repositioned inside its cell, default-stack bracket recentered
- free-tier-budget.svg: bar and grid rebuilt from computeFreeModelTotals() — 21 -> 19
  countable pools (LongCat-2.0 moved to one-time credit, Inclusion provider removed),
  huggingchat entry is now ERNIE 4.5 VL, kiro shows Claude Sonnet 4.5, signup credits
  ~616M -> ~626M (+longcat 10M pill), aria said 'about 1.6 billion' -> 1.4/2.0,
  lower sections shifted up 30px (viewBox 872 -> 842)
- promise-pillars.svg: 26 -> 33 coding agents
- mcp-tools-94.mmd -> mcp-tools-104.mmd: real per-collection unique contributions
  (42 base + memory 3 + skill 4 + githubSkill 3 + pool 6 + gamification 8 + plugin 8
  + notion 6 + obsidian 22 + compression 2), exported SVG regenerated, zh-CN ref synced
- request-pipeline.mmd: 17 -> 18 strategies, exported SVG regenerated
- README free-tier alt + docs/diagrams/README.md synced to the same numbers

Both edited cards pass validate-svg.sh and were render-verified at 4 timestamps
(animation runs; first frame is the finished composition).

* docs(env): register the 4 env vars missing from the .env.example contract (base-red unblock)

FREE_PROXY_AUTO_SYNC_ENABLED / FREE_PROXY_AUTO_SYNC_INTERVAL_MS (scheduler.ts) and
MITM_ROOT_CA_ENABLED / MITM_CERT_MODE (mitm manager/server, cheaper-inference#6684) landed on
release/v3.8.49 without their .env.example + ENVIRONMENT.md entries, turning the
docs-gates job red for every PR on the branch. Documented with their real defaults
and the set-by-manager caveat for MITM_CERT_MODE.

* fix(dashboard): narrow the Codex session ParseResult with an equality check (base-red unblock)

cheaper-inference#7725 landed 'if (!result.ok)' in OAuthModal — under this repo's strict:false,
tsc 6 only narrows a discriminated union on the equality form, so the negation
raised TS2339 (Property 'error' does not exist on ParseResult) and turned the
dashboard-typecheck gate red for every PR on release/v3.8.49. Runtime semantics
are identical (ok is a strict boolean).

Also ratchets the frozen baseline down 260 -> 259: the real fix here plus 3
baselined errors that other merges already fixed (CostOverviewTab TS2304,
SidebarTab TS2322, FreePoolTab TS2304). Baseline diff is deletions-only.

* fix(dashboard): keep OAuthModal within the frozen file-size cap

The narrowing comment pushed the file to 1032 > 1030 frozen; the rationale lives
in the previous commit message and the dashboard-typecheck gate itself guards the
'=== false' form from being refactored back to '!result.ok'.

* test(providers): align the grok-web credential assertion with the cheaper-inference#7567 hint (base-red unblock)

cheaper-inference#7713 added hintKey/hintFallback (proactive cf_clearance/User-Agent guidance) to the
grok-web web-session metadata without touching this test's deepEqual, turning unit
shard 2/4 red for every PR on release/v3.8.49. Rewritten in the same contract-only
style the file already uses for lmarena: structural fields stay strictly asserted,
the hint asserts key + intent (cf_clearance / User-Agent) without freezing operator
copy. Net stronger than before — the old assertion never checked the hint at all.

* test(golden): regenerate translate-path snapshot for the notion-web endpoint move (base-red unblock)

cheaper-inference#7768 switched notion-web to app.notion.com without regenerating the golden,
turning unit shard 3/4 red for every PR on release/v3.8.49. Two-line regen,
reflects the deliberate production change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants