Skip to content

refactor(gateway): split declared agent config from user-scoped auth state#207

Merged
buremba merged 1 commit into
mainfrom
refactor/split-declared-from-user-auth
Apr 17, 2026
Merged

refactor(gateway): split declared agent config from user-scoped auth state#207
buremba merged 1 commit into
mainfrom
refactor/split-declared-from-user-auth

Conversation

@buremba
Copy link
Copy Markdown
Member

@buremba buremba commented Apr 17, 2026

Summary

  • End the conflation of declared agent config (lobu.toml / SDK GatewayConfig.agents) with user-scoped OAuth/BYOK state. Declared agents move to an in-memory DeclaredAgentRegistry; user credentials move to a dedicated UserAuthProfileStore keyed by (userId, agentId). AgentSettingsStore now only holds runtime UI/sandbox agents.
  • Eliminates the "(from lobu.toml)" ghost auth profiles that lingered in Redis after a provider was removed from the file, and removes the startup file→Redis sync loop that produced them.
  • AuthProfilesManager.getBestProfile now resolves user → ephemeral → declared with expiry-aware dedupe, so an expired user OAuth no longer masks a valid declared fallback.

Notable changes

  • New: DeclaredAgentRegistry (in-memory, populated from files/SDK/hot-reload) and UserAuthProfileStore (Redis user:auth-profiles:{userId}:{agentId}, secrets at users/{userId}/agents/{agentId}/auth-profiles/{profileId}/{kind}).
  • AgentSettingsStore constructor slimmed to (redis) — no more encryption-key wiring or runtime-credential resolver.
  • Provider catalog throws when callers try to mutate providers on declared agents (provider list is declared in lobu.toml; edit the file and restart).
  • Token refresh job scans the new user-scoped store instead of agent:settings:*.
  • OAuth modules and the OAuth route plumb session.userId end-to-end through setCredentials / deleteCredentials.
  • gateway oauth CLI gains a --user flag (defaults to a reserved \$ADMIN principal).
  • Drops AgentSettings.authProfiles and the file→Redis seeding pass in core-services.

Migration

  • Old OAuth tokens stored under agent:settings:*.authProfiles are dropped. Users will need to re-OAuth after upgrade. No back-compat shim, per repo conventions.
  • One-shot startup cleanup deletes agent:settings:{agentId} for any agentId present in the registry.

Test plan

  • bun run typecheck (gateway + core)
  • make build-packages
  • bun test — 468 gateway pass, 102 core pass; new tests for DeclaredAgentRegistry, UserAuthProfileStore, expired-OAuth-vs-declared precedence, and declared settings flowing through getEffectiveSettings
  • make dev + FLUSHALL + ./scripts/test-bot.sh "@me hello" against careops example (declared Gemini)
  • Admin UI OAuth-connect a provider as userA; confirm user:auth-profiles:userA:careops exists and agent:settings:careops does not
  • Restart gateway; userA OAuth survives, declared provider unchanged
  • Edit examples/careops/lobu.toml to swap declared provider; restart; declared switches; userA token unchanged
  • ./scripts/test-bot.sh "@me upload hi" — Slack file upload lands in originating thread (the bug that prompted this work)

…state

Declared agents (lobu.toml / SDK GatewayConfig.agents) now live in an
in-memory DeclaredAgentRegistry instead of being synced into Redis on
startup. User OAuth/BYOK credentials move to a dedicated
UserAuthProfileStore keyed by (userId, agentId), eliminating the
"(from lobu.toml)" ghost profiles that lingered after a provider was
removed from the file. AgentSettingsStore now only holds runtime
UI/sandbox agents.

- DeclaredAgentRegistry replaces the file→Redis sync loop and the
  authProfile seeding pass in core-services.
- UserAuthProfileStore owns OAuth/BYOK persistence at
  user:auth-profiles:{userId}:{agentId} with secrets under
  users/{userId}/agents/{agentId}/auth-profiles/...
- AuthProfilesManager.getBestProfile now resolves user → ephemeral →
  declared, with expiry-aware dedupe so an expired user OAuth no longer
  masks a valid declared fallback.
- Provider catalog throws when mutating providers on declared agents.
- Token refresh job scans the new user-scoped store.
- OAuth modules and the oauth route plumb userId end-to-end.
- Drops AgentSettings.authProfiles and the encryption-key-bound
  AgentSettingsStore constructor; store now takes a single redis arg.
@buremba buremba merged commit 77da458 into main Apr 17, 2026
11 checks passed
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 283c66a034

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 214 to +220
const modules = getModelProviderModules();
if (installedProviderIds.size > 0) {
return modules.some(
(module) =>
installedProviderIds.has(module.providerId) && module.hasSystemKey()
);
if (installedProviderIds.size === 0) {
return modules.some((m) => m.hasSystemKey());
}

return modules.some((module) => module.hasSystemKey());
return modules.some(
(m) => installedProviderIds.has(m.providerId) && m.hasSystemKey()
);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Include user-scoped creds in provider readiness checks

This helper now treats an agent as configured only when it has declared credentials or a system key, but after this refactor most OAuth/BYOK data lives in UserAuthProfileStore per (userId, agentId). Callers such as the message bridges gate message processing on hasConfiguredProvider, so a user with a valid per-user token on an installed provider will still be rejected as "no provider configured" whenever no system key is set. This blocks normal chat execution for user-scoped auth setups and should consult user-scoped profiles (with user context) before returning false.

Useful? React with 👍 / 👎.

Comment on lines 213 to +217
async deleteSettings(agentId: string): Promise<void> {
const key = this.buildKey(agentId);

// Cascade-delete every secret owned by this agent's auth profiles.
// Using a prefix sweep catches credentials and refresh tokens for all
// profile IDs without requiring us to re-read + parse the JSON.
const secretsDeleted = await deleteSecretsByPrefix(
this.secretStore,
`agents/${agentId}/auth-profiles/`
);

// Drop ephemeral profiles too so a subsequent getProfiles doesn't
// surface stale in-memory entries after the agent is torn down.
this.ephemeralAuthProfiles.delete(agentId);

await this.delete(key);
this.logger.info(
`Deleted settings for agent ${agentId} (cascade-deleted ${secretsDeleted} secret(s))`
);
this.logger.info(`Deleted settings for agent ${agentId}`);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cascade-delete user auth profiles on agent deletion

deleteSettings now only removes agent:settings:{agentId} and ephemeral in-memory profiles, but it no longer removes persistent user auth records/secrets for that agent. Agent deletion routes still call this method as the settings cleanup step, so user:auth-profiles:{userId}:{agentId} entries (and their secret-store values) are left behind; if the agent ID is reused later, stale credentials can be unexpectedly applied. The delete path should also purge user-scoped auth profiles for the deleted agent.

Useful? React with 👍 / 👎.

buremba added a commit that referenced this pull request Apr 20, 2026
After the owletto monorepo merge, the build-app step in build-images.yml
fails on `bun run typecheck` because of three classes of drift between
upstream owletto and lobu:

- Hono 4.12 narrows `c.req.param('id')` to `string | undefined`
  (was `string` in 4.10). Guard parseInt with `?? ''` so the existing
  NaN check catches the missing-param case, and add early-return guards
  where the param flows into a typed call.
- BetterAuth 1.6.5 tightened `BetterAuthOptions.secret` and
  `database` typing; the inferred per-call return no longer assigns
  to `Auth<BetterAuthOptions>`. Conditionally spread `secret` only when
  defined and widen the cache-set cast through `unknown`.
- Hono's `c.body` Data type expects `Uint8Array<ArrayBuffer>`, but
  `fs.readFile` returns `Buffer<ArrayBufferLike>`. Copy into a fresh
  ArrayBuffer.

Also fix four broken relative imports in owletto-backend that still
referenced the pre-merge `../../packages/{web,worker}` and
`../../connectors` layout instead of the new
`packages/owletto-{web,worker,connectors}` paths.

Re-add `AgentSettings.authProfiles` in `@lobu/core` (removed in #207)
since the Postgres agent-config store in owletto-backend still
round-trips this column. The lobu gateway runtime continues to use
UserAuthProfileStore — this is a persistence-layer-only field.
buremba added a commit that referenced this pull request Apr 20, 2026
…218)

After the owletto monorepo merge, the build-app step in build-images.yml
fails on `bun run typecheck` because of three classes of drift between
upstream owletto and lobu:

- Hono 4.12 narrows `c.req.param('id')` to `string | undefined`
  (was `string` in 4.10). Guard parseInt with `?? ''` so the existing
  NaN check catches the missing-param case, and add early-return guards
  where the param flows into a typed call.
- BetterAuth 1.6.5 tightened `BetterAuthOptions.secret` and
  `database` typing; the inferred per-call return no longer assigns
  to `Auth<BetterAuthOptions>`. Conditionally spread `secret` only when
  defined and widen the cache-set cast through `unknown`.
- Hono's `c.body` Data type expects `Uint8Array<ArrayBuffer>`, but
  `fs.readFile` returns `Buffer<ArrayBufferLike>`. Copy into a fresh
  ArrayBuffer.

Also fix four broken relative imports in owletto-backend that still
referenced the pre-merge `../../packages/{web,worker}` and
`../../connectors` layout instead of the new
`packages/owletto-{web,worker,connectors}` paths.

Re-add `AgentSettings.authProfiles` in `@lobu/core` (removed in #207)
since the Postgres agent-config store in owletto-backend still
round-trips this column. The lobu gateway runtime continues to use
UserAuthProfileStore — this is a persistence-layer-only field.
@buremba buremba deleted the refactor/split-declared-from-user-auth branch April 21, 2026 21:41
buremba added a commit that referenced this pull request May 20, 2026
Picks up the Owletto menubar fixes that surfaced while testing the local
`lobu run` getting-started flow: Open-log button on runner failure, single
signed-out message, and selected-agent-only CONNECTED AGENTS header.
buremba added a commit that referenced this pull request May 20, 2026
Picks up the Owletto menubar fixes that surfaced while testing the local
`lobu run` getting-started flow: Open-log button on runner failure, single
signed-out message, and selected-agent-only CONNECTED AGENTS header.
buremba added a commit that referenced this pull request May 20, 2026
…-apply) (#987)

* fix(agent-worker): route to configured provider without splitting the model

When an agent has an explicitly configured provider, pass the model string
as-is to that provider instead of splitting on '/' and letting the prefix
override it. OpenRouter slugs like "anthropic/claude-sonnet-4" or
"openai/gpt-4o" are OpenRouter's own model namespace, not a directive to
switch to the anthropic/openai provider — splitting mis-routed them and crashed
the worker with "model not found for provider". The provider/model split now
applies only in auto/no-configured-provider mode.

* fix(cli,server): getting-started chat works without a manual apply

- lobu run (embedded) auto-applies the project lobu.toml after boot, so a
  scaffolded agent is usable via `lobu chat -c local` with no separate
  `lobu apply`. Best-effort; embedded/local only (never mutates an external DB).
- Agent API returns 404 "Agent <id> not found. Run `lobu apply` to deploy it."
  when a session targets an undeployed agent, instead of a bare 403; existing
  but unauthorized agents still return 403.
- lobu chat surfaces that 404 message; README quick-start updated to reflect
  zero-config embedded default + auto-apply.

Adds dev.ts to the dynamic-import allow-list (apply graph stays out of
lobu run module-load path).

* fix(cli): pin lobu run auto-apply to the local server

Address review: auto-apply used the active CLI context, so if local-init /
sign-in failed to establish + activate the `local` context, `lobu run` could
apply the local project to whatever cloud/prod context was active. Now
announceLocalSignIn reports whether the local context was registered + made
active; auto-apply runs only when it was, and passes url=<gatewayUrl> so
resolveApiTarget matches the local context by URL (and refuses to send any
other context's credentials to a different URL). Auto-apply can no longer
target a remote org.

* test(cli): cover lobu run auto-apply gating; dedup resolver slug test

- Extract shouldAutoApplyLocalProject() pure predicate and unit-test the gating:
  applies only for embedded + local-context-ready + lobu.toml present; skips on
  failed sign-in (the prod-context guard), external backend, or no lobu.toml.
- Drop the duplicate openai/gpt-4o configured-provider case from
  model-resolver.test.ts; the full slug matrix lives in model-resolver-harden.

* style: apply biome formatting

* fix(cli): treat file:// embedded DATABASE_URL as local, not shared

The shared-DB boot guard called isSharedDatabaseUrl on any non-empty
DATABASE_URL, but file:// embedded paths (e.g. the menubar app passes
DATABASE_URL=file:///Users/me/lobu/data via the environment) parse to an empty
URL hostname — wrongly flagged "non-loopback / shared", so `lobu run` refused to
start and the app could never connect. Only postgres:// URLs can be shared, so
isSharedDatabaseUrl now returns false for non-external (file:/path) URLs.

The CLI getting-started escaped this because init pins DATABASE_URL in the
project .env (the guard exempts project-owned URLs); the menubar app sets it via
env with no project .env, so the guard fired.

* fix(server): keep agent session-create denials uniform (no enumeration)

Drops the actionable 404-for-missing-agent added earlier: returning 404 for a
missing agent id vs 403 for an existing-but-unauthorized one let a caller probe
ids to discover other tenants agents (security review finding). Denials are now
uniformly 403 regardless of existence. The getting-started UX is carried by
lobu run auto-applying the project, so the scaffolded agent exists by chat time
— the hint is unnecessary. Adds an enumeration-safe route test.

* docs(claude): sync CLAUDE.md guidance from main checkout

* chore: bump owletto pointer to 83c9a09 (mac menubar fixes #207)

Picks up the Owletto menubar fixes that surfaced while testing the local
`lobu run` getting-started flow: Open-log button on runner failure, single
signed-out message, and selected-agent-only CONNECTED AGENTS header.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant