Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions db/migrations/20260515120000_agents_per_org_pk.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
-- migrate:up
-- Phase A of moving `agents` from a globally-unique `id` PK to a per-org
-- composite PK `(organization_id, id)`. The application has always treated
-- agents as org-scoped (every read/delete/list filters by organization_id),
-- so the global PK is a latent footgun: two orgs cannot share an agent ID,
-- and a stale agent in one org silently blocks another org from using the
-- same name.
--
-- This phase is INTENTIONALLY NON-BREAKING. It only:
-- 1. Adds an `organization_id` column to each FK-holding child table
-- (NULLABLE — backfilled here, set NOT NULL in a later phase once the
-- app-code refactor lands so every INSERT writes the value).
-- 2. Backfills `organization_id` from agents (no orphan rows in prod).
-- 3. Adds a parallel UNIQUE constraint on `agents (organization_id, id)`
-- so the schema is ready for the eventual PK swap.
-- 4. Adds composite indexes on each child table so the upcoming
-- org-scoped query patterns are fast from day one.
--
-- The single-column PK on agents and the single-column FKs on child tables
-- stay in place. App code keeps working unmodified. The PK swap and FK
-- composite migration ship in a separate PR after the storage interfaces
-- are plumbed with `organization_id`.

-- ── 1. Add organization_id columns (nullable for now).
ALTER TABLE agent_grants ADD COLUMN organization_id text;
ALTER TABLE agent_connections ADD COLUMN organization_id text;
ALTER TABLE agent_users ADD COLUMN organization_id text;
ALTER TABLE agent_channel_bindings ADD COLUMN organization_id text;
ALTER TABLE grants ADD COLUMN organization_id text;
Comment on lines +25 to +29
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include scheduled_jobs in the agent org backfill

For deployments that have scheduled jobs created by an agent, this Phase A prep omits the existing scheduled_jobs.created_by_agent FK to agents(id) (db/schema.sql:4783). That leaves schedules without a backfilled agent-organization column path, so the later composite FK/PK swap cannot drop the single-column agents(id) dependency without either failing on that FK or preserving global agent-id uniqueness for scheduled agents; include scheduled_jobs in this migration and the embedded patch mirror while it already carries organization_id.

Useful? React with 👍 / 👎.


-- ── 2. Backfill from agents.
UPDATE agent_grants SET organization_id = a.organization_id FROM agents a WHERE agent_grants.agent_id = a.id;
UPDATE agent_connections SET organization_id = a.organization_id FROM agents a WHERE agent_connections.agent_id = a.id;
UPDATE agent_users SET organization_id = a.organization_id FROM agents a WHERE agent_users.agent_id = a.id;
UPDATE agent_channel_bindings SET organization_id = a.organization_id FROM agents a WHERE agent_channel_bindings.agent_id = a.id;
UPDATE grants SET organization_id = a.organization_id FROM agents a WHERE grants.agent_id = a.id;

-- ── 3. Parallel UNIQUE on agents (organization_id, id). The single-column
-- PK on (id) stays — this is purely additive and signals to readers
-- that org-scoped uniqueness is the eventual model. The PK swap in a
-- later migration will drop this UNIQUE and reuse the index for the
-- new composite PK.
ALTER TABLE agents
ADD CONSTRAINT agents_organization_id_id_key UNIQUE (organization_id, id);

-- ── 4. Composite indexes on child tables for upcoming org-scoped queries.
CREATE INDEX agent_grants_org_agent_idx ON agent_grants (organization_id, agent_id);
CREATE INDEX agent_connections_org_agent_idx ON agent_connections (organization_id, agent_id);
CREATE INDEX agent_users_org_agent_idx ON agent_users (organization_id, agent_id);
CREATE INDEX agent_channel_bindings_org_agent_idx ON agent_channel_bindings (organization_id, agent_id);
CREATE INDEX grants_org_agent_idx ON grants (organization_id, agent_id);

-- migrate:down
DROP INDEX IF EXISTS grants_org_agent_idx;
DROP INDEX IF EXISTS agent_channel_bindings_org_agent_idx;
DROP INDEX IF EXISTS agent_users_org_agent_idx;
DROP INDEX IF EXISTS agent_connections_org_agent_idx;
DROP INDEX IF EXISTS agent_grants_org_agent_idx;

ALTER TABLE agents DROP CONSTRAINT IF EXISTS agents_organization_id_id_key;

ALTER TABLE grants DROP COLUMN IF EXISTS organization_id;
ALTER TABLE agent_channel_bindings DROP COLUMN IF EXISTS organization_id;
ALTER TABLE agent_users DROP COLUMN IF EXISTS organization_id;
ALTER TABLE agent_connections DROP COLUMN IF EXISTS organization_id;
ALTER TABLE agent_grants DROP COLUMN IF EXISTS organization_id;
53 changes: 48 additions & 5 deletions db/schema.sql
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,8 @@ CREATE TABLE public.agent_channel_bindings (
platform text NOT NULL,
channel_id text NOT NULL,
team_id text,
created_at timestamp with time zone DEFAULT now() NOT NULL
created_at timestamp with time zone DEFAULT now() NOT NULL,
organization_id text
Comment on lines +116 to +117
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Enforce org/agent consistency for the new org-scoped columns.

These columns are backfilled once, but new writes can still store NULL or mismatched (organization_id, agent_id) pairs. That creates tenant-isolation drift once reads rely on org-scoped filters.

Suggested migration pattern
+-- 1) Keep new writes consistent (example approach: trigger to derive org from agents)
+-- 2) Add composite FKs (initially NOT VALID to reduce rollout risk)
+ALTER TABLE public.agent_channel_bindings
+  ADD CONSTRAINT agent_channel_bindings_org_agent_fkey
+  FOREIGN KEY (organization_id, agent_id)
+  REFERENCES public.agents (organization_id, id)
+  ON DELETE CASCADE
+  NOT VALID;
+
+ALTER TABLE public.agent_connections
+  ADD CONSTRAINT agent_connections_org_agent_fkey
+  FOREIGN KEY (organization_id, agent_id)
+  REFERENCES public.agents (organization_id, id)
+  ON DELETE CASCADE
+  NOT VALID;
+
+ALTER TABLE public.agent_grants
+  ADD CONSTRAINT agent_grants_org_agent_fkey
+  FOREIGN KEY (organization_id, agent_id)
+  REFERENCES public.agents (organization_id, id)
+  ON DELETE CASCADE
+  NOT VALID;
+
+ALTER TABLE public.agent_users
+  ADD CONSTRAINT agent_users_org_agent_fkey
+  FOREIGN KEY (organization_id, agent_id)
+  REFERENCES public.agents (organization_id, id)
+  ON DELETE CASCADE
+  NOT VALID;
+
+ALTER TABLE public.grants
+  ADD CONSTRAINT grants_org_agent_fkey
+  FOREIGN KEY (organization_id, agent_id)
+  REFERENCES public.agents (organization_id, id)
+  ON DELETE CASCADE
+  NOT VALID;

Also applies to: 135-135, 149-150, 193-194, 1111-1112, 2248-2249

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@db/schema.sql` around lines 116 - 117, Backfill any NULL organization_id
values, then add NOT NULL and a cross-row consistency CHECK to each table/column
pair that introduced organization_id/agent_id (e.g., the columns named
organization_id and agent_id near created_at) so new writes cannot store NULL or
mismatched pairs; specifically, run an UPDATE to set organization_id for
existing rows, then ALTER TABLE to set organization_id NOT NULL and add a CHECK
constraint like: agent_id IS NULL OR organization_id = (SELECT organization_id
FROM agents WHERE agents.id = agent_id) (or equivalent using your agents table
PK), and include the same pattern for the other affected columns/locations (the
other occurrences noted in the comment) so tenant isolation is enforced going
forward.

);

--
Expand All @@ -131,6 +132,7 @@ CREATE TABLE public.agent_connections (
error_message text,
created_at timestamp with time zone DEFAULT now() NOT NULL,
updated_at timestamp with time zone DEFAULT now() NOT NULL,
organization_id text,
CONSTRAINT agent_connections_status_check CHECK ((status = ANY (ARRAY['active'::text, 'stopped'::text, 'error'::text])))
);

Expand All @@ -144,7 +146,8 @@ CREATE TABLE public.agent_grants (
pattern text NOT NULL,
expires_at timestamp with time zone,
granted_at timestamp with time zone DEFAULT now() NOT NULL,
denied boolean DEFAULT false
denied boolean DEFAULT false,
organization_id text
);

--
Expand Down Expand Up @@ -187,7 +190,8 @@ CREATE TABLE public.agent_users (
agent_id text NOT NULL,
platform text NOT NULL,
user_id text NOT NULL,
created_at timestamp with time zone DEFAULT now() NOT NULL
created_at timestamp with time zone DEFAULT now() NOT NULL,
organization_id text
);

--
Expand Down Expand Up @@ -1104,7 +1108,8 @@ CREATE TABLE public.grants (
pattern text NOT NULL,
expires_at timestamp with time zone,
granted_at timestamp with time zone DEFAULT now() NOT NULL,
denied boolean DEFAULT false NOT NULL
denied boolean DEFAULT false NOT NULL,
organization_id text
);

--
Expand Down Expand Up @@ -2236,6 +2241,13 @@ ALTER TABLE ONLY public.agent_secrets
ALTER TABLE ONLY public.agent_users
ADD CONSTRAINT agent_users_pkey PRIMARY KEY (agent_id, platform, user_id);

--
-- Name: agents agents_organization_id_id_key; Type: CONSTRAINT; Schema: public; Owner: -
--

ALTER TABLE ONLY public.agents
ADD CONSTRAINT agents_organization_id_id_key UNIQUE (organization_id, id);

--
-- Name: agents agents_pkey; Type: CONSTRAINT; Schema: public; Owner: -
--
Expand Down Expand Up @@ -2777,12 +2789,24 @@ CREATE INDEX agent_channel_bindings_agent_id_idx ON public.agent_channel_binding

CREATE UNIQUE INDEX agent_channel_bindings_no_team_unique ON public.agent_channel_bindings USING btree (platform, channel_id) WHERE (team_id IS NULL);

--
-- Name: agent_channel_bindings_org_agent_idx; Type: INDEX; Schema: public; Owner: -
--

CREATE INDEX agent_channel_bindings_org_agent_idx ON public.agent_channel_bindings USING btree (organization_id, agent_id);

--
-- Name: agent_connections_agent_id_idx; Type: INDEX; Schema: public; Owner: -
--

CREATE INDEX agent_connections_agent_id_idx ON public.agent_connections USING btree (agent_id);

--
-- Name: agent_connections_org_agent_idx; Type: INDEX; Schema: public; Owner: -
--

CREATE INDEX agent_connections_org_agent_idx ON public.agent_connections USING btree (organization_id, agent_id);

--
-- Name: agent_connections_platform_idx; Type: INDEX; Schema: public; Owner: -
--
Expand All @@ -2795,6 +2819,12 @@ CREATE INDEX agent_connections_platform_idx ON public.agent_connections USING bt

CREATE INDEX agent_grants_agent_id_idx ON public.agent_grants USING btree (agent_id);

--
-- Name: agent_grants_org_agent_idx; Type: INDEX; Schema: public; Owner: -
--

CREATE INDEX agent_grants_org_agent_idx ON public.agent_grants USING btree (organization_id, agent_id);

--
-- Name: agent_secrets_expires_at_idx; Type: INDEX; Schema: public; Owner: -
--
Expand All @@ -2813,6 +2843,12 @@ CREATE INDEX agent_secrets_name_prefix_idx ON public.agent_secrets USING btree (

CREATE INDEX agent_secrets_org_id_idx ON public.agent_secrets USING btree (organization_id);

--
-- Name: agent_users_org_agent_idx; Type: INDEX; Schema: public; Owner: -
--

CREATE INDEX agent_users_org_agent_idx ON public.agent_users USING btree (organization_id, agent_id);

--
-- Name: agents_organization_id_idx; Type: INDEX; Schema: public; Owner: -
--
Expand Down Expand Up @@ -2903,6 +2939,12 @@ CREATE INDEX grants_agent_id_idx ON public.grants USING btree (agent_id);

CREATE INDEX grants_expires_at_idx ON public.grants USING btree (expires_at) WHERE (expires_at IS NOT NULL);

--
-- Name: grants_org_agent_idx; Type: INDEX; Schema: public; Owner: -
--

CREATE INDEX grants_org_agent_idx ON public.grants USING btree (organization_id, agent_id);

--
-- Name: idx_cc_classifier_version_id; Type: INDEX; Schema: public; Owner: -
--
Expand Down Expand Up @@ -4952,4 +4994,5 @@ INSERT INTO public.schema_migrations (version) VALUES
('20260514000000'),
('20260514120000'),
('20260514130000'),
('20260514160000');
('20260514160000'),
('20260515120000');
5 changes: 2 additions & 3 deletions examples/office-bot/lobu.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ key = "$Z_AI_API_KEY"
# Slack app. `channel` surface so it works in a channel, not just a DM.
[agents.food-ordering.preview.slack]
enabled = true
provider = "lobu-public"
surfaces = ["dm", "channel"]
code_ttl_minutes = 15

Expand Down Expand Up @@ -69,8 +68,8 @@ fail closed and deny with a reason.

[memory]
enabled = true
org = "office-bot"
name = "Office Bot"
org = "lobu-team"
name = "Lobu Team"
description = "Office-ops agents — first up: the weekday lunch order"
models = "./models"
data = "./data"
27 changes: 27 additions & 0 deletions packages/cli/src/commands/_lib/apply/apply-cmd.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import chalk from "chalk";
import { resolveContext } from "../../../internal/context.js";
import { parseEnvContent } from "../../../internal/env-file.js";
import { loadProjectLink } from "../../../internal/project-link.js";
import { CONFIG_FILENAME } from "../../../config/loader.js";
import { ApiError, ValidationError } from "../../memory/_lib/errors.js";
Expand Down Expand Up @@ -89,6 +90,26 @@ function checkRequiredSecrets(state: DesiredState): { missing: string[] } {
return { missing };
}

/**
* Merge `.env` values from the project dir into `process.env` (without
* overriding values already set in the shell). Quietly noop if the file
* doesn't exist or can't be parsed — `checkRequiredSecrets` will surface a
* clear "Missing required secret" error downstream.
*/
async function loadProjectEnvFile(cwd: string): Promise<void> {
const envPath = join(cwd, ".env");
let raw: string;
try {
raw = await readFile(envPath, "utf-8");
} catch {
return;
}
const vars = parseEnvContent(raw);
for (const [key, value] of Object.entries(vars)) {
if (process.env[key] === undefined) process.env[key] = value;
}
}
Comment on lines +93 to +111
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Handle .env parse errors so apply keeps the intended fallback behavior.

parseEnvContent(raw) can throw, and that currently escapes before checkRequiredSecrets. A malformed .env will hard-fail lobu apply instead of gracefully falling through to the existing missing-secret reporting path.

💡 Suggested patch
 async function loadProjectEnvFile(cwd: string): Promise<void> {
   const envPath = join(cwd, ".env");
   let raw: string;
   try {
     raw = await readFile(envPath, "utf-8");
   } catch {
     return;
   }
-  const vars = parseEnvContent(raw);
+  let vars: Record<string, string>;
+  try {
+    vars = parseEnvContent(raw);
+  } catch {
+    return;
+  }
   for (const [key, value] of Object.entries(vars)) {
     if (process.env[key] === undefined) process.env[key] = value;
   }
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/**
* Merge `.env` values from the project dir into `process.env` (without
* overriding values already set in the shell). Quietly noop if the file
* doesn't exist or can't be parsed `checkRequiredSecrets` will surface a
* clear "Missing required secret" error downstream.
*/
async function loadProjectEnvFile(cwd: string): Promise<void> {
const envPath = join(cwd, ".env");
let raw: string;
try {
raw = await readFile(envPath, "utf-8");
} catch {
return;
}
const vars = parseEnvContent(raw);
for (const [key, value] of Object.entries(vars)) {
if (process.env[key] === undefined) process.env[key] = value;
}
}
/**
* Merge `.env` values from the project dir into `process.env` (without
* overriding values already set in the shell). Quietly noop if the file
* doesn't exist or can't be parsed `checkRequiredSecrets` will surface a
* clear "Missing required secret" error downstream.
*/
async function loadProjectEnvFile(cwd: string): Promise<void> {
const envPath = join(cwd, ".env");
let raw: string;
try {
raw = await readFile(envPath, "utf-8");
} catch {
return;
}
let vars: Record<string, string>;
try {
vars = parseEnvContent(raw);
} catch {
return;
}
for (const [key, value] of Object.entries(vars)) {
if (process.env[key] === undefined) process.env[key] = value;
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/commands/_lib/apply/apply-cmd.ts` around lines 93 - 111, The
loadProjectEnvFile function currently calls parseEnvContent(raw) which can throw
and abort the apply flow; wrap the parseEnvContent call in a try/catch so
malformed .env content is swallowed (or logged at debug) and the function
returns early, preserving the existing behavior of not overriding process.env
and allowing checkRequiredSecrets to run; update the code paths around
parseEnvContent/raw so any parse exception does not propagate out of
loadProjectEnvFile.


// ── source_url: confirmed-before-fetch, https-only, bounded fetch ──────────

const CONNECTOR_SOURCE_MAX_BYTES = 2 * 1024 * 1024; // 2 MiB
Expand Down Expand Up @@ -808,6 +829,12 @@ function slugToTitle(slug: string): string {
export async function applyCommand(opts: ApplyOptions = {}): Promise<void> {
const cwd = opts.cwd ?? process.cwd();
const fetchImpl = opts.fetchImpl ?? fetch;

// Auto-load `.env` from the project dir so $VAR refs in lobu.toml resolve
// without the user having to `set -a; source .env; set +a`. Mirrors what
// `lobu dev` does. Existing process.env values win (don't clobber the shell).
await loadProjectEnvFile(cwd);

const { state, configPath } = await loadDesiredState({
cwd,
...(opts.only ? { only: opts.only } : {}),
Expand Down
4 changes: 2 additions & 2 deletions packages/cli/src/config/loader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ export async function loadConfig(cwd: string): Promise<LoadResult | LoadError> {
parsed = parseToml(raw) as Record<string, unknown>;
} catch (err) {
return {
error: `Invalid TOML syntax in ${CONFIG_FILENAME}`,
error: `Invalid TOML syntax in ${configPath}`,
details: [err instanceof Error ? err.message : String(err)],
};
}
Expand All @@ -46,7 +46,7 @@ export async function loadConfig(cwd: string): Promise<LoadResult | LoadError> {
const details = result.error.issues.map(
(issue) => `${issue.path.join(".")}: ${issue.message}`
);
return { error: `Invalid ${CONFIG_FILENAME}`, details };
return { error: `Invalid ${configPath}`, details };
}

return { config: result.data, path: configPath };
Expand Down
44 changes: 44 additions & 0 deletions packages/server/src/db/embedded-schema-patches.ts
Original file line number Diff line number Diff line change
Expand Up @@ -518,4 +518,48 @@ export const EMBEDDED_SCHEMA_PATCHES: EmbeddedSchemaPatch[] = [
`);
},
},
{
// Mirrors db/migrations/20260515120000_agents_per_org_pk.sql — phase A only.
// Adds organization_id (NULLABLE) to the 5 FK-holding child tables, backfills
// from agents, adds a parallel UNIQUE (organization_id, id) on agents, and
// creates composite indexes for upcoming org-scoped queries. The PK swap
// and FK composite migration ship in a later phase once the storage
// interfaces are plumbed with organization_id everywhere.
id: 'agents-per-org-pk-phase-a',
apply: async (sql) => {
for (const t of [
'agent_grants',
'agent_connections',
'agent_users',
'agent_channel_bindings',
'grants',
]) {
await sql.unsafe(`
ALTER TABLE public.${t}
ADD COLUMN IF NOT EXISTS organization_id text
`);
await sql.unsafe(`
UPDATE public.${t} c
SET organization_id = a.organization_id
FROM public.agents a
WHERE c.agent_id = a.id AND c.organization_id IS NULL
`);
await sql.unsafe(`
CREATE INDEX IF NOT EXISTS ${t}_org_agent_idx
ON public.${t} (organization_id, agent_id)
`);
}
await sql.unsafe(`
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint WHERE conname = 'agents_organization_id_id_key'
) THEN
ALTER TABLE public.agents
ADD CONSTRAINT agents_organization_id_id_key UNIQUE (organization_id, id);
END IF;
END$$;
`);
},
},
];
Loading