chore(pgvector-embedded): rebuild prebuilt artifacts (v0.8.1)#972
Closed
github-actions[bot] wants to merge 15 commits into
Closed
chore(pgvector-embedded): rebuild prebuilt artifacts (v0.8.1)#972github-actions[bot] wants to merge 15 commits into
github-actions[bot] wants to merge 15 commits into
Conversation
…; earthdistance geo Local `lobu run` and the test backend now run a real embedded PostgreSQL 18 (embedded-postgres) instead of PGlite, with pgvector injected from the new @lobu/pgvector-embedded package. This removes the entire dual-backend: the LOBU_DISABLE_PREPARE single-connection pool pin, the conversation-lock no-op, the PGlite test harness, and the @electric-sql/pglite* deps are all gone — one engine everywhere. - DATABASE_URL is the sole backend selector: postgres:// = external; a path (or file:) = embedded PG with its cluster at <path>/.lobu/pgdata. - geo_lookup reimplemented on core-contrib cube+earthdistance (no PostGIS, works on every backend); restores the geo schema lost in the migration squash. - pgvector ships as prebuilt per-platform artifacts; a CI workflow rebuilds them, and only the control + pinned install SQL are vendored. - Test backend auto-spawns an ephemeral embedded PG when DATABASE_URL is unset, so `make test` needs no external database. - review.sh now unsets DATABASE_URL so the suite runs against an isolated embedded PG instead of whatever (often shared/prod) DB .env points at. BREAKING CHANGE: the local `lobu run` / test database engine is now a real embedded PostgreSQL instead of PGlite. Existing ~/.lobu PGlite data dirs are not migrated — a fresh embedded PG cluster is created. Production (external Postgres via DATABASE_URL) is unchanged.
…SE_URL `lobu init` now prompts for the database backend and writes an explicit DATABASE_URL into the scaffolded .env: - "Local embedded Postgres" (default) → DATABASE_URL=file://. — an isolated per-project cluster at ./.lobu/pgdata (gitignored). - "Connect to an existing Postgres" → prompts for the postgres:// URL. Both already work at runtime (the DATABASE_URL path/URL routing); init is just the front-door UX. Modernized .env.tmpl off the PGlite/LOBU_DATA_DIR model onto the file:// vs postgres:// scheme.
server.ts is now the single entry for both backends, branching on DATABASE_URL: postgres:// connects to an external Postgres; a path/file:// spawns a local embedded Postgres. The embedded boot lives in the new embedded-runtime.ts and is loaded ONLY via `await import(...)` in the embedded branch, so the external/prod path never resolves or loads the embedded-postgres binary even though it sits in node_modules. - Deleted start-local.ts; build one bundle (server.bundle.mjs) instead of two. - dev.ts drops bundle-switching (one bundle, self-selects on DATABASE_URL) and maps LOBU_DATA_DIR → an embedded DATABASE_URL, so the Mac app (which spawns `lobu run` with LOBU_DATA_DIR) keeps working with no Swift change. - dev:local + e2e-lobu-apply.sh point at server.ts. Validated both modes from the one bundle: embedded (file://) spawns PG, runs migrations, all 4 extensions live; external (postgres://) connects, passes the schema-version check, and never triggers the lazy embedded-postgres import (0 spawn lines). typecheck 0, gateway tests 24/24.
…elector LOBU_DATA_DIR was a second, redundant way to point at the embedded DB's data location. It's now purely the embedded-PG root (nothing else reads it — the old bootstrap-pat/ephemeral-key persistence claims were stale), so it folds entirely into DATABASE_URL=file://<dir>. - embedded-runtime.ts requires DATABASE_URL (no LOBU_DATA_DIR fallback). - user-config `dataDir` and the CLI `--data-dir` now map to DATABASE_URL=file://. - dev.ts drops the LOBU_DATA_DIR fallback; dev-native.sh + e2e-lobu-apply.sh set DATABASE_URL=file://<dir>. Validated: embedded boots from DATABASE_URL=file:// alone (cluster at <dir>/.lobu/pgdata), typecheck 0. (Mac app LocalLobuRunner switch to DATABASE_URL=file:// lands via the owletto submodule bump.)
…rops LOBU_DATA_DIR)
…istence (0c54927)
…le binaries The build matrix never produced non-darwin artifacts: - build.sh copied vector.control / vector--<v>.sql from the OS Postgres SHAREDIR, but the workflow only runs `make` (not `make install`), so those files never land there — every fresh-PG cell died on `cp: cannot stat '.../extension/vector.control'`. Stage straight from the built pgvector tree instead (vector.control is in-repo, the SQL is make-generated). Validated locally: reproduces the committed darwin-arm64 artifact byte-for-bit. - Pass OPTFLAGS="" so the redistributed binary drops pgvector's default `-march=native`; otherwise it bakes in the CI runner's microarch and SIGILLs on older user CPUs. - Linux install used a hand-rolled curl|gpg --dearmor pipe that flaked with "gpg: cannot open '/dev/tty'"; switch to the official pgdg apt.postgresql.org.sh setup script.
simpleQuery(query) just returned query — a no-op left over from the removed PGlite `.simple()` prepared-statement path. Unwrapped all 30 call sites (resolve_path.ts, multi-tenant.ts) and removed the export.
Built by build-pgvector-embedded.yml (pgvector v0.8.1, PG18, OPTFLAGS="" for portability). darwin-x64 follows once the macos-13 cell clears the runner queue; darwin-arm64 already committed.
…ded PG from runtime server now depends on @lobu/pgvector-embedded (workspace:*), but the image never copied or stubbed it — bun install + the builder tsc would have failed the first time this hit main (build-images runs only on main). COPY its manifest + source and build its dist before the server type-check. Then prune embedded-postgres + @embedded-postgres/* (~145MB platform binary) from node_modules after the bundle: prod always uses an external DATABASE_URL, so the embedded path (the only loader, via await import) never runs. Validated with a full --target builder build: install + dist + tsc + bundle succeed and the runtime image no longer carries the 145MB binary.
…al.ts comment drift applyDevProjectPathDefault had zero callers (the LOBU_DEV_PROJECT_PATH fallback is inlined in server.ts main()). Removed it, and corrected the server-lifecycle.ts / dev-vite.ts headers that still described a two-entry-point design — start-local.ts was deleted in this PR.
Resolves conflicts from main's context-config flatten (#955) + shared-DB guard refactor against this branch's embedded-PG work: - cli/index.ts: context add adopts main's flat --url/--cwd/--lifecycle (drops the nested --port/--host/--database-url/--data-dir options). - cli/commands/dev.ts: keep embedded helpers (isExternalDatabaseUrl, resolveEmbeddedDataRoot, mode selection) + adopt main's shouldRefuseSharedDatabaseUrl guard; drop dead BackendBundleKind (pglite removed) and the userServerConfig.databaseUrl/dataDir block (flattened away). - server/utils/user-config.ts: drop the dataDir→DATABASE_URL mapping (dataDir no longer in the flattened config schema). - server/start-local.ts: stays deleted (consolidated into server.ts). - owletto: bumped to merged 872a6ea (Keychain key + main's flat-context). Verified: cli + server tsc clean, dev.test.ts 12/12, init still writes DATABASE_URL to .env (embedded-vs-external feature intact).
The integration job (vitest under Node) failed with ERR_RESOLVE_PACKAGE_ENTRY_FAIL: the test backend imports @lobu/pgvector-embedded, so vite resolves its `import` condition (./dist/index.js) at transform time — but CI never built that package's dist (it's a new workspace dep in this PR). Build it alongside the other workspace packages, same as the Docker builder fix. Validated: full integration suite vs pg16 with dist freshly built → 0 resolve errors, 287/294 pass (the 5 fails are isolated-vm prebuild gaps local to the dev Mac; they load under CI's pinned Node 22). Also fix a stale comment: the test backend mirrors src/embedded-runtime.ts, not the deleted src/start-local.ts.
…os-13 GitHub retired the macos-13 (Intel) hosted runner in Dec 2025, so the darwin-x64 matrix cell queued indefinitely (no runner ever assigned). macos-15-intel is GitHub's current x86_64 macOS runner (Intel on macOS 15), the supported replacement until x86_64 macOS is dropped in 2027.
Member
|
Redundant — these artifacts are folded into #965 (the embedded-Postgres PR), built from the same workflow run. Closing. |
buremba
added a commit
that referenced
this pull request
May 25, 2026
…1051) The push trigger used a bare paths filter, which GitHub ignores for freshly-created refs (it can't diff them). So every release-please tag (lobu-vX.Y.Z) ran the rebuild and, because the binaries aren't byte-reproducible, opened a no-op 'rebuild artifacts' PR each release (#972, #974, #975, #977, #979, #980, #981, #996, #1002, #1005, #1025, #1029). Scope the push trigger to branches: [main] so tag refs no longer match, and drop the workflow file from paths (only build.sh changes the artifacts). workflow_dispatch still covers deliberate pgvector/PG bumps.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Regenerated by the build-pgvector-embedded workflow (pgvector v0.8.1, PG 18).