feat: add Electric Cloud managed service support#1572
Conversation
…uting Route Electric shape requests to Electric Cloud (api.electric-sql.cloud) when clients send the `X-Electric-Backend: cloud` header, while preserving the existing self-hosted Fly.io backend as the default for older clients.
📝 WalkthroughWalkthroughThis pull request introduces cloud backend support for Electric SQL by adding conditional routing logic, new environment variables for cloud credentials, and a header to signal backend selection. The API route now routes requests to the cloud-hosted Electric service when configured, while maintaining backward compatibility with local backend deployments. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Tip Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🧹 Preview Cleanup CompleteThe following preview resources have been cleaned up:
Thank you for your contribution! 🎉 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@apps/api/src/app/api/electric/`[...path]/route.ts:
- Around line 27-43: The cloud-auth query parameter is wrong and non-null
assertions are used unnecessarily: in the block building originUrl (symbols:
useCloud, originUrl) replace the query key "source_secret" with "secret" when
adding the cloud secret (env.ELECTRIC_SOURCE_SECRET) so requests authenticate
correctly, and to remove the biome-ignore and "!" assertions, first read
env.ELECTRIC_SOURCE_ID and env.ELECTRIC_SOURCE_SECRET into local consts and use
a boolean check (e.g., const hasCloudCreds = !!(sourceId && sourceSecret) &&
request.headers.get("x-electric-backend")==="cloud") to guard setting search
params; keep env.ELECTRIC_SECRET for the non-cloud branch.
| const useCloud = | ||
| request.headers.get("x-electric-backend") === "cloud" && | ||
| env.ELECTRIC_SOURCE_ID && | ||
| env.ELECTRIC_SOURCE_SECRET; | ||
|
|
||
| const originUrl = useCloud | ||
| ? new URL("/v1/shape", "https://api.electric-sql.cloud") | ||
| : new URL(env.ELECTRIC_URL); | ||
|
|
||
| if (useCloud) { | ||
| // biome-ignore lint/style/noNonNullAssertion: guarded by useCloud check above | ||
| originUrl.searchParams.set("source_id", env.ELECTRIC_SOURCE_ID!); | ||
| // biome-ignore lint/style/noNonNullAssertion: guarded by useCloud check above | ||
| originUrl.searchParams.set("source_secret", env.ELECTRIC_SOURCE_SECRET!); | ||
| } else { | ||
| originUrl.searchParams.set("secret", env.ELECTRIC_SECRET); | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's examine the file in question
head -60 apps/api/src/app/api/electric/[...path]/route.ts | tail -40Repository: superset-sh/superset
Length of output: 1330
🏁 Script executed:
# Search for Electric Cloud references in the codebase
rg -i "electric.*cloud" --type ts --type tsx --type js -C 3Repository: superset-sh/superset
Length of output: 91
🏁 Script executed:
# Search for Electric API parameter usage
rg "(source_secret|source_id|ELECTRIC_SOURCE)" --type ts --type tsx --type js -C 2Repository: superset-sh/superset
Length of output: 91
🏁 Script executed:
# Check for tests related to Electric Cloud
fd -e test.ts -e test.tsx -e spec.ts | xargs rg -l "electric" -iRepository: superset-sh/superset
Length of output: 48
🏁 Script executed:
# Search for Electric Cloud references without file type restrictions
rg -i "source_secret|source_id" -C 2Repository: superset-sh/superset
Length of output: 1627
🏁 Script executed:
# Search for ELECTRIC env vars and configuration
rg "ELECTRIC_SOURCE|ELECTRIC_URL|ELECTRIC_SECRET" -C 2Repository: superset-sh/superset
Length of output: 6006
🏁 Script executed:
# Look for README, docs, or configuration files
fd -i "readme|doc|config|env" -e md -e txt -e example | head -20Repository: superset-sh/superset
Length of output: 210
🏁 Script executed:
# Search for Electric Cloud references in comments or URLs
rg "electric-sql|electric.*cloud" -iRepository: superset-sh/superset
Length of output: 8311
🏁 Script executed:
# Check if there's a package.json or any Electric-related dependencies
fd package.json -exec cat {} \; 2>/dev/null | head -100Repository: superset-sh/superset
Length of output: 46
🌐 Web query:
Electric SQL Cloud API source_secret parameter documentation
💡 Result:
Electric Cloud’s HTTP API does not document a source_secret query parameter. For Cloud, you pass:
source_id— your Cloud “Source ID”secret— your Cloud “Source secret” value, sent in the query string assecret=...(notsource_secret=...) [1]
Example (from Electric Cloud docs): .../v1/shape?...&source_id=...&secret=... [1]
In the official HTTP API (OpenAPI) spec, the authentication parameter is also named secret (with the older api_secret marked deprecated) [2].
Sources: Electric Cloud Usage docs [1], Electric HTTP API (OpenAPI / Redoc) [2].
🌐 Web query:
Electric SQL Cloud v1/shape endpoint parameters authentication
💡 Result:
ElectricSQL Cloud GET https://api.electric-sql.cloud/v1/shape — auth + key query parameters
Authentication (Electric Cloud)
Electric Cloud authenticates requests using query parameters:
source_id— identifies your registered Postgres sourcesecret— the source secret/token granting access
Example: .../v1/shape?table=items&offset=-1&source_id=...&secret=... (don’t put the secret in a public client; add it in a backend/edge proxy). [1]
Core /v1/shape parameters (shape definition + syncing)
From the HTTP API spec, the commonly used query parameters are: [2][3]
table(required): root table, optionally schema-qualified (foo.issues), otherwise defaults topublic. [2]where(optional): PostgreSQLWHEREclause to filter rows. [2]params[n](optional): parameter values for positional placeholders inwhere(e.g.,$1), passed asparams[1]=.... [2]columns(optional): comma-separated columns list to include. [2]
Sync control parameters:
offset(required):-1for initial sync from the beginning- a specific offset like
26800584_4to continue nowto skip historical data and start “from current” [2]
handle(required whenoffsetis not-1ornow): theelectric-handleyou got from the previous response. [2][3]live(optional, boolean): settrueto long-poll for realtime updates once up-to-date. [2][3]cursor(optional): value from the previous response’selectric-cursorheader (cache-busting for live polling). [2][3]live_sse(optional, boolean): withlive=true, use Server-Sent Events instead of long-polling. [2]
Change message shaping:
replica(optional:default|full): whether updates/deletes include only changed fields vs full rows. [2]log(optional:full|changes_only): whether to stream an initial snapshot or only changes. [2]
Subset snapshot parameters (optional)
The spec also defines subset__* parameters (e.g., subset__where, subset__params, subset__limit, subset__offset, subset__order_by) for requesting subset snapshots. [2]
Sources: Electric Cloud Usage docs [1], Electric HTTP API (OpenAPI/Redoc) [2], Client development guide (handle/offset/live/cursor flow) [3].
🏁 Script executed:
# Search for any tests or examples using the Electric Cloud endpoint
rg "api\.electric-sql\.cloud" -C 3Repository: superset-sh/superset
Length of output: 573
🏁 Script executed:
# Check the complete route.ts file to understand the full context
wc -l apps/api/src/app/api/electric/[...path]/route.tsRepository: superset-sh/superset
Length of output: 115
🏁 Script executed:
# Look for any comments or documentation about Electric Cloud
rg -B 5 -A 5 "useCloud|electric-backend"Repository: superset-sh/superset
Length of output: 3556
Change the Electric Cloud secret parameter name from source_secret to secret.
The code incorrectly uses source_secret as the query parameter name. According to the official Electric Cloud API documentation, the parameter must be named secret:
.../v1/shape?table=...&source_id=...&secret=...
The current line:
originUrl.searchParams.set("source_secret", env.ELECTRIC_SOURCE_SECRET!);Should be:
originUrl.searchParams.set("secret", env.ELECTRIC_SOURCE_SECRET!);With the current parameter name, all Cloud requests will fail authentication. This must be fixed before merging.
Additionally, the useCloud variable infers type string | false | undefined rather than boolean. This forces the two biome-ignore suppressions below. Extracting the env vars into local consts and using them directly in a single if branch eliminates the need for ! assertions and the suppression comments:
♻️ Optional refactor — avoid non-null assertions
- const useCloud =
- request.headers.get("x-electric-backend") === "cloud" &&
- env.ELECTRIC_SOURCE_ID &&
- env.ELECTRIC_SOURCE_SECRET;
-
- const originUrl = useCloud
- ? new URL("/v1/shape", "https://api.electric-sql.cloud")
- : new URL(env.ELECTRIC_URL);
-
- if (useCloud) {
- // biome-ignore lint/style/noNonNullAssertion: guarded by useCloud check above
- originUrl.searchParams.set("source_id", env.ELECTRIC_SOURCE_ID!);
- // biome-ignore lint/style/noNonNullAssertion: guarded by useCloud check above
- originUrl.searchParams.set("source_secret", env.ELECTRIC_SOURCE_SECRET!);
- } else {
- originUrl.searchParams.set("secret", env.ELECTRIC_SECRET);
- }
+ const sourceId = env.ELECTRIC_SOURCE_ID;
+ const sourceSecret = env.ELECTRIC_SOURCE_SECRET;
+ const isCloudRequest = request.headers.get("x-electric-backend") === "cloud";
+
+ let originUrl: URL;
+ if (isCloudRequest && sourceId !== undefined && sourceSecret !== undefined) {
+ originUrl = new URL("/v1/shape", "https://api.electric-sql.cloud");
+ originUrl.searchParams.set("source_id", sourceId);
+ originUrl.searchParams.set("secret", sourceSecret);
+ } else {
+ originUrl = new URL(env.ELECTRIC_URL);
+ originUrl.searchParams.set("secret", env.ELECTRIC_SECRET);
+ }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/api/src/app/api/electric/`[...path]/route.ts around lines 27 - 43, The
cloud-auth query parameter is wrong and non-null assertions are used
unnecessarily: in the block building originUrl (symbols: useCloud, originUrl)
replace the query key "source_secret" with "secret" when adding the cloud secret
(env.ELECTRIC_SOURCE_SECRET) so requests authenticate correctly, and to remove
the biome-ignore and "!" assertions, first read env.ELECTRIC_SOURCE_ID and
env.ELECTRIC_SOURCE_SECRET into local consts and use a boolean check (e.g.,
const hasCloudCreds = !!(sourceId && sourceSecret) &&
request.headers.get("x-electric-backend")==="cloud") to guard setting search
params; keep env.ELECTRIC_SECRET for the non-cloud branch.
…tric CORS - Store agent command separately in PendingTerminalSetup instead of appending to initialCommands, so setup script and agent run in separate panes within the same tab - Add X-Electric-Backend to CORS Access-Control-Allow-Headers (needed after Electric Cloud header was added in #1572) - Use wildcard for Access-Control-Expose-Headers to expose all Electric response headers (electric-offset, electric-handle, electric-schema, etc.) - Fix auth.apikeys where clause to use LIKE instead of ::jsonb cast which Docker Electric doesn't support
* fix(desktop): run agent prompt in separate terminal pane and fix Electric CORS - Store agent command separately in PendingTerminalSetup instead of appending to initialCommands, so setup script and agent run in separate panes within the same tab - Add X-Electric-Backend to CORS Access-Control-Allow-Headers (needed after Electric Cloud header was added in #1572) - Use wildcard for Access-Control-Expose-Headers to expose all Electric response headers (electric-offset, electric-handle, electric-schema, etc.) - Fix auth.apikeys where clause to use LIKE instead of ::jsonb cast which Docker Electric doesn't support * fix: explicitly list CORS expose headers and remove debug console.logs - Replace wildcard Access-Control-Expose-Headers with explicit list (wildcard is ignored when Access-Control-Allow-Credentials is true) - Remove stray console.log calls from trpc-storage.ts * fix: remove empty line left by console.log removal
1 participant
Summary
X-Electric-Backend: cloud) in the API Electric proxy to support Electric Cloud alongside the existing self-hosted Fly.io backendELECTRIC_SOURCE_IDandELECTRIC_SOURCE_SECRETenv vars (optional, already set in production)cloudheader so new releases use Electric Cloud while older clients keep hitting FlyTest plan
superset-electricFly app and remove old env varsSummary by CodeRabbit