Rfc/dump #3
Closed
Rfc/dump #3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We would like to create a basic CLI for Supabase. At this stage we can assume that the user won't need to create orgs/projects from the CLI, but they will need to: - develop locally - push database migrations to their Postgres database
kiwicopple
commented
Feb 14, 2021
Comment on lines
+7
to
+13
| alter table "public"."users" add column | ||
| "data" | ||
| jsonb -- type | ||
| default -- default | ||
| ; | ||
| comment on column "public"."users"."data" is | ||
| ""; |
Member
Author
There was a problem hiding this comment.
The trickiest part of making the models "clean" seems to be figuring out how to group the comments with the column.
This looks clean:
create table users (
id uuid primary_key,
first_name text,
last_name text
)but then the comments have to sit below the actual table:
create table users (
id uuid primary_key,
first_name text,
last_name text
)
comment on column "public"."users"."firstname" is "Some comment";
comment on column "public"."users"."lastname" is "Some comment";As a result it may be better to create all the columns in separate alter statements
Member
There was a problem hiding this comment.
The problem with the ALTER COLUMNs is that they're too verbose. I think that our structure should make the point that "SQL is code". Verbosity won't aid in that goal, and instead it discourages looking into the generated SQL.
So the comments at the bottom look better to me.
Closed
Member
|
Closing this as we're transitioning to the Go CLI. |
nhickster
added a commit
to nhickster/supabase-cli
that referenced
this pull request
Oct 20, 2025
…nd cosmetic function changes ## Summary Fixes spurious schema differences detected by `supabase db pull` when comparing identical local and remote databases. Resolves two separate issues: 1. Column default normalization differences (ALTER TABLE statements) 2. Cosmetic-only function regeneration (CREATE OR REPLACE FUNCTION statements) ## Issues Resolved ### Issue supabase#1: Spurious REVOKE Statements (RESOLVED ✅) **Problem:** Migra generated hundreds of REVOKE/GRANT statements even when privileges were identical between remote and shadow databases. **Solution:** Removed `--with-privileges` flag for user schemas (public) in bash migra. User schemas rely on RLS policies rather than raw GRANTs, making privilege checking unnecessary and overly sensitive. **Files:** `internal/db/diff/templates/migra.sh` ### Issue supabase#2: Column Default Normalization (RESOLVED ✅) **Problem:** pg_get_expr() normalized column defaults differently based on search_path at query time, causing false differences: - With extensions in search_path: `uuid_generate_v4()` (unqualified) - Without extensions: `extensions.uuid_generate_v4()` (qualified) **Solution:** Three-part fix: 1. Query remote database's search_path before creating shadow (GetSearchPath) 2. Configure shadow database container with matching search_path via -c flag 3. Set PGOPTIONS environment variable for migra to ensure consistent pg_get_expr() normalization **Files:** - `internal/db/diff/diff.go`: GetSearchPath(), CreateShadowDatabaseWithSearchPath() - `internal/db/diff/migra.go`: PGOPTIONS environment variable for migra - `internal/db/pull/pull.go`: ALTER DATABASE SET search_path for shadow - `internal/db/start/start.go`: Conditional search_path in NewContainerConfig() ### Issue supabase#3: Cosmetic Function Regeneration (RESOLVED ✅) **Problem:** Migra generated 1,459 lines of CREATE OR REPLACE FUNCTION statements due to textual formatting differences in stored function definitions (dollar quote styles, schema qualification), even though functions were functionally identical. **Solution:** Post-processing filter (filterCosmeticFunctionChanges) that: - Parses migra output to detect and remove function-only changes - Preserves real schema changes (ALTER TABLE, CREATE TABLE, policies, triggers, etc.) - Handles three scenarios: 1. Pure cosmetic (only functions changed) → Skip migration entirely 2. Mixed (functions + real changes) → Filter functions, keep real DDL 3. Clean (no cosmetic changes) → Return unchanged **Files:** - `internal/db/diff/migra.go`: filterCosmeticFunctionChanges() and integration - `internal/db/pull/pull.go`: Graceful handling of errInSync (no stack trace) ## Technical Details ### Search Path Consistency Mechanism - **Remote query:** `SHOW search_path` retrieves target database's configuration - **Shadow creation:** Docker container starts with `-c search_path='<remote_value>'` flag - **Migra execution:** `PGOPTIONS=-c search_path=<remote_value>` ensures psycopg2 uses same path - **Result:** Both databases queried with identical search_path, producing consistent pg_get_expr() normalization ### Function Filtering Logic State machine parser that: - Tracks function boundaries (CREATE FUNCTION ... $function$) - Identifies real DDL statements (ALTER TABLE, CREATE POLICY, etc.) - Removes function blocks while preserving all other content - Returns filtered migration + boolean flag for informational messages ### User Experience Improvements - No empty migration files created when only cosmetic changes detected - Clean "No schema changes found" message (no stack traces) - Informational messages for filtered cosmetic changes - Proper handling of mixed migrations (functions + real changes) ## Debug Logging (Temporary) Multiple DEBUG statements added for troubleshooting during development. These provide visibility into: - Shadow database search_path at various stages - Column defaults in both target and shadow databases - Migration execution flow and search_path changes - Function filtering decisions **Note:** Debug logging can be removed once fix is verified in production use. ## Files Modified - `.gitignore` - Added /debug directory exclusion - `internal/db/diff/diff.go` - Search path detection and shadow database configuration - `internal/db/diff/migra.go` - Function filtering and PGOPTIONS for migra - `internal/db/diff/templates/migra.sh` - Conditional --with-privileges flag - `internal/db/pull/pull.go` - ALTER DATABASE search_path and graceful errInSync handling - `internal/db/start/start.go` - Conditional default search_path in container config - `pkg/migration/apply.go` - Migration execution diagnostics - `pkg/migration/file.go` - Session-level search_path and batch execution diagnostics ## Testing Tested with Nathan's Haddy project database containing: - 51 functions with SET search_path TO '' - Multiple tables with uuid_generate_v4() defaults - Complex RLS policies and triggers **Results:** - ✅ 0 ALTER TABLE statements (was 13) - ✅ 0 spurious function regenerations (was 1,459 lines) - ✅ 0 empty migration files - ✅ Clean "No schema changes found" message when schemas identical ## References - Issue: supabase#4068 - Related PR: (pending)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Mocks up a dump concept