Add tests for FORCE_NULL * and FORCE_NOT_NULL * options for COPY FROM #7812
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
31 tasks
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## release-13.0 #7812 +/- ##
===============================================
Coverage ? 89.63%
===============================================
Files ? 274
Lines ? 59723
Branches ? 7451
===============================================
Hits ? 53534
Misses ? 4057
Partials ? 2132 |
These options already existed in PG17, and we have support them and have tests for them in multi_copy.sql. In PG17, their capability was extended to specify ALL columns at once using *. Citus performs the COPY correctly, as is validated by the added tests in this PR. Relevant PG commit: postgres/postgres@f6d4c9cf1
naisila
force-pushed
the
naisila/pg17_copy_forcenull
branch
from
87ea9dc to
4c8fcc5
Compare
onurctirtir
approved these changes
Dec 26, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These options already existed in PG17, and we support them and have tests for them in
multi_copy.sql.In PG17, their capability was extended to specify ALL columns at once using *.
Citus performs the COPY correctly, as is validated by the added tests in this PR.
Relevant PG commit:
postgres/postgres@f6d4c9cf1
Copy-pasting from Postgres documentation what these options do, such that the reviewer may better understand the tests added:
FORCE_NOT_NULL: Do not match the specified columns' values against the null string. In the default case where the null string is empty, this means that empty values will be read as zero-length strings rather than nulls, even when they are not quoted. If * is specified, the option will be applied to all columns. This option is allowed only inCOPY FROM, and only when usingCSVformat.FORCE_NULL: Match the specified columns' values against the null string, even if it has been quoted, and if a match is found set the value toNULL. In the default case where the null string is empty, this converts a quoted empty string intoNULL. If * is specified, the option will be applied to all columns. This option is allowed only inCOPY FROM, and only when usingCSVformat.FORCE_NULLandFORCE_NOT_NULLcan be used simultaneously on the same column. This results in converting quoted null strings to null values and unquoted null strings to empty strings.Explain it to me like I'm a 5-year-old, for a text column:
FORCE_NULLlooks for empty strings and registers them asNULLFORCE_NOT_NULLlooks for null values and registers them as empty strings.