Skip to content

[SPARK-34343][SQL][TESTS] Add missing test for some non-array types in PostgreSQL#31456

Closed
sarutak wants to merge 3 commits intoapache:masterfrom
sarutak:test-for-some-types-postgresql
Closed

[SPARK-34343][SQL][TESTS] Add missing test for some non-array types in PostgreSQL#31456
sarutak wants to merge 3 commits intoapache:masterfrom
sarutak:test-for-some-types-postgresql

Conversation

@sarutak
Copy link
Member

@sarutak sarutak commented Feb 3, 2021

What changes were proposed in this pull request?

This PR added tests for some non-array types in PostgreSQL.
PostgreSQL supports wide range of types (https://www.postgresql.org/docs/13/datatype.html) and PostgresIntegrationSuite contains tests for some types but ones for the following types are missing.

  • bit varying
  • point
  • line
  • lseg
  • box
  • path
  • polygon
  • circle
  • pg_lsn
  • macaddr
  • macaddr8
  • numeric
  • pg_snapshot
  • real
  • time
  • timestamp
  • tsquery
  • tsvector
  • txid_snapshot
  • xml

NOTE: Handling money types can be buggy so this PR doesn't add tests for those types.

Why are the changes needed?

To ensure those types work with Spark well.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Extended PostgresIntegrationSuite.

@sarutak
Copy link
Member Author

sarutak commented Feb 3, 2021

For reviews: As I mentioned in the description, handling money types can be buggy and I'll fix it in #31442 .
Also, arrays of some types are not supported so I'll fix the issue in #31419 .

@SparkQA
Copy link

SparkQA commented Feb 3, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39422/

@SparkQA
Copy link

SparkQA commented Feb 3, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39422/

Copy link
Member

@srowen srowen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good if it passes.

@SparkQA
Copy link

SparkQA commented Feb 3, 2021

Test build #134835 has finished for PR 31456 at commit 2739020.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@github-actions github-actions bot added the SQL label Feb 3, 2021
@sarutak
Copy link
Member Author

sarutak commented Feb 4, 2021

SPARK-34357 (#31473) changed the type-mapping of JDBC time from Integer to Timestamp so I've modified the test for time in this PR to follow the previous change.
I also confirmed the modified test passes.

@SparkQA
Copy link

SparkQA commented Feb 4, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39472/

@SparkQA
Copy link

SparkQA commented Feb 4, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39472/

@SparkQA
Copy link

SparkQA commented Feb 4, 2021

Test build #134887 has finished for PR 31456 at commit 9536f75.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@maropu maropu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, I've checked that PostgresIntegrationSuite passes.

@maropu
Copy link
Member

maropu commented Feb 5, 2021

Thanks! Merged to master.

@maropu maropu closed this Feb 5, 2021
maropu pushed a commit that referenced this pull request Feb 5, 2021
…n PostgreSQL

### What changes were proposed in this pull request?

This PR added tests for some non-array types in PostgreSQL.
PostgreSQL supports wide range of types (https://www.postgresql.org/docs/13/datatype.html) and `PostgresIntegrationSuite` contains tests for some types but ones for the following types are missing.

* bit varying
* point
* line
* lseg
* box
* path
* polygon
* circle
* pg_lsn
* macaddr
* macaddr8
* numeric
* pg_snapshot
* real
* time
* timestamp
* tsquery
* tsvector
* txid_snapshot
* xml

NOTE: Handling money types can be buggy so this PR doesn't add tests for those types.

### Why are the changes needed?

To ensure those types work with Spark well.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Extended `PostgresIntegrationSuite`.

Closes #31456 from sarutak/test-for-some-types-postgresql.

Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com>
Signed-off-by: Takeshi Yamamuro <yamamuro@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants