Make the ZetaSQLTypeParser case insensitive #35
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merged
ppaglilla
added a commit
that referenced
this pull request
May 6, 2024
* Update ZetaSQL to version 2023.10.1 * Rewrite queries to fully quote all name paths before analysis * Avoid no longer necessary nesting of tables in catalogs * Update query rewritting to only re-quote name paths that refer to resources (i.e. tables, functions, etc) * Avoid no longer necessary nesting of functions in catalogs * Avoid no longer necessary nesting of TVFs in catalogs * Remove unnecessary slf4j dependency Closes #24 * Reduce code duplication in BigQueryCatalog * Reduce the amount of nesting for procedures in catalogs * Avoid duplicate code when creating different types of resources in catalogs * Use the DOUBLE type kind for BigQuery FLOAT64 columns Closes #25 * Added extractColumnLevelLineage for ResolvedQueryStmt Closes #28 * Changed the access modifier of ParentColumnFinder to public Closes #28 * Make ColumnLineageExtractor accept only concrete statement types Previously, the API for ColumnLineageExtractor used the method ::extractColumnLevelLineage(ResolvedStatement). Since introducing support for ResolvedQueryStmts, which needs specifying an output table separately; maintaining the generic ResolvedStatement API required making it confusing, since it would optionally need to accept an output table. This makes it so that teams building lineage applications need to explicitly determine the statements they support and call the corresponding ::extractColumnLevelLineage() method. Such as ::extractColumnLevelLineage(ResolvedInsertStmt) or ::extractColumnLevelLineage(ResolvedQueryStmt, String). * Bump development version to 0.5.0-SNAPSHOT * vuln-fix: Use HTTPS instead of HTTP to resolve deps CVE-2021-26291 (#30) This fixes a security vulnerability in this project where the `pom.xml` files were configuring Maven to resolve dependencies over HTTP instead of HTTPS. Weakness: CWE-829: Inclusion of Functionality from Untrusted Control Sphere Severity: High CVSS: 8.1 Detection: CodeQL & OpenRewrite (https://app.moderne.io/recipes/org.openrewrite.maven.security.UseHttpsForRepositories) Reported-by: Jonathan Leitschuh <[email protected]> Bug-tracker: JLLeitschuh/security-research#8 Detection: CodeQL (https://codeql.github.com/codeql-query-help/java/java-maven-non-https-url/) & OpenRewrite (https://app.moderne.io/recipes/org.openrewrite.maven.security.UseHttpsForRepositories) Reported-by: Jonathan Leitschuh <[email protected]> Bug-tracker: JLLeitschuh/security-research#8 Use this link to re-run the recipe: https://app.moderne.io/recipes/org.openrewrite.maven.security.UseHttpsForRepositories?organizationId=R29vZ2xl Co-authored-by: Moderne <[email protected]> * Make the type parser case insensitive (#35) The type parser was previously case sensitive, while SQL types are case insensitive. This went unnoticed for a while since upper-cased types are usually always used, but is fundamentally incorrect. Fixes #32 * Add reflection-based patching of GRPC's default max nesting depth (#36) ZetaSQL's Java API uses a GRPC service to call into the actual C++ implementation of ZetaSQL. By default, the serialization logic of that communication allows for a nesting depth in protobuf messages of up to 100. However, long queries can exceed that level of nesting and as a result cannot be analyzed by default. This implements a reflection-based patch that allows users to override that limit to a greater number. This is brittle by design and should be used with caution. Fixes #31 * Upgrade to zetasql-2024-03-01 and bump deps (#33) * Upgrade to zetasql-2024-03-01 and bump deps * Enable all features * Rollback Mockito to version 4.11.0 * Remove some v1.4 language options not supported by BigQuery --------- Co-authored-by: Pablo Paglilla <[email protected]> * Add missing @return in catalog Javadocs * Update version to v0.5.0 --------- Co-authored-by: Dion Ricky Saputra <[email protected]> Co-authored-by: Jonathan Leitschuh <[email protected]> Co-authored-by: Moderne <[email protected]> Co-authored-by: Erlend Hamnaberg <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The type parser was previously case sensitive, while SQL types are case insensitive. This went unnoticed for a while since upper-cased types are usually always used, but is fundamentally incorrect.