Skip to content

[TVF] Support table functions with pushdown to connector 1/9#25018

Closed
mohsaka wants to merge 10 commits intoprestodb:masterfrom
mohsaka:tvf_port_11336
Closed

[TVF] Support table functions with pushdown to connector 1/9#25018
mohsaka wants to merge 10 commits intoprestodb:masterfrom
mohsaka:tvf_port_11336

Conversation

@mohsaka
Copy link
Copy Markdown
Contributor

@mohsaka mohsaka commented Apr 30, 2025

Description

Documentation and Release notes will be in the last PR. This is a new feature so it does not affect any current functionality.

Presto Port of
trinodb/trino#11336

This PR introduces partial support for Table Functions (including Polymorphic Table Functions).

Temporarily, only constant scalar arguments are supported (no Table or Descriptor arguments).
The supported execution path for a Table Function is to be pushed down into connector, and replaced with a TableHandle.

Later, Descriptor and Table arguments will be supported, and operator-based execution will be enabled for Table Functions.

This PR is part of the entire change which can be seen here.
#25017

Motivation and Context

#24648

Impact

Introduce Table Function SQL consistent with Trino and SQL Standard

Test Plan

SQL Parser tested via TestSqlParser.java

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== NO RELEASE NOTE ==

mohsaka and others added 10 commits April 30, 2025 08:37
Changes adapted from trino/PR#11336
Original commit: 2e00c8e64c32d6fdd813999b2e04b3b3415235c8
Author: kasiafi

Modifications were made to adapt to Presto including:
Addition of KEEP in the parser.
Adjustment of the TestSqlParser.java to apply to Presto concepts.
Switch from Trino's DataType based datatypes to Presto's String based datatypes.

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
Changes adapted from trino/PR#11336
Original commit: 395fd91c6480a993241eeabd9599873d0d05b24b
Author: kasiafi

Modifications were made to adapt to Presto including:
Removal of ConnectorExpression. Will be required in a future commit.

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
Changes adapted from trino/PR#11336
Original commit: e69a052e570718ca568114e61644946feab4383e
Author: kasiafi

Modifications were made to adapt to Presto including:
Removal of the StatementAnalyzerFacotry calls.
Add TransactionManager to all StatementAnalyzer constructor calls.

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
Changes adapted from trino/PR#11336
Original commit: 493f639a9daa5e6aaadbb4364587196cb5240fc2
Author: kasiafi

Modifications were made to adapt to Presto including:
Addition of TableFunctionRegistry into FunctionAndTypeManager
Removal of FunctionResolver
Addition of toPath to TableFunctionRegistry

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
Changes adapted from trino/PR#11336
Original commit: 9e8d51ad45f57267f5f7fa6bf8e8c4ec56103dda
Author: kasiafi

Modifications were made to adapt to Presto including:
Removal of Node Location from TrinoException
Added new SemanticErrorCodes
Changed Void context to SqlPlannerContext in RelationPlanner.java
Add newUnqualified to Field class with Presto specification
Add getCanonicalValue to Identifier.java

Co-authored-by: Pratik Joseph Dabre <pdabre12@gmail.com>
Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
Changes adapted from trino/PR#11336
Original commit: d4c73389bbdb6b48c24a0969b259286b05a99ade
Author: kasiafi

Modifications were made to adapt to Presto including:
Change CatalogName to ConnectorId
Modified and removed outputSymbols -> VariableReferenceExpression
Following visitTable example, created and used VariableReferenceExpression List Builder.
TableFunctionNode extends InternalPlanNode instead of PlanNode. Circular dependency and following other Node classes.

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
Changes adapted from trino/PR#11336
Original commit: 565700985baff0c4b29fdb1e3e26139a29318b9e
Author: kasiafi

Modifications were made to adapt to Presto including:
Add applyTableFunction to all implementations of Metadata
Change CatalogName to ConnectorId
Add empty ConnectorTableLayoutHandle to TableHandle in MetadataManger::applyTableFunction

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
Changes adapted from trino/PR#11336
Original commit: ec8b9fd2b2cc9c8bc78c0ca1317dc34fcf2c48c7
Author: kasiafi

Modifications were made to adapt to Presto including:
Change Symbol to VariableReferenceExpression
Add additional arguments to TableScanNode calls
Removal of PlannerContext and replaced with Metadata

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
Preparatory commit for supporting table functions in JDBC connectors

Changes adapted from trino/PR#11336
Original commit: b4d4b5102e878b7e38e13c0440432543a18f913e
Author: kasiafi

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
@mohsaka mohsaka added the from:IBM PR from IBM label Apr 30, 2025
@mohsaka mohsaka requested review from a team and shrinidhijoshi as code owners April 30, 2025 17:52
@mohsaka mohsaka requested a review from jaystarshot April 30, 2025 17:52
@prestodb-ci prestodb-ci requested review from a team, pdabre12 and sh-shamsan and removed request for a team April 30, 2025 17:52
@github-actions
Copy link
Copy Markdown

Codenotify: Notifying subscribers in CODENOTIFY files for diff 7424bf5...09281ac.

Notify File(s)
@aditi-pandit presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4
@elharo presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4
@kaikalur presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4
@rschlussel presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4

@mohsaka mohsaka closed this Apr 30, 2025
@prestodb-ci
Copy link
Copy Markdown
Contributor

Failed to query rebased items linked to #25018:

Error 1054 (42S22): Unknown column 'auto_removed_by' in 'where clause'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:IBM PR from IBM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants