Skip to content

Add Python language functions#24378

Merged
electrum merged 1 commit intotrinodb:masterfrom
electrum:function
Dec 8, 2024
Merged

Add Python language functions#24378
electrum merged 1 commit intotrinodb:masterfrom
electrum:function

Conversation

@electrum
Copy link
Copy Markdown
Member

@electrum electrum commented Dec 5, 2024

Description

This adds support for functions using the Python language. Example:

WITH FUNCTION xor(a boolean, b boolean)
RETURNS boolean
LANGUAGE PYTHON
WITH (handler = 'bool_xor')
AS $$
import operator
def bool_xor(a, b):
    return operator.xor(a, b)
$$
SELECT xor(true, false), xor(false, true)

The handler property specifies the Python function to execute when the Trino function is called.

See TestPythonFunctions for more examples.

Implementation

Python functions are executed in the same JVM as Trino using Chicory, a native JVM WebAssembly runtime, which is secure and fully sandboxed. The Python environment is limited to 32MB of memory (with the engine creating separate environments for each concurrent function invocation). As with other functions, there is no time or CPU limit, but execution is interruptible, and thus will be aborted if the query is cancelled.

Data Types

The following table shows supported Trino types and their corresponding Python types:

Trino Type Python Type
row tuple
array list
map dict
boolean bool
tinyint int
smallint int
integer int
bigint int
real float
double float
decimal decimal.Decimal
varchar str
varbinary bytes
date datetime.date
time datetime.time
time with time zone datetime.time with datetime.tzinfo
timestamp datetime.datetime
timestamp with time zone datetime.datetime with datetime.tzinfo 1
interval year to month int as the number of months
interval day to second datetime.timedelta
json str
uuid uuid.UUID
ipaddress ipaddress.IPv4Address or ipaddress.IPv6Address

Python datetime objects only support microsecond precision. Trino argument values with greater precision will be rounded when converted to Python values, and Python return values will be rounded if the Trino return type has less than microsecond precision.

Dependencies

Python WASM integration: trinodb/trino-wasm-python#1

Support for language functions: #24105

Release notes

(x) Release notes are required. Please propose a release note for me.

Footnotes

  1. Currently, only fixed offset time zones are supported. Timestamps with political time zones will have the zone converted to the zone's offset for the timestamp's instant.

@cla-bot cla-bot bot added the cla-signed label Dec 5, 2024
@github-actions github-actions bot added the docs label Dec 5, 2024
Copy link
Copy Markdown
Member

@dain dain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comments, but looks good

@electrum electrum merged commit 6b806ae into trinodb:master Dec 8, 2024
@electrum electrum deleted the function branch December 8, 2024 03:02
@github-actions github-actions bot added this to the 468 milestone Dec 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants