[Enhancement] Use UUIDv7 for QueryId

[dolfinus]

QueryId is currently generated as {timestamp:yyyyMMdd_HHmmss}_{index:05}_{coordinator_id:05}. Could it be changed to match RFC 9562 UUID versions 6-8?

Benefits are:

• UUIDv7 format implementations already uses timestamp + counter + random part for generation, so the value ordering and uniqueness should be just the same. More than that, UUIDv7 timestamp precision is milliseconds compairing to whole seconds of QueryId.
• UUID is 128 bit compairing to 216 bit of string in UTF-8 encoding.
• Better support by third-party systems. Imagine storing audit logs in some database, and user who wants to filter data by queryId field. UUID data type is quite common in modern DBMS, and index takes less space than same index for a string field.

[wendigo]

@dolfinus this is backward incompatible change so we are not considering this that at the moment. Do you have any concrete use case in mind?

[dolfinus]

I'm working on OpenLineage compatible server called Data.Rentgen. It receives OpenLineage events from different sources (Spark, Airflow, and so on), and stores them in database (Postgres). I'm currently investigating Trino OpenLineage integration to find and fix several spots which prevent integration from being used in my compary.

One of main features of this server is ability to handle a large stream of events, like several millions per day. This is implemented by storing all the received data in partitioned tables:

CREATE TABLE run (
  created_at timestamptz,
  id uuid,
  external_id varchar,
  ...
) PARTITION BY RANGE (created_at);

CREATE TABLE run_y2025m04 PARTITION OF run
    FOR VALUES FROM ('2025-04-01') TO ('2025-05-01');

So all events received within a month are stored in the same partition, and can be quickly fetched, filtered, or dropped with minimal effort.

This requires having information about query creation time in every OpenLineage event. For Spark, Airflow, Flink integrations this is done by using UUIDv7, so created_at field value is actually internal timestamp of UUID, the time it was generated. But for Trino, runId is generated as UUIDv5(queryId), so there is no timestamp embedded into the id.

As queryId already contains timestamp and some counter/random part, it has virtually the same information as UUIDv7 value would. So I thought that QueryId could be generated as UUIDv7, but still passed within a source code as a plain string. But if someone relied on the value structure, and used queryId not as opaque string, it could be a breaking change.

If this is not something that could be done, I have another solution, just for OpenLineage integration. Both QueryCreatedEvent and QueryCompletedEvent have getCreateTime() method, so I can change OpenLineage runId from UUIDv5(queryId) to UUIDv7(createTime, queryId). At least this allows to store run events in the same way the other OpenLineage events are stored.

But I still think that using UUIDv7 for queryId/transactionId could be beneficial not just for my case and not for OpenLineage integration alone, but for other EventListener implementations as well. Storing events in any external database (audit logs, other integrations) with partitioning by event creation time may increase the speed of accessing data.
Even if partitioning cannot be used in some database or in a specific case, UUIDv7 are still better solution for storing than UUIDv4 - all values are ordered by creation time, and it's much easier to filter by specific queryId in array of sorted values, than in randomly distributed ones. For example, by using binary search or searching by value prefix in B-Tree indexes. See OpenLineage/OpenLineage#2541 (comment)

[wendigo]

I know benefits of UUIDv7 but the current QueryId is human readable and lots of systems and integrations assume its' form

[dolfinus]

If this is not something that could be done, I have another solution, just for OpenLineage integration. Both QueryCreatedEvent and QueryCompletedEvent have getCreateTime() method, so I can change OpenLineage runId from UUIDv5(queryId) to UUIDv7(createTime, queryId). At least this allows to store run events in the same way the other OpenLineage events are stored.

Used this approach instead, changes are minimal. #25683

[dolfinus]

Probably related: #25595