Open lineage plugin modularization #26134
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ebyhr
reviewed
Jul 7, 2025
...n/trino-openlineage/src/main/java/io/trino/plugin/openlineage/OpenLineageClientProvider.java
Outdated
Show resolved
Hide resolved
plugin/trino-openlineage/src/main/java/io/trino/plugin/openlineage/OpenLineageListener.java
Show resolved
Hide resolved
.../trino-openlineage/src/main/java/io/trino/plugin/openlineage/OpenLineageListenerFactory.java
Outdated
Show resolved
Hide resolved
plugin/trino-openlineage/src/main/java/io/trino/plugin/openlineage/OpenLineageTransport.java
Outdated
Show resolved
Hide resolved
...nlineage/src/main/java/io/trino/plugin/openlineage/transport/OpenLineageTransportConfig.java
Outdated
Show resolved
Hide resolved
oskar-szwajkowski
force-pushed
the
osz/open-lineage-plugin-modularization
branch
3 times, most recently
from
49f7e3b to
39fbf6e
Compare
oskar-szwajkowski
force-pushed
the
osz/open-lineage-plugin-modularization
branch
from
39fbf6e to
0c32f1b
Compare
Contributor
Author
|
@ebyhr could you PTAL again? I've updated commits |
oskar-szwajkowski
force-pushed
the
osz/open-lineage-plugin-modularization
branch
from
0c32f1b to
8aa4c31
Compare
ebyhr
reviewed
Jul 11, 2025
.../trino-openlineage/src/test/java/io/trino/plugin/openlineage/BaseTestOpenLineageQueries.java
Show resolved
Hide resolved
.../trino-openlineage/src/test/java/io/trino/plugin/openlineage/BaseTestOpenLineageQueries.java
Show resolved
Hide resolved
.../trino-openlineage/src/test/java/io/trino/plugin/openlineage/BaseTestOpenLineageQueries.java
Outdated
Show resolved
Hide resolved
.../trino-openlineage/src/test/java/io/trino/plugin/openlineage/BaseTestOpenLineageQueries.java
Outdated
Show resolved
Hide resolved
...rc/test/java/io/trino/plugin/openlineage/TestOpenLineageEventListenerMarquezIntegration.java
Outdated
Show resolved
Hide resolved
plugin/trino-openlineage/src/main/java/io/trino/plugin/openlineage/OpenLineageTransport.java
Outdated
Show resolved
Hide resolved
.../trino-openlineage/src/test/java/io/trino/plugin/openlineage/BaseTestOpenLineageQueries.java
Outdated
Show resolved
Hide resolved
oskar-szwajkowski
force-pushed
the
osz/open-lineage-plugin-modularization
branch
from
8aa4c31 to
40fc6b9
Compare
oskar-szwajkowski
force-pushed
the
osz/open-lineage-plugin-modularization
branch
from
40fc6b9 to
68129fa
Compare
ebyhr
approved these changes
Jul 14, 2025
plugin/trino-openlineage/src/main/java/io/trino/plugin/openlineage/OpenLineageListener.java
Outdated
Show resolved
Hide resolved
...n/trino-openlineage/src/main/java/io/trino/plugin/openlineage/OpenLineageListenerModule.java
Outdated
Show resolved
Hide resolved
oskar-szwajkowski
force-pushed
the
osz/open-lineage-plugin-modularization
branch
2 times, most recently
from
c25af12 to
38cb7b2
Compare
Contributor
Author
|
@ebyhr CI is green |
ebyhr
reviewed
Jul 14, 2025
...n/trino-openlineage/src/main/java/io/trino/plugin/openlineage/OpenLineageListenerModule.java
Outdated
Show resolved
Hide resolved
...n/trino-openlineage/src/main/java/io/trino/plugin/openlineage/OpenLineageListenerModule.java
Outdated
Show resolved
Hide resolved
oskar-szwajkowski
force-pushed
the
osz/open-lineage-plugin-modularization
branch
from
38cb7b2 to
8944fea
Compare
Member
|
Could you rebase on master to resolve conflicts? |
Split transport and lineage configuration to prevent passing unnecessary configs if they are not needed
This should allow easier testing of more integrations, using same queries
Add tests for open lineage events shape
oskar-szwajkowski
force-pushed
the
osz/open-lineage-plugin-modularization
branch
from
8944fea to
cc0de9a
Compare
Contributor
Author
sure, done |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR aims to modularize config for OpenLineage plugin, with ability to decouple transport selection from other required configuration.
It also extracts base test, that should allow testing open lineage with different integrations, using the same queries.
New tests for various scenarios has been added.
Release notes
(x) This is not user-visible or is docs only, and no release notes are required.