Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create build pipelines for Java dependencies #940

Open
Tracked by #583
dervoeti opened this issue Nov 19, 2024 · 1 comment
Open
Tracked by #583

Create build pipelines for Java dependencies #940

dervoeti opened this issue Nov 19, 2024 · 1 comment

Comments

@dervoeti
Copy link
Member

Problem:
Let's say we apply a patch to e.g. Hadoop 3.4.0 to fix a vulnerability. We bump a dependency to the latest version, the vulnerability is gone. But all our products that have dependencies on Hadoop Java artifacts will still pull the original Hadoop 3.4.0 components from the default public Maven repository, which does not contain our patched version. We could maybe monkeypatch the product by directly depending on the latest version of the dependency, but then we have to do this across all products, all versions of that product and for each bumped dependency, which likely does not scale.

We could instead contribute the patch upstream, which is nice, since we also get additional validation of the patch by the maintainers and other people can easily profit from the patch as well. But: To actually use the patch in all our products, we'd have to wait for the next release of Hadoop.

Idea:
Build a patched version of Hadoop and publish it to our own Maven repo. Patch downstream products like Hive, Trino etc. to use that version of Hadoop. There might be multiple steps involved, example: A vulnerability originating in Hadoop is present in a Trino image. It's in the Trino Phoenix plugin, so we'd have to build (and patch) that plugin ourselves. For that, we have to build and patch Phoenix ourselves first.

We should still try to contribute patches upstream in the long-term though, because:

  1. We can give something back
  2. Get validation from maintainers
  3. We have to maintain less custom patches
@lfrancke
Copy link
Member

This is related to (or a duplicate of) stackabletech/issues#674

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants