GH-43469: [Java] Change the default CompressionCodec.Factory to leverage compression support transparently

[ccciudatu]

Rationale for this change

Add compression support to Flight RPC and others by just including the arrow-compression jar in the module path (or classpath).

What changes are included in this PR?

Change the default compression factory to the new CompressionCodec.Factory.INSTANCE, a ServiceLoader-backed singleton that delegates to the best suited available implementation in the module/class path for each codec type.

Are these changes tested?

yes

Are there any user-facing changes?

No.

• GitHub Issue: Change the default CompressionCodec.Factory to leverage compression support transparently #43469

[github-actions]

⚠️ GitHub issue #43469 has been automatically assigned in GitHub to PR creator.

[vibhatha]

cc @lidavidm needs approval for the CIs

[lidavidm]

CC @danepitkin @vibhatha as well; I think we will have to update various documentation/cookbook recipes since now you don't need explicit configuration

[danepitkin]

CC @danepitkin @vibhatha as well; I think we will have to update various documentation/cookbook recipes since now you don't need explicit configuration

After grepping around, there is no documentation/cookbook that currently uses CompressionCodec. I agree, it would be good to add it.

[danepitkin]

LGTM!

[danepitkin]

Added Cookbook issue here apache/arrow-cookbook#354

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 7e50097.

There were 6 benchmark results indicating a performance regression:

• Commit Run on test-mac-arm at 2024-07-31 00:32:31Z
  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-18, scale_factor=1
  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
• and 4 more (see the report linked below)

The full Conbench report has more details. It also includes information about 27 possible false positives for unstable benchmarks that are known to sometimes produce them.

[vibhatha]

@danepitkin there are performance regressions 🤔

[danepitkin]

Good catch. I took a look and the regressions are all in Arrow R. This looks unrelated to this PR so we can ignore.

[vibhatha]

Thanks for checking it out @danepitkin