[3.4] Ensure all modules are included in INTEG_TEST testcluster distribution

[cwperks]

Description

This PR ensures that all modules are included when using the INTEG_TEST testcluster distribution. Currently, when using a testcluster with INTEG_TEST distribution, only modules matching transport-* are included and as a result many plugins cannot use this distribution adequately because its missing modules like autotagging-commons.

Related Issues

See discussion on #20229

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Summary by CodeRabbit

• Chores
  • Refactored build distribution to replace transport-specific module handling with an integrated test-focused workflow, streamlining how modules are assembled for test distributions.
• Documentation
  • Changelog updated: noted change ensuring all modules are included in the INTEG_TEST testcluster distribution.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

❌ Gradle check result for aaa1183: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for aaa1183: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for aaa1183: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for aaa1183: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[cwperks]

Closing the PR against 3.4 as this is getting fixed in the following ways in CCR:

• [3.4.0] Fix Gradle cache issue cross-cluster-replication#1614
• [3.4] Use Test instead of RestIntegTestTask for integTestRemote cross-cluster-replication#1613

The PR against main should be merged as a stable long term solution for running integ tests with testclusters for plugins