Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lock jib execution to avoid OverlappingFileLockException in parallel builds #35308

Merged
merged 1 commit into from
Aug 14, 2023
Merged

Lock jib execution to avoid OverlappingFileLockException in parallel builds #35308

merged 1 commit into from
Aug 14, 2023

Conversation

famod
Copy link
Member

@famod famod commented Aug 10, 2023

It's not pretty (and the locking scope is more coarse than actually needed), but it fixes sproadic OverlappingFileLockExceptions when running a parallel Maven build (e.g. mvn -T2) in a multi-module project that has at least two modules that build a Quarkus app concurrently with jib.

Reproducer can be found here: #34873 (comment)
#34873 also has more info of how I ended up with this "solution".

There is a good chance this will also fix #11903, but we need a gradle user to test it.

@famod famod requested a review from geoand August 10, 2023 12:51
@famod famod mentioned this pull request Aug 10, 2023
Closed
@famod famod marked this pull request as ready for review August 11, 2023 21:55
@famod
Copy link
Member Author

famod commented Aug 11, 2023

So I generated a gradle sample app with jib and it's still working with this change.
The classloaders in Gradle are as follow:

  • QuarkusClassLoader:Augmentation Class Loader: PROD@26027e30
  • VisitableURLClassLoader(worker-loader)
  • FilteringClassLoader(MixInLegacyTypesClassLoader(legacy-mixin-loader))
  • MixInLegacyTypesClassLoader(legacy-mixin-loader)
  • jdk.internal.loader.ClassLoaders$AppClassLoader@5cb0d902

Which means it's loading QuarkusAugmentor from VisitableURLClassLoader and that should be just fine because the next parent FilteringClassLoader is not able to load it.
So while I didn't test the actual gradle parallel build scenario that was described in #11903, I'm pretty confident that this change should fix that issue. TBH, I don't think anyone from that issue will report back/try this PR.
Worst thing that can happen is that this change only fixes the Maven case but does nothing for Gradle.

I'm also proposing to backport this to 3.3.

@famod famod requested a review from aloubyansky August 11, 2023 22:07
@quarkus-bot
Copy link

quarkus-bot bot commented Aug 11, 2023

✔️ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

@aloubyansky aloubyansky merged commit 42cd3ea into quarkusio:main Aug 14, 2023
20 checks passed
@quarkus-bot quarkus-bot bot added this to the 3.4 - main milestone Aug 14, 2023
@famod famod deleted the jib-parallel branch August 15, 2023 08:23
@gsmet gsmet modified the milestones: 3.4 - main, 3.3.0 Aug 15, 2023
@gsmet gsmet modified the milestones: 3.3.0, 3.2.5.Final Aug 24, 2023
benkard pushed a commit to benkard/mulkcms2 that referenced this pull request Aug 29, 2023
Update all non-major dependencies (mulk/mulkcms2!32)
2f015c7 
This MR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [flow-bin](https://github.com/flowtype/flow-bin) ([changelog](https://github.com/facebook/flow/blob/master/Changelog.md)) | devDependencies | minor | [`^0.214.0` -> `^0.215.0`](https://renovatebot.com/diffs/npm/flow-bin/0.214.0/0.215.1) |
| [org.liquibase:liquibase-maven-plugin](http://www.liquibase.org/liquibase-maven-plugin) ([source](https://github.com/liquibase/liquibase)) | build | patch | `4.23.0` -> `4.23.1` |
| [io.quarkus:quarkus-maven-plugin](https://github.com/quarkusio/quarkus) | build | minor | `3.2.3.Final` -> `3.3.0` |
| [io.quarkus:quarkus-universe-bom](https://github.com/quarkusio/quarkus-platform) | import | minor | `3.2.3.Final` -> `3.3.0` |
| [org.apache.maven.plugins:maven-enforcer-plugin](https://maven.apache.org/enforcer/) | build | minor | `3.3.0` -> `3.4.0` |

---

### Release Notes

<details>
<summary>flowtype/flow-bin</summary>

### [`v0.215.1`](flow/flow-bin@a92ce80...cbb038f)

[Compare Source](flow/flow-bin@a92ce80...cbb038f)

### [`v0.215.0`](flow/flow-bin@ca11e28...a92ce80)

[Compare Source](flow/flow-bin@ca11e28...a92ce80)

</details>

<details>
<summary>liquibase/liquibase</summary>

### [`v4.23.1`](https://github.com/liquibase/liquibase/blob/HEAD/changelog.txt#Liquibase-4231-is-a-patch-release)

[Compare Source](liquibase/liquibase@v4.23.0...v4.23.1)

</details>

<details>
<summary>quarkusio/quarkus</summary>

### [`v3.3.0`](https://github.com/quarkusio/quarkus/releases/tag/3.3.0)

[Compare Source](quarkusio/quarkus@3.2.4.Final...3.3.0)

##### Complete changelog

-   [#&#8203;35350](quarkusio/quarkus#35350) - Fix package type system property clearing
-   [#&#8203;35348](quarkusio/quarkus#35348) - quarkus-maven-plugin runs native building even if the profile is commented out
-   [#&#8203;35343](quarkusio/quarkus#35343) - ArC: fix StackOverflowError in AutoAddScopeBuildItem
-   [#&#8203;35319](quarkusio/quarkus#35319) - Register arrays of Hibernate ORM's JDBC basic types for reflection
-   [#&#8203;35315](quarkusio/quarkus#35315) - Fix Datasource timing issues with Liquibase / Flyway and OpenTelemetry
-   [#&#8203;35314](quarkusio/quarkus#35314) - Regression in 3.3.0.CR1: Synthetic bean instance for io.opentelemetry.api.OpenTelemetry not initialized yet
-   [#&#8203;35312](quarkusio/quarkus#35312) - Updates Infinispan to 14.0.13.Final
-   [#&#8203;35308](quarkusio/quarkus#35308) - Lock jib execution to avoid OverlappingFileLockException in parallel builds
-   [#&#8203;35305](quarkusio/quarkus#35305) - Fix the titles of the tables in RESTEasy Reactive doc
-   [#&#8203;35302](quarkusio/quarkus#35302) - Docs: Mention wilcard support in resteasy reactive XML serialisation exclude classes configuration
-   [#&#8203;35301](quarkusio/quarkus#35301) - Fix potential NPE in quarkus-csrf-reactive when no MediaType is found
-   [#&#8203;35299](quarkusio/quarkus#35299) - Output build graph using `quarkus.builder.graph-output` property
-   [#&#8203;35285](quarkusio/quarkus#35285) - NullPointerException during http post request when quarkus-csrf-reactive extension is added to a project
-   [#&#8203;35283](quarkusio/quarkus#35283) - Upgrade proto-google-common-protos to 2.23.0
-   [#&#8203;35282](quarkusio/quarkus#35282) - Avoid keeping references to BytecodeRecorderImpl
-   [#&#8203;35276](quarkusio/quarkus#35276) - Reinstate DevModeTestUtil to avoid breaking other projects that depend on it
-   [#&#8203;35273](quarkusio/quarkus#35273) - Fix small typo in comment
-   [#&#8203;35263](quarkusio/quarkus#35263) - Stop the recovery service while ArC is still around
-   [#&#8203;35245](quarkusio/quarkus#35245) - Add missing info to init Jobs
-   [#&#8203;35244](quarkusio/quarkus#35244) - Init Jobs are missing ServiceAccount and Image Pull Secrets
-   [#&#8203;35240](quarkusio/quarkus#35240) - Update SmallRye Health to 4.0.4
-   [#&#8203;34071](quarkusio/quarkus#34071) - 3.1.1 Final - java.lang.IllegalArgumentException: Class java.util.UUID\[] is instantiated reflectively but was never registered
-   [#&#8203;32800](quarkusio/quarkus#32800) - Duplicated checks in health check response
-   [#&#8203;11903](quarkusio/quarkus#11903) - Gradle multimodule project + quarkus-container-image-jib: OverlappingFileLockException

### [`v3.2.4.Final`](https://github.com/quarkusio/quarkus/releases/tag/3.2.4.Final)

[Compare Source](quarkusio/quarkus@3.2.3.Final...3.2.4.Final)

##### Complete changelog

-   [#&#8203;35300](quarkusio/quarkus#35300) - Fix `jandex-gradle-plugin-version` in CDI Reference
-   [#&#8203;35296](quarkusio/quarkus#35296) - Upgrade H2 to 2.2.220
-   [#&#8203;35258](quarkusio/quarkus#35258) - CDI Reference 1.1 has incomplete information for gradle
-   [#&#8203;35255](quarkusio/quarkus#35255) - Quartz: QuarkusMSSQLDelegate should extends MSSQLDelegate
-   [#&#8203;35211](quarkusio/quarkus#35211) - Document Maven config options that may be relevant when running tests
-   [#&#8203;35203](quarkusio/quarkus#35203) - Pass Maven user settings when initializing artifact resolver
-   [#&#8203;35193](quarkusio/quarkus#35193) - OpenTelemetry service name should have higher priority than app name when no resource attributes are set
-   [#&#8203;35189](quarkusio/quarkus#35189) - Quarkus CLI fixes
-   [#&#8203;35188](quarkusio/quarkus#35188) - SmallRyeGraphQLOverWebSocketHandler: use order value > Integer.MIN_VALUE
-   [#&#8203;35181](quarkusio/quarkus#35181) - REST Data with Panache should not produce links when hal is disabled
-   [#&#8203;35174](quarkusio/quarkus#35174) - Ensure the narayana-jta extension fully shuts down the recovery manager
-   [#&#8203;35172](quarkusio/quarkus#35172) - Kafka Streams: restore the feature name at Quarkus startup
-   [#&#8203;35171](quarkusio/quarkus#35171) - kafka-streams: feature not listed on startup
-   [#&#8203;35165](quarkusio/quarkus#35165) - Propagate all user methods in REST Data with Panache
-   [#&#8203;35160](quarkusio/quarkus#35160) - Properly use internal links to point to other guides
-   [#&#8203;35140](quarkusio/quarkus#35140) - ArC: fix deadlock when calling guarded methods on the same thread
-   [#&#8203;35136](quarkusio/quarkus#35136) - Deadlock while calling write-locked method from read-locked method
-   [#&#8203;34908](quarkusio/quarkus#34908) - `@RouteFilter` stopped working with WebSocket requests Quarkus 3.2.0.Final
-   [#&#8203;34875](quarkusio/quarkus#34875) - Quarkus build does not work since 3.2.0 with teamcity/plexus launcher
-   [#&#8203;34713](quarkusio/quarkus#34713) - Option to track build configuration for changes between builds
-   [#&#8203;34576](quarkusio/quarkus#34576) - Live reload stopped working on 3.2 when using XA transactions
-   [#&#8203;34514](quarkusio/quarkus#34514) - Support `@WithUnnamedKey` in documentation
-   [#&#8203;34065](quarkusio/quarkus#34065) - Add support for project Java version update based on extensions
-   [#&#8203;33317](quarkusio/quarkus#33317) - OpenTelemetry SDK autoconfiguration ignores OTEL service name in favor of Quarkus app name
-   [#&#8203;15461](quarkusio/quarkus#15461) - Quarkus tests fails mTLS authentication against internal Maven repository

</details>

<details>
<summary>quarkusio/quarkus-platform</summary>

### [`v3.3.0`](quarkusio/quarkus-platform@3.2.4.Final...3.3.0)

[Compare Source](quarkusio/quarkus-platform@3.2.4.Final...3.3.0)

### [`v3.2.4.Final`](quarkusio/quarkus-platform@3.2.3.Final...3.2.4.Final)

[Compare Source](quarkusio/quarkus-platform@3.2.3.Final...3.2.4.Final)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox.

👻 **Immortal**: This MR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [x] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4yNC4wIiwidXBkYXRlZEluVmVyIjoiMzQuMjQuMCJ9-->
@famod famod mentioned this pull request Sep 28, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@geoand geoand geoand approved these changes

@aloubyansky aloubyansky aloubyansky approved these changes

Assignees
No one assigned
Labels
area/container-image kind/bugfix
Projects
None yet
Milestone
3.2.5.Final
Development

Successfully merging this pull request may close these issues.

Gradle multimodule project + quarkus-container-image-jib: OverlappingFileLockException
4 participants
@famod @aloubyansky @geoand @gsmet