Build fails with static OpenAPI file in META-INF folder #43940
Comments
quarkus-bot
bot
added
area/openapi
area/smallrye
env/windows
|
/cc @EricWittmann (openapi), @MikeEdgar (openapi), @phillip-kruger (openapi) |
|
We are going to need a reproducer please. |
|
Okay I have created a reproducer and attached it here. The build will fail with the reported error unless you remove the static openapi.yaml file. Also we would like to be able to load the static openapi.yaml from a different directory not in the classpath even, but that does not work for us. Can you explain the limits of the additional-docs-directory property? E.g. where does that property assume the root path is? |
|
@dhoffer are you having the problem with a simple |
|
I'm building from CMD, no IDE open to project. Here is the results of mvn -X clean package I am sort of assuming this is a Windows issue. |
|
What is the status of this? This is a blocking issue for us regarding upgrading to Quarkus 3 and Microprofile OpenAPI. All developers build test with Windows only our CI system is Linux so we have to have good Windows support. |
|
I don't have a Windows machine to try this one. Does this same thing happen with the latest Quarkus release, rather than the SNAPSHOT? |
|
I'm shooting in the dark as I don't have a Windows machine either but from what I can see the OpenAPI extension code is not very careful about closing the opened A good example of this issue is the following code: It's cute but from what I can see it's opening If I'm actually correct, I think it would be worth auditing this area a bit. And by area, I mean anything opening an Windows is a lot more sensitive to file locking but in this case it might actually report an actual issue in our code. @MikeEdgar could you make sure I'm not hallucinating? |
Yeah, I'll tighten this up. |
|
I would have expected them to be closed via the |
|
Yeah, that's why I asked you. It sure seems like these particular ones should be fine. It might be a false alarm but I think something somewhere keeps a lock. I'll have another look if I can find something. |
|
I had another look at the report and second question: are you closing the |
|
Yes, that is a few lines up [1] from the additional/custom static file. |
|
It couldn't possibly be from the |
|
This shouldn't affect things outside of dev mode but you never know. Let me try another strategy. |
|
@dhoffer , in case it was lost can you confirm if this same thing happens with the latest Quarkus release, specifically 3.15.1, rather than the SNAPSHOT? There have been changes in OpenAPI since then and that would help rule out or confirm if this is a regression. |
|
I had a look at a heap dump just before the uberjar was built and couldn't find anything obvious e.g. no suspicious So really not a clue what's going on. |
|
Just to mention it, seems odd that the exception mentions that jars, as if one of them is open in a file explorer or something (?). I would think a lock on the openapi.yaml file would be reported as such. This makes it seem as if the problem is entirely with the jar or the jar.original. |
|
@gastaldi I think you have a Windows box around? Could you check if you can reproduce it with the reproducer? Be careful, it refers to |
|
Yeah so it could also be a file actually loaded from the jar. But I don't think we load anything from the produced jar so this looks extremely odd. Let's see if it can be reproduced independently. |
|
@dhoffer , one more thing - you've mentioned elsewhere that you're using filters, etc. Can you confirm that none of your OASFilter implementations are loading |
|
I tried the reproducer in my Windows machine and I see the error described in #43940 (comment) |
|
@gastaldi @gsmet @MikeEdgar Yes we do use OASFilter but we don't touch the META-INF/openapi.json file, but just to be certain I have removed OASFilter usage in my reproducer. This is what I see building with various Quarkus versions. 999-SNAPSHOT Fails |
|
I reproduced it in github using the reproducer project, so I'll try to dig in that way. |
|
There seems to be something very odd occurring with an InputStream obtained by the classloader. I've disabled the entire Smallrye OpenAPI builder method that handles static files [1] and added a simple lookup of an existing static file, opening the stream in a try-with-resources, and the locked file still occurs. Note, that removing the block of code below from the linked method [1] results in a passing/green build. I.e., any static file passed to the OpenAPI builder is ignored. This must be something lower-level than the OpenAPI processor. What do you think @gsmet @gastaldi ? I have the build running on a GH Windows image in a repo with the reproducer project [2]. The CI there builds a branch of smallrye-open-api in my fork and overrides the version managed by Quarkus to be the local snapshot. URL metaInfYaml = Thread.currentThread().getContextClassLoader().getResource("META-INF/openapi.yaml");
if (metaInfYaml != null) {
try (InputStream stream = metaInfYaml.openStream()) {
} catch (IOException e) {
String msg = "IOException reading static file: " + metaInfYaml;
throw new OpenApiRuntimeException(msg, e);
}
}[1] https://github.com/MikeEdgar/smallrye-open-api/blob/8557535dc2a39b88cb694d80c2e6da71ad55bbb9/core/src/main/java/io/smallrye/openapi/api/SmallRyeOpenAPI.java#L560-L636 |
|
@dmlloyd any idea? ^ |
|
I'd guess it's another problem in the bowels of the |
|
Perhaps try with |
|
I don't have a Windows machine to test but here is what I would try first main...aloubyansky:quarkus:build-file-move That will close the application (along with its classloaders) before moving the file. @dhoffer would you be able to give it a try? |
|
@aloubyansky I gave your branch a try and I still get the same error: |
|
Thanks @gastaldi |
That makes sense, and it will probably require a fix in smallrye-open-api then. I'll look at it from that perspective. |
|
@MikeEdgar smallrye/smallrye-open-api#2049 may help. Could someone help test it? |
|
@aloubyansky I confirmed that avoiding loading resources directly via URL fixes the issue. In addition to the changes in smallrye, there will need to be a fix here in Quarkus also. I'll plan to make it for |
|
Great, thanks @MikeEdgar |
|
@MikeEdgar just create your PR for main, and we can discuss then what to do for 3.15. |
|
Awesome, btw, thanks everyone! |
Sounds good. I have #44055 open currently to bump to smallrye-open-api 4.0.0. Would you prefer I modify that to bump to 4.0.1 with the fix or should we have two separate PRs? |
This is a good point; additionally we should probably be returning only URLs with custom |
|
@MikeEdgar I merged the previous PR, I let you open a new one as we want a clear PR with the fix. As for backporting, 3.16 is using 3.13.0 and 3.15 is using 3.10.0. I don't think we will upgrade in 3.15 to a new minor so ideally, we would need a 3.13.1 and 3.10.1 with the fixes. Unsure how practical it will be though. If not easy, we can probably wait a bit as there is a workaround. |
|
Ah, I forgot that 3.15 is not affected by this. The regression came in smallrye-open-api 3.11 and the associated bump in Quarkus, so the question is whether we backport for 3.16 then. |
Describe the bug
When we put a static OpenAPI file in META-INF folder (the default location), the build fails with this error:
C:\projects\acctmgt-back\target\acctmgmt-back.jar -> C:\projects\acctmgt-back\target\acctmgmt-back.jar.original: The process cannot access the file because it is being used by another process.
If we remove the static OpenAPI file then the build is successful.
So apparently Quarkus has a file lock on this file.
We are using 999-SNAPSHOT to pickup some other recent changes.
Note we are building on Windows and generating an uber-jar type.
Expected behavior
Build should not fail
Actual behavior
Build fails
How to Reproduce?
No response
Output of
uname -aorverNo response
Output of
java -versionjava 21.0.4 2024-07-16 LTS
Quarkus version or git rev
No response
Build tool (ie. output of
mvnw --versionorgradlew --version)Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Additional information
No response
The text was updated successfully, but these errors were encountered: