Failed auth with blocking authentication & a custom authentication failure mapper cause HTTP/2 error #34912

kdubb · 2023-07-21T10:57:41Z

Describe the bug

When the following combination is enabled:

The client request is HTTP/2
quarkus.smallrye-jwt.blocking-authentication is true
Authentication is enabled (e.g. SmallRye JWT)
Authentication throws exception

The request fails with Request has already been read and results in a 500 response instead of the mapped exception response.

Expected behavior

HTTP/2 connections correctly support blocking authentication and authentication exception mappers.

Actual behavior

An exception is thrown with Request has already been read and a 500 response is returned instead of the response returned by the exception mapper.

How to Reproduce?

blocking-auth-mapper-fail-2.16.8.zip
blocking-auth-mapper-fail-3.2.1.zip

Running the included test reproduces the issue in both 2.16 and 3.2 series.

Output of `uname -a` or `ver`

macOS 13.4

Output of `java -version`

Java 17.0.4

GraalVM version (if different from Java)

No response

Quarkus version or git rev

3.2.1 & 2.16.8

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

Gradle 8.1

Additional information

No response

The text was updated successfully, but these errors were encountered:

quarkus-bot · 2023-07-21T11:01:44Z

/cc @FroMage (resteasy-reactive), @Sgitario (resteasy-reactive), @geoand (resteasy-reactive), @pedroigor (bearer-token), @sberyozkin (bearer-token,jwt,security), @stuartwdouglas (resteasy-reactive)

Sgitario · 2023-07-24T09:09:30Z

To give better context to this issue, this is the stacktrace I'm seeing:

Unhandled exception in router
java.lang.IllegalStateException: Request has already been read
        at io.vertx.core.http.impl.Http2ServerRequest.checkEnded(Http2ServerRequest.java:217)
        at io.vertx.core.http.impl.Http2ServerRequest.pause(Http2ServerRequest.java:269)
        at io.quarkus.vertx.http.runtime.ResumingRequestWrapper.pause(ResumingRequestWrapper.java:30)
        at io.vertx.core.http.impl.HttpServerRequestWrapper.pause(HttpServerRequestWrapper.java:67)
        at org.jboss.resteasy.reactive.server.vertx.VertxResteasyReactiveRequestContext.<init>(VertxResteasyReactiveRequestContext.java:83)
        at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.<init>(QuarkusResteasyReactiveRequestContext.java:31)
        at io.quarkus.resteasy.reactive.server.runtime.ResteasyReactiveRecorder$5.createContext(ResteasyReactiveRecorder.java:175)
        at org.jboss.resteasy.reactive.server.handlers.RestInitialHandler.beginProcessing(RestInitialHandler.java:52)
        at io.quarkus.resteasy.reactive.server.runtime.ResteasyReactiveRecorder$7.handle(ResteasyReactiveRecorder.java:240)
        at io.quarkus.resteasy.reactive.server.runtime.ResteasyReactiveRecorder$7.handle(ResteasyReactiveRecorder.java:225)
        at io.vertx.ext.web.impl.RouteState.handleFailure(RouteState.java:1290)
        at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:174)
        at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:141)
        at io.vertx.ext.web.impl.RoutingContextImpl.doFail(RoutingContextImpl.java:548)
        at io.vertx.ext.web.impl.RoutingContextImpl.fail(RoutingContextImpl.java:197)
        at io.vertx.ext.web.impl.RoutingContextImpl.fail(RoutingContextImpl.java:186)
        at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$AbstractAuthenticationHandler$1.proceed(HttpSecurityRecorder.java:395)
        at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$DefaultAuthFailureHandler$1.accept(HttpSecurityRecorder.java:303)
        at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$DefaultAuthFailureHandler$1.accept(HttpSecurityRecorder.java:299)
        ....
        at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$DefaultAuthFailureHandler.accept(HttpSecurityRecorder.java:299)
        at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$DefaultAuthFailureHandler.accept(HttpSecurityRecorder.java:285)
        at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$AbstractAuthenticationHandler$3.onFailure(HttpSecurityRecorder.java:463)

The problem is that after getting a response with auth failures, RESTEasy Reactive tries to customise the response (see this line), and for doing it, it starts a new context (see here) and pauses the already ended request in here which is fine for HTTP/1.1, but not for HTTP/2 (Vert.x throws the above exception).

I'm not very familiar with the above flow, but at first, I don't follow why RESTEasy Reactive creates a new context with the already ended request to customize the response, which I think it's the issue. @michalvavrik can you clarify this?

If the above is done on purpose, then we should somehow reset the HTTP/2 connection in VertxResteasyReactiveRequestContext or RoutingContext.request() (which provides the request).

cc @michalvavrik @geoand @sberyozkin

By the way, I had to modify the test from the reproducer to assert that the expected status code is 401, not 200.

michalvavrik · 2023-07-24T10:00:25Z

The problem is that after getting a response with auth failures, RESTEasy Reactive tries to customise the response (see this line), and for doing it, it starts a new context (see here) and pauses the already ended request in here which is fine for HTTP/1.1, but not for HTTP/2 (Vert.x throws the above exception).

I'm not very familiar with the above flow, but at first, I don't follow why RESTEasy Reactive creates a new context with the already ended request to customize the response, which I think it's the issue. @michalvavrik can you clarify this?

Next context should only be started if RESTEasy Reactive didn't start yet processing. The fact that it is happening after

quarkus/independent-projects/resteasy-reactive/server/vertx/src/main/java/org/jboss/resteasy/reactive/server/vertx/VertxResteasyReactiveRequestContext.java

Line 83 in ab18d1b

request.pause();

signal bug in that auth failure handler should not be applied and exception should be handled in RESTEasy Reactive fail chain (=we don't catch exception then abort chain handlers apply exception mapper IIRC).

@Sgitario ^^ I only base on your analysis. I'll need to debug this myself to tell more, but proper fix should be avoiding default auth failure handler. I'll have a look this week, unless you want to handle it yourself.

michalvavrik · 2023-07-24T10:14:34Z

Basically, my expectation is to remove default auth failure handler from routing context inside same Vert.X route handler that begin processing, which does a trick. And only start new context when Routing context was failed before RR created one.

michalvavrik · 2023-07-24T10:20:20Z

quarkus/independent-projects/resteasy-reactive/server/vertx/src/main/java/org/jboss/resteasy/reactive/server/vertx/VertxResteasyReactiveRequestContext.java

Line 83 in ab18d1b

request.pause();

Ah, I think I understand you now. So the problem is that I create context and then invoke handleException?

public void beginProcessing(Object externalHttpContext, Throwable throwable) {
        ResteasyReactiveRequestContext rq = requestContextFactory.createContext(deployment, externalHttpContext,
                requestContext,
                initialChain, deployment.getAbortHandlerChain());
        rq.handleException(throwable);
        rq.run();
    }

If not, I'll leave any other comment till I have time to look so that I don't waste your time.

Sgitario · 2023-07-24T10:31:42Z

quarkus/independent-projects/resteasy-reactive/server/vertx/src/main/java/org/jboss/resteasy/reactive/server/vertx/VertxResteasyReactiveRequestContext.java

Line 83 in ab18d1b

request.pause();

Ah, I think I understand you now. So the problem is that I create context and then invoke handleException?
public void beginProcessing(Object externalHttpContext, Throwable throwable) {
        ResteasyReactiveRequestContext rq = requestContextFactory.createContext(deployment, externalHttpContext,
                requestContext,
                initialChain, deployment.getAbortHandlerChain());
        rq.handleException(throwable);
        rq.run();
    }
If not, I'll leave any other comment till I have time to look so that I don't waste your time.

Exactly, this is creating a new context with an existing request (which in the case of HTTP/2.0, can't be paused because it's already closed).

Take your time next week to have a look at this flow. I'm not familiar with it, but I'd be rather than happy to help with anything).

michalvavrik · 2023-07-24T10:44:20Z

Thanks, will do.

michalvavrik · 2023-07-24T13:42:00Z

FYI it seems like a general problem when exception is thrown inside this

quarkus/extensions/security/runtime/src/main/java/io/quarkus/security/runtime/IdentityProviderManagerCreator.java

Line 49 in c20d19e

ExecutorRecorder.getCurrent().execute(command);

blocking executor and HTTP2 is used, because I can't see any part of our code that results directly in ending HTTP2 request. When I replaced

quarkus/extensions/smallrye-jwt/runtime/src/main/java/io/quarkus/smallrye/jwt/runtime/auth/MpJwtValidator.java

Line 69 in ab18d1b

return context.runBlocking(() -> createSecurityIdentity(request));

with

final io.vertx.core.Context ctx = Vertx.currentContext();
            return Uni.createFrom()
                    .completionStage(ctx
                            .<SecurityIdentity> executeBlocking(tPromise -> tPromise.complete(createSecurityIdentity(request)))
                            .toCompletionStage());

reproducer was fixed. IMHO we should provide similar executor for IdentityProviderManager on Vert.X HTTP level. I'll have to verify what would such change mean and whether it is viable option. In the meanwhile, @Sgitario I don't see this as RESTEasy Reactive bug. Please feel free to prove me wrong. Thanks

also cc @sberyozkin

Sgitario · 2023-07-24T13:45:55Z

FYI it seems like a general problem when exception is thrown inside this

quarkus/extensions/security/runtime/src/main/java/io/quarkus/security/runtime/IdentityProviderManagerCreator.java

Line 49 in c20d19e

ExecutorRecorder.getCurrent().execute(command);

blocking executor and HTTP2 is used, because I can't see any part of our code that results directly in ending HTTP2 request. When I replaced

quarkus/extensions/smallrye-jwt/runtime/src/main/java/io/quarkus/smallrye/jwt/runtime/auth/MpJwtValidator.java

Line 69 in ab18d1b

return context.runBlocking(() -> createSecurityIdentity(request));

with
final io.vertx.core.Context ctx = Vertx.currentContext();
            return Uni.createFrom()
                    .completionStage(ctx
                            .<SecurityIdentity> executeBlocking(tPromise -> tPromise.complete(createSecurityIdentity(request)))
                            .toCompletionStage());
reproducer was fixed. IMHO we should provide similar executor for IdentityProviderManager on Vert.X HTTP level. I'll have to verify what would such change mean and whether it is viable option. In the meanwhile, @Sgitario I don't see this as RESTEasy Reactive bug. Please feel free to prove me wrong. Thanks

also cc @sberyozkin

I'm sure you're totally right. Removing the resteasy-reactive label. Thanks for checking!

kdubb added the kind/bug Something isn't working label Jul 21, 2023

quarkus-bot bot added the triage/needs-triage label Jul 21, 2023

kdubb added area/security area/rest and removed triage/needs-triage labels Jul 21, 2023

Sgitario removed the area/rest label Jul 24, 2023

michalvavrik mentioned this issue Jul 24, 2023

Run authentication request blocking tasks on Vert.x duplicated context #34963

Merged

sberyozkin closed this as completed in #34963 Jul 27, 2023

quarkus-bot bot added this to the 3.3 - main milestone Jul 27, 2023

michalvavrik mentioned this issue Feb 29, 2024

Optionally run DNS lookup for OIDC server requests on worker thread #39068

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failed auth with blocking authentication & a custom authentication failure mapper cause HTTP/2 error #34912

Failed auth with blocking authentication & a custom authentication failure mapper cause HTTP/2 error #34912

kdubb commented Jul 21, 2023

quarkus-bot bot commented Jul 21, 2023

Sgitario commented Jul 24, 2023

michalvavrik commented Jul 24, 2023 •

edited

Loading

michalvavrik commented Jul 24, 2023 •

edited

Loading

michalvavrik commented Jul 24, 2023 •

edited

Loading

Sgitario commented Jul 24, 2023

michalvavrik commented Jul 24, 2023

michalvavrik commented Jul 24, 2023

Sgitario commented Jul 24, 2023

Failed auth with blocking authentication & a custom authentication failure mapper cause HTTP/2 error #34912

Failed auth with blocking authentication & a custom authentication failure mapper cause HTTP/2 error #34912

Comments

kdubb commented Jul 21, 2023

Describe the bug

Expected behavior

Actual behavior

How to Reproduce?

Output of uname -a or ver

Output of java -version

GraalVM version (if different from Java)

Quarkus version or git rev

Build tool (ie. output of mvnw --version or gradlew --version)

Additional information

quarkus-bot bot commented Jul 21, 2023

Sgitario commented Jul 24, 2023

michalvavrik commented Jul 24, 2023 • edited Loading

michalvavrik commented Jul 24, 2023 • edited Loading

michalvavrik commented Jul 24, 2023 • edited Loading

Sgitario commented Jul 24, 2023

michalvavrik commented Jul 24, 2023

michalvavrik commented Jul 24, 2023

Sgitario commented Jul 24, 2023

Output of `uname -a` or `ver`

Output of `java -version`

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

michalvavrik commented Jul 24, 2023 •

edited

Loading

michalvavrik commented Jul 24, 2023 •

edited

Loading

michalvavrik commented Jul 24, 2023 •

edited

Loading