ResponseStatusException no longer returning response body in 2.6.1 using spring security

[simon-mitchell]

we found that after upgrading to spring boot 2.6.1 that the response body from the ResponseStatusException is no longer being populated even for authenticated users

It looks like the spring boot forwards onto the /error page but the BearerTokenAuthenticationFIlter which extends the OncePerRequestFilter doesn't add the necessary authentication to the spring security context when in error state.
This means that we then hit #26356 and the body is empty.

an example project is https://github.com/ministryofjustice/hmpps-spring-boot-2.6-bug

if you run ./gradlew test then it will fail

the branch https://github.com/ministryofjustice/hmpps-spring-boot-2.6-bug/tree/previous-working-version shows it working in the previous version of spring boot 2.5.6 alternately allowlisting /error fixes it too ( but we don't want to allowlist /error )

[wilkinsona]

Thanks for the sample.

The behaviour you have observed is intended with Spring Boot 2.6. Your security configuration requires authentication to access everything but /favicon.ico, /csrf, /health/**, /info, /webjars/**, and /v2/api-docs. This means that you have secured Spring Boot's error page at /error and, thanks to the changes in #26356, an unauthenticated user no longer receives a full error response.

If you want any user, authenticated or not, to receive Spring Boot's default error response, you should permit access to /error. Applying this diff to your application fixes the failing test:

diff --git a/src/main/kotlin/uk/gov/justice/digital/hmpps/token/config/ResourceServerConfiguration.kt b/src/main/kotlin/uk/gov/justice/digital/hmpps/token/config/ResourceServerConfiguration.kt
index aa77ab2..6a4d0a6 100644
--- a/src/main/kotlin/uk/gov/justice/digital/hmpps/token/config/ResourceServerConfiguration.kt
+++ b/src/main/kotlin/uk/gov/justice/digital/hmpps/token/config/ResourceServerConfiguration.kt
@@ -25,7 +25,7 @@ open class ResourceServerConfiguration : WebSecurityConfigurerAdapter() {
       .authorizeRequests { auth ->
         auth.antMatchers(
           "/favicon.ico", "/csrf", "/health/**", "/info",
-          "/webjars/**", "/v2/api-docs"
+          "/webjars/**", "/v2/api-docs", "/error"
         )
           .permitAll().anyRequest().authenticated()
       }

[petergphillips]

I think the important behavioural change here in 2.6 is that authenticated users no longer receive a full error response if the application relies on the default error handler. It would be good to document this change and provide the recommended solution.

It does feel a bit strange though that due to Error page is accessible when no credentials are provided we have now ended up with Error page is not accessible when credentials are provided and the recommended solution to that problem is to allow the error page to be accessible to everyone!

It does sound therefore that the default error handler should not be used. Instead I guess the application could catch the exception and return a ResponseEntity directly or alternatively throw a custom exception and then implement a @ExceptionHandler that returns a ResponseEntity.

[wilkinsona]

I think the important behavioural change here in 2.6 is that authenticated users no longer receive a full error response if the application relies on the default error handler.

That shouldn't be the case.

the recommended solution to that problem is to allow the error page top be accessible to everyone

That is not the recommendation. If a user is authenticated, they should received a body in an error response without permitting all to access the error page. If that's not happening, then we need to investigate further. It could be that there's a faulty filter involved (as I believe was the case in the second issue to which you linked above), you're hitting a limitation of Spring Security, or something else.

[simon-mitchell]

The user in the sample project above is fully authenticated but they are not receiving a body in the error response.

It looks like the spring boot forwards onto the /error page but the BearerTokenAuthenticationFIlter which extends the OncePerRequestFilter doesn't add the necessary authentication to the spring security context when in error state.

Should I therefore raise a bug against the BearerTokenAuthenticationFIlter? (I presume other filters will be affected as well)

[wilkinsona]

Apologies, @SimonMitchellMOJ, I had misunderstood the problem that the sample was demonstrating.

The problem is due to Spring Security's SecurityContextPersistenceFilter behaving differently depending on the authentication method that is being used. When using basic authentication, the authentication from the initial request is reinstated when the request fails and is forwarded to Boot's error page:

2021-12-10 10:11:08.830 DEBUG 5568 --- [o-auto-1-exec-1] o.s.security.web.FilterChainProxy        : Securing GET /fail
2021-12-10 10:11:08.835 DEBUG 5568 --- [o-auto-1-exec-1] s.s.w.c.SecurityContextPersistenceFilter : Set SecurityContextHolder to empty SecurityContext
2021-12-10 10:11:09.002 DEBUG 5568 --- [o-auto-1-exec-1] o.s.s.a.dao.DaoAuthenticationProvider    : Authenticated user
2021-12-10 10:11:09.003 DEBUG 5568 --- [o-auto-1-exec-1] o.s.s.w.a.www.BasicAuthenticationFilter  : Set SecurityContextHolder to UsernamePasswordAuthenticationToken [Principal=org.springframework.security.core.userdetails.User [Username=username, Password=[PROTECTED], Enabled=true, AccountNonExpired=true, credentialsNonExpired=true, AccountNonLocked=true, Granted Authorities=[]], Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=127.0.0.1, SessionId=null], Granted Authorities=[]]
2021-12-10 10:11:09.023 DEBUG 5568 --- [o-auto-1-exec-1] w.c.HttpSessionSecurityContextRepository : Created HttpSession as SecurityContext is non-default
2021-12-10 10:11:09.023 DEBUG 5568 --- [o-auto-1-exec-1] w.c.HttpSessionSecurityContextRepository : Stored SecurityContextImpl [Authentication=UsernamePasswordAuthenticationToken [Principal=org.springframework.security.core.userdetails.User [Username=username, Password=[PROTECTED], Enabled=true, AccountNonExpired=true, credentialsNonExpired=true, AccountNonLocked=true, Granted Authorities=[]], Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=127.0.0.1, SessionId=null], Granted Authorities=[]]] to HttpSession [org.apache.catalina.session.StandardSessionFacade@651af14a]
2021-12-10 10:11:09.028 DEBUG 5568 --- [o-auto-1-exec-1] o.s.s.w.a.i.FilterSecurityInterceptor    : Authorized filter invocation [GET /fail] with attributes [fullyAuthenticated]
2021-12-10 10:11:09.029 DEBUG 5568 --- [o-auto-1-exec-1] o.s.security.web.FilterChainProxy        : Secured GET /fail
2021-12-10 10:11:09.048 DEBUG 5568 --- [o-auto-1-exec-1] w.c.HttpSessionSecurityContextRepository : Stored SecurityContextImpl [Authentication=UsernamePasswordAuthenticationToken [Principal=org.springframework.security.core.userdetails.User [Username=username, Password=[PROTECTED], Enabled=true, AccountNonExpired=true, credentialsNonExpired=true, AccountNonLocked=true, Granted Authorities=[]], Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=127.0.0.1, SessionId=null], Granted Authorities=[]]] to HttpSession [org.apache.catalina.session.StandardSessionFacade@651af14a]
2021-12-10 10:11:09.048 DEBUG 5568 --- [o-auto-1-exec-1] s.s.w.c.SecurityContextPersistenceFilter : Cleared SecurityContextHolder to complete request
2021-12-10 10:11:09.059 DEBUG 5568 --- [o-auto-1-exec-1] o.s.security.web.FilterChainProxy        : Securing GET /error
2021-12-10 10:11:09.059 DEBUG 5568 --- [o-auto-1-exec-1] w.c.HttpSessionSecurityContextRepository : Retrieved SecurityContextImpl [Authentication=UsernamePasswordAuthenticationToken [Principal=org.springframework.security.core.userdetails.User [Username=username, Password=[PROTECTED], Enabled=true, AccountNonExpired=true, credentialsNonExpired=true, AccountNonLocked=true, Granted Authorities=[]], Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=127.0.0.1, SessionId=null], Granted Authorities=[]]]
2021-12-10 10:11:09.059 DEBUG 5568 --- [o-auto-1-exec-1] s.s.w.c.SecurityContextPersistenceFilter : Set SecurityContextHolder to SecurityContextImpl [Authentication=UsernamePasswordAuthenticationToken [Principal=org.springframework.security.core.userdetails.User [Username=username, Password=[PROTECTED], Enabled=true, AccountNonExpired=true, credentialsNonExpired=true, AccountNonLocked=true, Granted Authorities=[]], Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=127.0.0.1, SessionId=null], Granted Authorities=[]]]
2021-12-10 10:11:09.059 DEBUG 5568 --- [o-auto-1-exec-1] o.s.security.web.FilterChainProxy        : Secured GET /error
2021-12-10 10:11:09.120 DEBUG 5568 --- [o-auto-1-exec-1] s.s.w.c.SecurityContextPersistenceFilter : Cleared SecurityContextHolder to complete request

By contrast, when using a bearer token, the forward to /error is done with an anonymous authentication rather than the AuthAwareAuthenticationToken that was initially established:

2021-12-10 10:09:46.611 DEBUG 5524 --- [o-auto-1-exec-1] o.s.security.web.FilterChainProxy        : Securing POST /token/verify
2021-12-10 10:09:46.615 DEBUG 5524 --- [o-auto-1-exec-1] s.s.w.c.SecurityContextPersistenceFilter : Set SecurityContextHolder to empty SecurityContext
2021-12-10 10:09:46.645 DEBUG 5524 --- [o-auto-1-exec-1] o.s.s.o.s.r.a.JwtAuthenticationProvider  : Authenticated token
2021-12-10 10:09:46.646 DEBUG 5524 --- [o-auto-1-exec-1] .o.s.r.w.BearerTokenAuthenticationFilter : Set SecurityContextHolder to AuthAwareAuthenticationToken [Principal=token-verification-api-client, Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=127.0.0.1, SessionId=null], Granted Authorities=[SCOPE_read, SCOPE_write]]
2021-12-10 10:09:46.651 DEBUG 5524 --- [o-auto-1-exec-1] o.s.s.w.a.i.FilterSecurityInterceptor    : Authorized filter invocation [POST /token/verify] with attributes [authenticated]
2021-12-10 10:09:46.652 DEBUG 5524 --- [o-auto-1-exec-1] o.s.security.web.FilterChainProxy        : Secured POST /token/verify
2021-12-10 10:09:46.707 DEBUG 5524 --- [o-auto-1-exec-1] s.s.w.c.SecurityContextPersistenceFilter : Cleared SecurityContextHolder to complete request
2021-12-10 10:09:46.711 DEBUG 5524 --- [o-auto-1-exec-1] o.s.security.web.FilterChainProxy        : Securing POST /error
2021-12-10 10:09:46.711 DEBUG 5524 --- [o-auto-1-exec-1] s.s.w.c.SecurityContextPersistenceFilter : Set SecurityContextHolder to empty SecurityContext
2021-12-10 10:09:46.712 DEBUG 5524 --- [o-auto-1-exec-1] o.s.s.w.a.AnonymousAuthenticationFilter  : Set SecurityContextHolder to anonymous SecurityContext
2021-12-10 10:09:46.712 DEBUG 5524 --- [o-auto-1-exec-1] o.s.security.web.FilterChainProxy        : Secured POST /error

The newly introduced ErrorPageSecurityFilter is built on the assumption that Spring Security will behave consistently in terms of the authentication that's available when a request is forwarded. The above shows that the assumption doesn't hold true when using a bearer token. We'll need to discuss this with the Security team to figure out the best course of action.

[cmdjulian]

I'm seeing the same problem in our applications. We wrote our own custom Authentication. We're also not seeing response bodys on our requests anymore. For 2.5.7 it was working fine.
It boils down to the ErrorController.error(HttpServletRequest request) not getting called in 2.6.1 from what I found out.
I tried the current snapshot of 2.6.2 with the same outcome, no response body. I tried adding /error to the list of permitted paths but this also didn't bring back the response body.
Its relay unfortunate because this is an update blocker for us. Especially regarding the new log4j cve it would be really nice to be able to upgrade as soon as 2.6.2 is out (of course we temporally fixed it by forcing log4j to be version 2.15.0 in our gradle file)