Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #3806 async sendError #3850

Closed
wants to merge 40 commits into from
Closed

Conversation

gregw
Copy link
Contributor

@gregw gregw commented Jul 3, 2019

Issue #3806 async sendError

Avoid using isHandled as a test withing sendError as this can be
called asynchronously and is in a race with the normal dispatch of the
request, which could also be setting handled status.

Signed-off-by: Greg Wilkins [email protected]

Avoid using isHandled as a test withing sendError as this can be
called asynchronously and is in a race with the normal dispatch of the
request, which could also be setting handled status.

Signed-off-by: Greg Wilkins <[email protected]>
@gregw gregw requested review from janbartel and sbordet July 3, 2019 10:38
The ErrorHandler was dispatching directly to a context from within
sendError.  This meant that an async thread can call sendError and be
dispatched to within the servlet container at the same time that the
original thread was still dispatched to the container.

This commit fixes that problem by using an async dispatch for error
pages within the ErrorHandler.  However, this introduces a new problem
that a well behaved async app will call complete after calling
sendError.  Thus we have ignore complete ISEs for the remainder of
the current async cycle.

Signed-off-by: Greg Wilkins <[email protected]>
@gregw gregw marked this pull request as ready for review July 3, 2019 12:59
@gregw
Copy link
Contributor Author

gregw commented Jul 3, 2019

@sbordet @janbartel This PR now fixes both problems: using isHandled and dispatching from within the ErrorHandler.

Fixed the closing of the output after calling sendError. Do not
close if the request was async (and thus might be dispatched to an
async error) or if it is now async because the error page itself is
async.

Signed-off-by: Greg Wilkins <[email protected]>
@sbordet
Copy link
Contributor

sbordet commented Jul 4, 2019

@gregw I am not sure I understand the fix. Would be great if you add a comment describing the design of your solution, and how it fixes the problem.

@gregw
Copy link
Contributor Author

gregw commented Jul 7, 2019

See discussion starting here.

I think we need a more involved solution. Both sync and async error dispatches need to be delayed until either sync returns or async completes... only then can an error dispatch be done - ie never from within sendError.
We will need a different resetBuffer (or one with an argument) that leaves the outputStream closed, to be reopened only just before the error dispatch.

@gregw
Copy link
Contributor Author

gregw commented Jul 7, 2019

Extending the delayed error dispatch approach to handle async sendError calls is promising, but it still needs careful though.

Consider this sequence:

  • Thread A Normal servlet dispatch
  • Thread A startAsync called
  • Thread A returns from servlet dispatch
  • Thread B sendError called (state set, output temporarily closed)
  • Thread B complete called (state update - no further action at this time)
  • Thread B ERROR dispatch generates response and returns
  • Thread B calls onComplete

This looks OK and could also play out as:

  • Thread A Normal servlet dispatch
  • Thread A startAsync called
  • Thread B sendError called (state set, output temporarily closed)
  • Thread B complete called (state update - no further action at this time)
  • Thread A returns from servlet dispatch
  • Thread A ERROR dispatch generates response and returns
  • Thread A calls onComplete

But what if ERROR dispatch is itself async?

  • Thread A Normal servlet dispatch
  • Thread A startAsync called
  • Thread A returns from servlet dispatch
  • Thread B sendError called (state set, output temporarily closed)
  • Thread B complete called (state update - no further action at this time)
  • Thread B ERROR dispatch
  • Thread B calls startAsync and returns
  • Thread C generates error response
  • Thread C calls complete
  • Thread C calls onComplete

OK as well I think.... and with async Dispatch:

  • Thread A Normal servlet dispatch
  • Thread A startAsync called
  • Thread A returns from servlet dispatch
  • Thread B sendError called (state set, output temporarily closed)
  • Thread B complete called (state update - no further action at this time)
  • Thread B ERROR dispatch
  • Thread B calls startAsync and returns
  • Thread C calls AsyncContext.dispatch
  • Thread C(or D) ASYNC dispatch
  • Thread C(or D) generates response and returns
  • Thread C(or D) calls onComplete

OK still. I think this can work.

gregw added 8 commits July 9, 2019 08:16
Signed-off-by: Greg Wilkins <[email protected]>
Signed-off-by: Greg Wilkins <[email protected]>
Signed-off-by: Greg Wilkins <[email protected]>
Signed-off-by: Greg Wilkins <[email protected]>
Signed-off-by: Greg Wilkins <[email protected]>
Signed-off-by: Greg Wilkins <[email protected]>
@joakime
Copy link
Contributor

joakime commented Jul 15, 2019

This PR somehow broke the HTTP/2 layer in this branch?

gregw added 3 commits July 17, 2019 08:35
Reworked HttpChannelState and sendError so that sendError is now
just a change of state. All the work is done in the ErrorDispatch
action, including calling the ErrorHandler.  Async not yet working.

Signed-off-by: Greg Wilkins <[email protected]>
Additional tests

Signed-off-by: Greg Wilkins <[email protected]>
Converted ERRORED state to a separate boolean so it can be used for
both Sync and Async dispatches.

Removed ASYNC_IO state as it was just the same as DISPATCHED

The async onError listener handling is now most likely broken.

Signed-off-by: Greg Wilkins <[email protected]>
gregw added 7 commits July 18, 2019 10:11
WIP making sendError simpler and more tests pass

Signed-off-by: Greg Wilkins <[email protected]>
WIP handling async and thrown exceptions

Signed-off-by: Greg Wilkins <[email protected]>
WIP passing tests

Signed-off-by: Greg Wilkins <[email protected]>
Improved thread handling

Signed-off-by: Greg Wilkins <[email protected]>
removed bad test

Signed-off-by: Greg Wilkins <[email protected]>
Implemented error dispatch on complete properly
more fixed tests

Signed-off-by: Greg Wilkins <[email protected]>
@gregw gregw requested a review from sbordet July 19, 2019 15:19
gregw added 6 commits July 20, 2019 10:05
sendError state looks committed

Signed-off-by: Greg Wilkins <[email protected]>
Signed-off-by: Greg Wilkins <[email protected]>
Signed-off-by: Greg Wilkins <[email protected]>
- Added resetContent method to leave more non-content headers during sendError
- Fixed security tests

Signed-off-by: Greg Wilkins <[email protected]>
- simplified the non dispatch error page writing.  Moved towards being able to write async

Signed-off-by: Greg Wilkins <[email protected]>
Signed-off-by: Greg Wilkins <[email protected]>
@gregw
Copy link
Contributor Author

gregw commented Jul 26, 2019

See discussion in jakartaee/servlet#256

gregw added 9 commits July 27, 2019 09:49
Updated handling of timeout errors.  According to servlet spec,
exceptions thrown from onTimeout should not be passed to onError, but
just logged and ignored:

   If an exception is thrown while invoking methods in an AsyncListener,
   it is logged and will not affect the invocation of any other AsyncListeners.

This changes several tests.

Dispatcher/ContextHandler changes for new ERROR dispatch handling. Feels
a bit fragile!

Signed-off-by: Greg Wilkins <[email protected]>
Fixed tests in jetty-servlets

Signed-off-by: Greg Wilkins <[email protected]>
Fixed tests in jetty-proxy

Signed-off-by: Greg Wilkins <[email protected]>
Signed-off-by: Greg Wilkins <[email protected]>
Fixed head handling

Signed-off-by: Greg Wilkins <[email protected]>
reverted HttpMethod fromString

Signed-off-by: Greg Wilkins <[email protected]>
reverted unnecessary changes

Signed-off-by: Greg Wilkins <[email protected]>
Improved reason handling

Signed-off-by: Greg Wilkins <[email protected]>
@gregw
Copy link
Contributor Author

gregw commented Aug 1, 2019

closing in favour of #3912

@gregw gregw closed this Aug 1, 2019
@gregw gregw deleted the jetty-9.4.x-3806-async-sendError branch August 13, 2019 01:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants