Support nonblocking writing to Jetty Websocket with possibility to assign callback [SPR-15527]

[spring-projects-issues]

Askar Ibragimov opened SPR-15527 and commented

Jetty websocket server has two ways to send messages:

• blocking Write http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/RemoteEndpoint.html#sendBytes-java.nio.ByteBuffer-
• non-blocking Write http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/RemoteEndpoint.html#sendBytes-java.nio.ByteBuffer-org.eclipse.jetty.websocket.api.WriteCallback- and http://download.eclipse.org/jetty/stable-9/apidocs/org/eclipse/jetty/websocket/api/RemoteEndpoint.html#sendBytesByFuture-java.nio.ByteBuffer-

Presently Spring Websocket uses blocking write in Jetty adapter. Which means that the message engine of Spring (ExecutorSubscribableChannel) has to have many threads and a thread remains parked till client acknowledges the reception of message.

There are few issues with this approach

• If client never acknowledges the reception on TCP level, the thread (from TaskExecutorChannel's threadpool) remains blocked. Actually due to a very funny decision of Jetty engineers to not set a timeout on these connections, so the block is never lifted (WebSocket hangs in blockingWrite jetty/jetty.project#272). In plain language, any Spring+Jetty app is exposed to a DDOS attack.
• This is a bad utilization of threads. I might want to write a logic that re-uses the thread for something else and just get notified via callback when web socket processed the message. This would allow me to have smaller thread pools and truly high-load-ready nonblocking architecture.

Suggested improvement
Permit in Spring usage of non-blockig websocket write with possibility to expose respective callback or future on upper level. Then custom or new ExecutorChannel implementation may use these callbacks to smartly utilize smaller number of threads and do not have blocked threads.

Useful contribution from Jetty people (caveats of using non blocking API of Jetty) http://stackoverflow.com/questions/43257736/make-websocket-sending-non-blocking

---

Issue Links:

• Jetty - use Async Send like Atmosphere does [SPR-13602] #18180 Jetty - use Async Send like Atmosphere does
• Expose asynchronous message sending in WebSocketSession [SPR-14099] #18671 Expose asynchronous message sending in WebSocketSession

Referenced from: commits 1b0e95d

1 votes, 3 watchers

[spring-projects-issues]

Rossen Stoyanchev commented

Indeed Jetty and JSR-356 have non-blocking write. The trouble is replacing blocking with non-blocking is never easy, usually not possible without a re-write.

When using the STOMP over WebSocket, the sending code is isolated in StompSubProtocolHandler so that may not be too hard. The challenge however is we can only do that for WebSocket. The SockJS support would require a complete re-write based on Servlet 3.1 non-blocking I/O. Moreover there is no way to make the broker slow down with the publication of messages so either way at some point we have to terminate a session that is taking too long to send or that has exceeded a send buffer. This a feature that we already support to help minimize the impact of slow clients (see my answer under #18180) but obviously it comes at the cost of blocking one thread per slow client.

The other side of the story is Spring WebFlux, new in 5.0, completely focused on non-blocking I/O from the ground up. In fact we already support non-blocking WebSocket client and server there (including Jetty) but we don't yet have STOMP messaging support on top.

Most likely we will focus on non-blocking support on the WebFlux side, so I'm setting this for consideration in 5.1.

[rstoyanchev]

WebFlux provides an async/non-blocking WebSocket API for Jetty included.