Corrupted WebSocket messages by multi-threaded publishing to external clients [SPR-11102] #15728
Description
Sergey Shcherbakov opened SPR-11102 and commented
I have an application which is running embedded Tomcat 8.0.0.RC3.
There is one 'request' and one 'reply' messaging channel in my app.
The 'reply' channel is piping messages in direction of external WebSocket clients and is backed by a single thread ThreadPoolExecutor.
Afaics, this single thread is the one that performs actual writing to the socket and is the only one serving all external application WebSocket clients.
If I expect many clients and many messages and I want to utilize the processor fully, a good idea would be to increase the number of threads that serve the 'reply' messaging channel.
But that doesn't really work.
A single connected WebSocket client which accepts messages that are being published by 4 threads I receive corrupted messages on the client side:
2013-11-19 17:37:04,242 14603 WARN [WebSocketClient@811955882-15] org.eclipse.jetty.websocket.common.Parser -
org.eclipse.jetty.websocket.api.ProtocolException: CONTINUATION frame without prior !FIN
at org.eclipse.jetty.websocket.common.Parser.parseFrame(Parser.java:366) ~[websocket-common-9.1.0.RC1.jar:9.1.0.RC1]
at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:240) ~[websocket-common-9.1.0.RC1.jar:9.1.0.RC1]
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.read(AbstractWebSocketConnection.java:597) [websocket-common-9.1.0.RC1.jar:9.1.0.RC1]
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:504) [websocket-common-9.1.0.RC1.jar:9.1.0.RC1]
I'm using the Jetty 9.1.0.RC1 client for the tests (I also saw the same issue using its last GA version).
It looks like a synchronization issue at writing to the socket. One thread per client works fine but I don't find the way to arrange a one thread per WS client when there are several of them connected.
I am using SubProtocolWebSocketHandler with StompProtocolHandler as default sub-protocol handler.
The simple synchronization around the WebSocketSession (I wrap StompProtocolHandler):
...
@Override
public void handleMessageToClient(WebSocketSession session,
Message<?> message) throws Exception {
synchronized (session) {
subProtocolHandler.handleMessageToClient(session, message);
}
}fixes the issue and I don't see any corrupted messages on the client side even when the 'reply' messaging channel is backed by multiple threads.
Affects: 4.0 RC1