Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to 4.6.0 -> Unexpected ASGI message 'websocket.close', after sending 'websocket.close' #327

Closed
detoyz opened this issue Aug 21, 2023 · 4 comments
Assignees
Labels

Comments

@detoyz
Copy link

detoyz commented Aug 21, 2023

Describe the bug
After updating to 4.6.0 we started to experience abnormal amount of errors on our production systems reported in sentry.
Screenshots from Sentry:
image

To Reproduce
Steps to reproduce the behavior:

  1. Update engineio from 4.5.1 to 4.6.0
  2. Deploy to production
  3. Start receiving errors

Note: I know that the "steps to reproduce" doesn't help here much, but this is exactly what happened before we started to receive errors. No other changes was made.

Expected behavior
No errors are reported by Sentry.

Logs

RuntimeError: Unexpected ASGI message 'websocket.close', after sending 'websocket.close'.
  File "starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "starlette/middleware/errors.py", line 149, in __call__
    await self.app(scope, receive, send)
  File "starlette/middleware/base.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "starlette/routing.py", line 443, in handle
    await self.app(scope, receive, send)
  File "engineio/async_drivers/asgi.py", line 58, in __call__
    await self.engineio_server.handle_request(scope, receive, send)
  File "socketio/asyncio_server.py", line 398, in handle_request
    return await self.eio.handle_request(*args, **kwargs)
  File "engineio/asyncio_server.py", line 257, in handle_request
    r = await self._handle_connect(environ, transport,
  File "engineio/asyncio_server.py", line 430, in _handle_connect
    ret = await s.handle_get_request(environ)
  File "engineio/asyncio_socket.py", line 89, in handle_get_request
    return await getattr(self, '_upgrade_' + transport)(environ)
  File "engineio/asyncio_socket.py", line 143, in _upgrade_websocket
    return await ws(environ)
  File "engineio/async_drivers/asgi.py", line 235, in __call__
    await self.handler(self)
  File "engineio/asyncio_socket.py", line 253, in _websocket_handler
    await asyncio.wait_for(writer_task, timeout=None)
  File "asyncio/tasks.py", line 442, in wait_for
    return await fut
  File "engineio/asyncio_socket.py", line 207, in writer
    await ws.close()
  File "engineio/async_drivers/asgi.py", line 238, in close
    await self.asgi_send({'type': 'websocket.close'})
  File "starlette/middleware/exceptions.py", line 65, in sender
    await send(message)
  File "starlette/middleware/exceptions.py", line 65, in sender
    await send(message)
  File "uvicorn/protocols/websockets/websockets_impl.py", line 345, in asgi_send
    raise RuntimeError(msg % message_type)

Additional context
Update that took place
image

Our setup on server:

import socketio

from src.infra import config
from src.infra.cache.redis import RedisDB
from src.infra.socket_server.namespaces_container import register_namespaces

mgr = socketio.AsyncRedisManager(f"{config.redis.url}/{RedisDB.websockets}")
sio = socketio.AsyncServer(
    async_mode="asgi",
    transports=["websocket"],
    logger=config.main.debug,
    client_manager=mgr,
)

# connect to the redis queue as an external process
register_namespaces(sio)
app = socketio.ASGIApp(sio)
import server

app = FastAPI(...)
app.mount(app=server.app, path="/ws")  # connect websocket routes
uvicorn = {version = "^0.23.2", extras = ["standard"]}
fastapi = "^0.101.1"
gunicorn = "^21.2.0"
uvloop = "^0.17.0"
python-socketio = "^5.8.0"

Our setup on client:

const [socket, setSocket] = useState<Socket | undefined>(undefined);
useEffect(() => {
    const socket = io(`${config.api()}${namespace}`, {
      transports: ['websocket'],
      path: '/ws/socket.io',
      withCredentials: true,
      autoConnect: false,
      query: { channel_resource_id: channelResourceId },
      forceNew: true,
      auth
    });
    setSocket(socket);
    socket.connect();
    socket.on('connect', () => {
      socket.emit('init', {
        count: MESSAGES_LOAD_LIMIT + 1
      });
      socket.emit('subscribe');
    });

    return () => {
      // unsubscribe
      socket.emit('unsubscribe');

      // remove all listeners
      socket.off();

      // close socket.io connection
      socket.disconnect();
    };
  }, [channelResourceId, auth, namespace]);
@detoyz detoyz changed the title Update to 4.6.0 tries to "double close" Update to 4.6.0 -> Unexpected ASGI message 'websocket.close', after sending 'websocket.close' Aug 21, 2023
@miguelgrinberg miguelgrinberg self-assigned this Aug 21, 2023
@miguelgrinberg
Copy link
Owner

@detoyz Can I ask you to install python-engineio from its main branch on this repo and retest? I believe the fix committed above addresses the problem. Once you confirm that the problem is addressed I'll do another release. Thanks.

@detoyz
Copy link
Author

detoyz commented Aug 21, 2023

@detoyz Can I ask you to install python-engineio from its main branch on this repo and retest? I believe the fix committed above addresses the problem. Once you confirm that the problem is addressed I'll do another release. Thanks.

Will do, thanks for the quick reply!
P.S. I really appreciate your work on socketio, it's an amazing library to work with ❤️

@detoyz
Copy link
Author

detoyz commented Aug 22, 2023

@miguelgrinberg new version did the trick, as of 2 hours after deploy 0 error messages
image

@miguelgrinberg
Copy link
Owner

Release 4.6.1 is out with this fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants