From d014ca534e20ad37c75484ae151e3cec3809c200 Mon Sep 17 00:00:00 2001 From: Miguel Grinberg Date: Sat, 8 Dec 2018 14:04:36 +0000 Subject: [PATCH] documentation updates --- docs/client.rst | 158 ++++++++++++++++++++++++ docs/index.rst | 10 +- docs/intro.rst | 181 ++++++++++++++-------------- docs/{deployment.rst => server.rst} | 156 +++++++++++++++++++++--- 4 files changed, 398 insertions(+), 107 deletions(-) create mode 100644 docs/client.rst rename docs/{deployment.rst => server.rst} (73%) diff --git a/docs/client.rst b/docs/client.rst new file mode 100644 index 00000000..a1762578 --- /dev/null +++ b/docs/client.rst @@ -0,0 +1,158 @@ +The Engine.IO Client +==================== + +This package contains two Engine.IO clients: + +- The :func:`engineio.Client` class creates a client compatible with the + standard Python library. +- The :func:`engineio.AsyncClient` class creates a client compatible with + the ``asyncio`` package. + +The methods in the two clients are the same, with the only difference that in +the ``asyncio`` client most methods are implemented as coroutines. + +Creating a Client Instance +-------------------------- + +To instantiate an Engine.IO client, simply create an instance of the +appropriate client class:: + + import engineio + + # standard Python + eio = engineio.Client() + + # asyncio + eio = engineio.AsyncClient() + +Defining Event Handlers +----------------------- + +To responds to events triggered by the connection or the server, event Handler +functions must be defined using the ``on`` decorator:: + + @eio.on('connect') + def on_connect(): + print('I'm connected!') + + @eio.on('message') + def on_message(data): + print('I received a message!') + + @eio.on('disconnect') + def on_disconnect(): + print('I'm disconnected!') + +For the ``asyncio`` server, event handlers can be regular functions as above, +or can also be coroutines:: + + @eio.on('message') + async def on_message(data): + print('I received a message!') + +The argument given to the ``on`` decorator is the event name. The events that +are supported are ``connect``, ``message`` and ``disconnect``. Note that the +``disconnect`` handler is invoked for application initiated disconnects, +server initiated disconnects, or accidental disconnects, for example due to +networking failures. + +The ``data`` argument passed to the ``'message'`` event handler contains +application-specific data provided by the server with the event. + +Connecting to a Server +---------------------- + +The connection to a server is established by calling the ``connect()`` +method:: + + eio.connect('http://localhost:5000') + +In the case of the ``asyncio`` client, the method is a coroutine:: + + await eio.connect('http://localhost:5000') + +Sending Messages +---------------- + +The client can send a message to the server using the ``send()`` method:: + + eio.send({'foo': 'bar'}) + +Or in the case of ``asyncio``, as a coroutine:: + + await eio.send({'foo': 'bar'}) + +The single argument provided to the method is the data that is passed on +to the server. The data can be of type ``str``, ``bytes``, ``dict`` or +``list``. The data included inside dictionaries and lists is also +constrained to these types. + +The ``send()`` method can be invoked inside an event handler as a response +to a server event, or in any other part of the application, including in +background tasks. + +Disconnecting from the Server +----------------------------- + +At any time the client can request to be disconnected from the server by +invoking the ``disconnect()`` method:: + + eio.disconnect() + +For the ``asyncio`` client this is a coroutine:: + + await eio.disconnect() + +Managing Background Tasks +------------------------- + +When a client connection to the server is established, a few background +tasks will be spawned to keep the connection alive and handle incoming +events. The application running on the main thread is free to do any +work, as this is not going to prevent the functioning of the Engine.IO +client. + +If the application does not have anything to do in the main thread and +just wants to wait until the connection ends, it can call the ``wait()`` +method:: + + eio.wait() + +Or in the ``asyncio`` version:: + + await eio.wait() + +For the convenience of the application, a helper function is +provided to start a custom background task:: + + def my_background_task(my_argument) + # do some background work here! + pass + + eio.start_background_task(my_background_task, 123) + +The arguments passed to this method are the background function and any +positional or keyword arguments to invoke the function with. + +Here is the ``asyncio`` version:: + + async def my_background_task(my_argument) + # do some background work here! + pass + + eio.start_background_task(my_background_task, 123) + +Note that this function is not a coroutine, since it does not wait for the +background function to end, but the background function is. + +The ``sleep()`` method is a second convenince function that is provided for +the benefit of applications working with background tasks of their own:: + + eio.sleep(2) + +Or for ``asyncio``:: + + await eio.sleep(2) + +The single argument passed to the method is the number of seconds to sleep +for. diff --git a/docs/index.rst b/docs/index.rst index 1694832b..4a6373e8 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -6,14 +6,16 @@ python-engineio =============== -This project implements an Engine.IO server that can run standalone or -integrated with a variety of Python web frameworks. +This project implements Python based Engine.IO client and server that can run +standalone or integrated with a variety of Python web frameworks and +applications. .. toctree:: - :maxdepth: 2 + :maxdepth: 3 intro - deployment + client + server api * :ref:`genindex` diff --git a/docs/intro.rst b/docs/intro.rst index 84bd9fb0..324c80e9 100644 --- a/docs/intro.rst +++ b/docs/intro.rst @@ -10,64 +10,78 @@ What is Engine.IO? ------------------ Engine.IO is a lightweight transport protocol that enables real-time -bidirectional event-based communication between clients (typically web -browsers) and a server. The official implementations of the client and -server components are written in JavaScript. - -The Engine.IO protocol is extremely simple. The example that follows shows the -client-side Javascript code required to setup an Engine.IO connection to -a server:: - - var socket = eio('http://chat.example.com'); - socket.on('open', function() { alert('connected'); }); - socket.on('message', function(data) { alert(data); }); - socket.on('close', function() { alert('disconnected'); }); - socket.send('Hello from the client!'); - -Features --------- - -- Fully compatible with the Javascript - `engine.io-client `_ library, - and with other Engine.IO clients. -- Compatible with Python 2.7 and Python 3.3+. -- Supports large number of clients even on modest hardware due to being - asynchronous. -- Compatible with `aiohttp `_, - `sanic `_, - `tornado `_, - `eventlet `_, - `gevent `_, - or any `WSGI `_ or - `ASGI `_ compatible server. -- Includes WSGI and ASGI middlewares that integrate Engine.IO traffic with - other web applications. +bidirectional event-based communication between clients (typically, though +not always, web browsers) and a server. The official implementations of the +client and server components are written in JavaScript. This package provides +Python implementations of both, each with standard and ``asyncio`` variants. + +The Engine.IO protocol is extremely simple. Once a connection between a client +and a server is established, either side can send "messages" to the other +side. Event handlers provided by the applications on both ends are invoked +when a message is received, or when a connection is established or dropped. + +Client Examples +--------------- + +The example that follows shows a simple Python client:: + + import engineio + + eio = engineio.Client() + + @eio.on('connect') + def on_connect(): + print('connection established') + + @eio.on('message') + def on_message(data): + print('message received with ', data) + eio.send({'response': 'my response'}) + + @eio.on('disconnect') + def on_disconnect(): + print('disconnected from server') + + eio.connect('http://localhost:5000') + eio.wait() + +And here is a similar client written using the official Engine.IO Javascript +client:: + + + + +Client Features +--------------- + +- Can connect to other Engine.IO complaint servers besides the one in this package. +- Compatible with Python 2.7 and 3.5+. +- Two versions of the client, one for standard Python and another for ``asyncio``. - Uses an event-based architecture implemented with decorators that hides the details of the protocol. - Implements HTTP long-polling and WebSocket transports. -- Supports XHR2 and XHR browsers as clients. -- Supports text and binary messages. -- Supports gzip and deflate HTTP compression. -- Configurable CORS responses to avoid cross-origin problems with browsers. -Examples --------- +Server Examples +--------------- The following application is a basic example that uses the Eventlet -asynchronous server and includes a small Flask application that serves the -HTML/Javascript to the client:: +asynchronous server:: import engineio import eventlet - from flask import Flask, render_template eio = engineio.Server() - app = Flask(__name__) - - @app.route('/') - def index(): - """Serve the client-side application.""" - return render_template('index.html') + app = engineio.WSGIApp(eio, static_files={ + '/': {'content_type': 'text/html', 'filename': 'index.html'} + }) @eio.on('connect') def connect(sid, environ): @@ -83,28 +97,18 @@ HTML/Javascript to the client:: print('disconnect ', sid) if __name__ == '__main__': - # wrap Flask application with engineio's middleware - app = engineio.Middleware(eio, app) - - # deploy as an eventlet WSGI server - eventlet.wsgi.server(eventlet.listen(('', 8000)), app) + eventlet.wsgi.server(eventlet.listen(('', 5000)), app) -Below is a similar application, coded for asyncio (Python 3.5+ only) with the -aiohttp framework:: +Below is a similar application, coded for asyncio (Python 3.5+ only) and the +Uvicorn web server:: - from aiohttp import web import engineio + import uvicorn eio = engineio.AsyncServer() - app = web.Application() - - # attach the Engine.IO server to the application - eio.attach(app) - - async def index(request): - """Serve the client-side application.""" - with open('index.html') as f: - return web.Response(text=f.read(), content_type='text/html') + app = engineio.ASGIApp(eio, static_files={ + '/': {'content_type': 'text/html', 'filename': 'index.html'} + }) @eio.on('connect') def connect(sid, environ): @@ -119,30 +123,31 @@ aiohttp framework:: def disconnect(sid): print('disconnect ', sid) - app.router.add_static('/static', 'static') - app.router.add_get('/', index) - if __name__ == '__main__': - # run the aiohttp application - web.run_app(app) + uvicorn.run('127.0.0.1', 5000) -The client-side application must include the -`engine.io-client `_ library -(version 1.5.0 or newer recommended). +Server Features +--------------- -Each time a client connects to the server the ``connect`` event handler is -invoked with the ``sid`` (session ID) assigned to the connection and the WSGI -environment dictionary. The server can inspect authentication or other headers -to decide if the client is allowed to connect. To reject a client the handler -must return ``False``. - -When the client sends a message to the server the ``message`` event handler is -invoked with the ``sid`` and the message. - -Finally, when the connection is broken, the ``disconnect`` event is called, -allowing the application to perform cleanup. - -Because Engine.IO is a bidirectional protocol, the server can send messages to -any connected client at any time. The ``engineio.Server.send()`` method takes -the client's ``sid`` and the message payload, which can be of type ``str``, -``bytes``, ``list`` or ``dict`` (the last two are JSON encoded). +- Can accept clients running other complaint Engine.IO clients besides the one in this + package. +- Compatible with Python 2.7 and Python 3.5+. +- Two versions of the server, one for standard Python and another for ``asyncio``. +- Supports large number of clients even on modest hardware due to being + asynchronous. +- Can be hosted on any `WSGI `_ and + `ASGI `_ web servers includind + `Gunicorn `_, `Uvicorn `_, + `eventlet `_ and `gevent `_. +- Can be integrated with WSGI applications written in frameworks such as Flask, Django, + etc. +- Can be integrated with `aiohttp `_, + `sanic `_ and `tornado `_ + ``asyncio`` applications. +- Uses an event-based architecture implemented with decorators that hides the + details of the protocol. +- Implements HTTP long-polling and WebSocket transports. +- Supports XHR2 and XHR browsers as clients. +- Supports text and binary messages. +- Supports gzip and deflate HTTP compression. +- Configurable CORS responses to avoid cross-origin problems with browsers. diff --git a/docs/deployment.rst b/docs/server.rst similarity index 73% rename from docs/deployment.rst rename to docs/server.rst index a92e0ba8..7dc59834 100644 --- a/docs/deployment.rst +++ b/docs/server.rst @@ -1,11 +1,137 @@ -Deployment -========== +The Engine.IO Server +==================== + +Creating a Server Instance +-------------------------- + +To instantiate an Engine.IO server, simply create an instance of the +appropriate client class:: + + import engineio + + # standard Python + eio = engineio.Server() + + # asyncio + eio = engineio.AsyncServer() + +Defining Event Handlers +----------------------- + +To responds to events triggered by the connection or the client, event Handler +functions must be defined using the ``on`` decorator:: + + @eio.on('connect') + def on_connect(sid): + print('A client connected!') + + @eio.on('message') + def on_message(sid, data): + print('I received a message!') + + @eio.on('disconnect') + def on_disconnect(sid): + print('Client disconnected!') + +For the ``asyncio`` server, event handlers can be regular functions as above, +or can also be coroutines:: + + @eio.on('message') + async def on_message(sid, data): + print('I received a message!') + +The argument given to the ``on`` decorator is the event name. The events that +are supported are ``connect``, ``message`` and ``disconnect``. Note that the +``disconnect`` handler is invoked for client initiated disconnects, +server initiated disconnects, or accidental disconnects, for example due to +networking failures. + +The ``sid`` argument passed into all the event handlers is a connection +identifier for the client. All the events from a client will use the same +``sid`` value. + +The ``data`` argument passed to the ``'message'`` event handler contains +application-specific data provided by the client with the event. + +Sending Messages +---------------- + +The server can send a message to any client using the ``send()`` method:: + + eio.send(sid, {'foo': 'bar'}) + +Or in the case of ``asyncio``, as a coroutine:: + + await eio.send(sid, {'foo': 'bar'}) + +The first argument provided to the method is the connection identifier for +the recipient client. The second argument is the data that is passed on +to the server. The data can be of type ``str``, ``bytes``, ``dict`` or +``list``. The data included inside dictionaries and lists is also +constrained to these types. + +The ``send()`` method can be invoked inside an event handler as a response +to a client event, or in any other part of the application, including in +background tasks. + +Disconnecting a Client +---------------------- + +At any time the server can disconnect a client from the server by invoking the +``disconnect()`` method and passing the ``sid`` value assigned to the client:: + + eio.disconnect(sid) + +For the ``asyncio`` client this is a coroutine:: + + await eio.disconnect(sid) + +Managing Background Tasks +------------------------- + +For the convenience of the application, a helper function is provided to +start a custom background task:: + + def my_background_task(my_argument) + # do some background work here! + pass + + eio.start_background_task(my_background_task, 123) + +The arguments passed to this method are the background function and any +positional or keyword arguments to invoke the function with. + +Here is the ``asyncio`` version:: + + async def my_background_task(my_argument) + # do some background work here! + pass + + eio.start_background_task(my_background_task, 123) + +Note that this function is not a coroutine, since it does not wait for the +background function to end, but the background function is. + +The ``sleep()`` method is a second convenince function that is provided for +the benefit of applications working with background tasks of their own:: + + eio.sleep(2) + +Or for ``asyncio``:: + + await eio.sleep(2) + +The single argument passed to the method is the number of seconds to sleep +for. + +Deployment Strategies +--------------------- The following sections describe a variety of deployment strategies for Engine.IO servers. aiohttp -------- +~~~~~~~ `aiohttp `_ provides a framework with support for HTTP and WebSocket, based on asyncio. Support for this framework is limited @@ -32,7 +158,7 @@ The aiohttp application is then executed in the usual manner:: web.run_app(app) Tornado -------- +~~~~~~~ `Tornado `_ is a web framework with support for HTTP and WebSocket. Support for this framework requires Python 3.5 and @@ -65,7 +191,7 @@ The tornado application is then executed in the usual manner:: tornado.ioloop.IOLoop.current().start() Sanic ------ +~~~~~ `Sanic `_ is a very efficient asynchronous web server for Python 3.5 and newer. @@ -91,7 +217,7 @@ The Sanic application is then executed in the usual manner:: app.run() Uvicorn, Daphne, and other ASGI servers ---------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``engineio.ASGIApp`` class is an ASGI compatible application that can forward Engine.IO traffic to an ``engineio.AsyncServer`` instance:: @@ -102,7 +228,7 @@ forward Engine.IO traffic to an ``engineio.AsyncServer`` instance:: The application can then be deployed with any ASGI compatible web server. Eventlet --------- +~~~~~~~~ `Eventlet `_ is a high performance concurrent networking library for Python 2 and 3 that uses coroutines, enabling code to be written in @@ -123,8 +249,8 @@ using the provided ``engineio.Middleware``:: import eventlet eventlet.wsgi.server(eventlet.listen(('', 8000)), app) -Using Gunicorn with Eventlet -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Eventlet with Gunicorn +~~~~~~~~~~~~~~~~~~~~~~ An alternative to running the eventlet WSGI server as above is to use `gunicorn `_, a fully featured pure Python web server. The @@ -145,7 +271,7 @@ versions. While python-engineio does not require monkey patching, other libraries such as database drivers are likely to require it. Gevent ------- +~~~~~~ `Gevent `_ is another asynchronous framework based on coroutines, very similar to eventlet. An Engine.IO server deployed with @@ -179,8 +305,8 @@ follows:: pywsgi.WSGIServer(('', 8000), app, handler_class=WebSocketHandler).serve_forever() -Using Gunicorn with Gevent -~~~~~~~~~~~~~~~~~~~~~~~~~~ +Gevent with Gunicorn +~~~~~~~~~~~~~~~~~~~~ An alternative to running the gevent WSGI server as above is to use `gunicorn `_, a fully featured pure Python web server. The @@ -203,7 +329,7 @@ versions. While python-engineio does not require monkey patching, other libraries such as database drivers are likely to require it. uWSGI ------ +~~~~~ When using the uWSGI server in combination with gevent, the Engine.IO server can take advantage of uWSGI's native WebSocket support. @@ -226,7 +352,7 @@ server for the ``latency.py`` example on port 5000:: $ uwsgi --http :5000 --gevent 1000 --http-websockets --master --wsgi-file latency.py --callable app Standard Threads ----------------- +~~~~~~~~~~~~~~~~ While not comparable to eventlet and gevent in terms of performance, the Engine.IO server can also be configured to work with multi-threaded web @@ -264,7 +390,7 @@ Note that servers that use worker processes instead of threads, such as gunicorn, do not support an Engine.IO server configured in threading mode. Scalability Notes ------------------ +~~~~~~~~~~~~~~~~~ Engine.IO is a stateful protocol, which makes horizontal scaling more difficult. To deploy a cluster of Engine.IO processes hosted on one or