Fugue is a Python implementation of the interceptor concept as seen in, and heavily inspired by, Pedestal. It is currently built on Twisted, the event-driven networking engine for Python, and Pyrsistent for immutable data structures.
Briefly, an interceptor is a reusable, composable component responsible for an individual aspect of the overall behaviour of a web service, such as parsing a query string or performing content negotiation. Combining interceptors produces an execution chain that is easily expressed, understood and tested; logic is kept small and isolated.
The motivation for Fugue was heavily inspired by Pedestal and the idea of a more composable, functional, reusable way of describing a web application that can (hopefully!) remain simple to reason about and test even as application complexity grows.
The goal for Fugue is to give web application developers the freedom to focus on the logic of their application and the ability to easily build up an applicaton out of small reusable functions, without having to concern themselves with the details of their web server.
Twisted Web is a production-ready HTTP (and HTTP2!) server implemented in pure Python using Twisted. It is mature, well supported and can be embedded (and customized!) in your Python application. Nevow is a web framework built on Twisted and Twisted Web, offering an HTTP server-push "widget" system, templates and other features.
Nevow's resource model is very closely based on Twisted Web's but is unfortunately incompatible, Twisted Web's resource model was implemented nearly two decades ago and hasn't seen much change since then. Using it can be quite cumbersome for complex web applications, and understanding such a system tends to be even more difficult. Various attempts exist—Klein, and even my own—to improve the developer experience of using Twisted Web, often attempting to smooth over or hide as much of the resource model as possible.
Ideally a web application developer would only concern themselves with processing some input data, applying some application / business logic to that data (possibly over several incremental steps) and producing an output. At the fringes of the application are the uninteresting, mechanical details: The resource model; writing a request back to the network; unserializing requests and serializing responses; and so forth.
Maybe Fugue can be that ideal.
Interceptors are the foundation of Fugue, and most of the library is dedicated to providing interceptors that are useful for building HTTP services.
An interceptor is a pair of unary functions that accept a context map—an
immutable data structure—and must eventually return a context map. One function
(enter) is called on the way "in" and another (leave) is called on the
way "out". Either function may be omitted and the effect is that the context map
remains unchanged.
Interceptors are combined to produce a particular order of execution, the "enter" stage is called in order for each interceptor with the—possibly modified—context map flowing from one to the next. Once all interceptors have been called, the "leave" stage is called in reverse order for each interceptor threading the context map—resulting from the "enter" stage—through them; illustrated below:
┌───────────┐ ┌───────────┐
│Context map│ │Context map│
└─────┬─────┘ └─────▲─────┘
│ │
┌───────┼─────────────────────┼───────┐
│ ┌──▼──┐ ┌──┴──┐ │
│ │Enter│ │Leave│ │ Interceptor
│ └──┬──┘ └──▲──┘ │
└───────┼─────────────────────┼───────┘
│ │
┌───────┼─────────────────────┼───────┐
│ ┌──▼──┐ ┌──┴──┐ │
│ │Enter│ │Leave│ │ Interceptor
│ └──┬──┘ └──▲──┘ │
└───────┼─────────────────────┼───────┘
│ │
┌─────▼─────┐ ┌─────┴─────┐
│Context map├ ─ ─ ─ ─ ▶Context map│
└───────────┘ └───────────┘
Asynchronous results, in the form of a Twisted Deferred, may be returned from any stage of an interceptor; the effect is that execution of the interceptor chain is paused until the result becomes available.
Fugue keeps a queue of interceptors that have yet to be called in the context
map itself. Since interceptors are free to modify the context map, this means
they are also able to modify the remaining flow of execution! Terminating the
"enter" stage is a matter of clearing the queue, extending it is a matter of
enqueuing new interceptors; achieved by terminate and enqueue
respectively.
A basic interceptor to attach a UUID to some uuid key on enter:
Interceptor(
name='uuid',
enter=lambda context: context.set(ns('uuid'), uuid4()))Interceptors executing after this example would find a ns('uuid') key in the
context map containing a random UUID. In this case ns is some function
intended to produce namespaced keys to avoid collisions with either internal or
external keys. Fugue provides a basic function to help achieve this in the form
of namespace.
A common pattern is to produce an interceptor from a function and capture the arguments of the function (via a closure) within the interceptor's enter or leave functions. For example attaching a database connection to each request:
def attach_database(uri):
return Interceptor(
name='db',
enter=lambda context: context.set(ns('db'), connect_db(uri)))Errors are a natural part of programming, however the normal methods of handling them are not as useful within the context of an interceptor chain, if only because they may arise asynchronously.
Instead Fugue traps synchronous and asynchronous errors within interceptors and
attaches them to an ERROR key in the context map. The "enter" stage is
terminated and the "leave" stage immediately begins, however as long as there is
an ERROR key in the context map only the error function of interceptors
along the chain will be invoked.
An error may be handled by returning a context map without the ERROR key.
When this happens the leave function of the next interceptor is invoked and
the "leave" stage continues as normal from that point.
If execution ends without the error having been handled it will be be raised
(asynchronously, via a Deferred errback.
An interceptor's error function is invoked with the context map (devoid of
an ERROR key, for convenience) and the value of the ERROR key.
The error function can do one of several things:
- Return the context map as-is. This is catching the error because there is no
longer an
ERRORkey present and execution will resume normally. - Return the context map with the error reattached to the
ERRORkey. This is reraising the error and the search for an error handler will continue. - Raise a new error. This is the error handler encountering a new error trying to handle the original error, the search for an error handler will continue but for the new error instead.
A context map is passed to each interceptor's enter and leave functions.
Below are the basic keys you can expect to find, any key not listed below should
be considered an implementation detail subject to change, either in Fugue itself
or the interceptor responsible for creating the key.
It should be noted that context map returned from each interceptor should be a transformed version of the one received and not a new map. Interceptors may arbitrarily add new keys that should be preserved.
| Key | Description |
|---|---|
ERROR |
An object indicating a Failure, in a failure attribute. |
EXECUTION_ID |
A unique identifier set when the chain is executed. |
QUEUE |
The interceptors left to execute, should be manipulated by
enqueue, terminate and terminate_when. |
TERMINATORS |
Predicates executed after each enter function, the
"enter" stage is terminated if any return a true value. |
When using Fugue's HTTP request handling the REQUEST and RESPONSE keys
will be present, containing information about the request to process and the
response to return.
The request map is attached before the first interceptor is executed, it describes the incoming HTTP request:
| Key | Description |
|---|---|
body |
file-like object containing the body of the request. |
content_type |
Content-Type header. |
content_length |
Content-Length header. |
character_encoding |
Content encoding of the Content-Type header, defaults
to utf-8. |
headers |
Map of header names to vectors of header values. |
request_method |
HTTP method. |
uri |
URL the request is being made to. |
The response map is attached by any interceptor in the chain wishing to influence the HTTP response. If no response map exists when execution completes an HTTP 404 response is generated.
| Key | Description |
|---|---|
status |
HTTP status code as an int. |
headers |
Optional map of HTTP response headers to include. |
body |
Response body as bytes. |
Adapters are the mechanism that bind the external world (such as a web server) to the internal world of interceptors. If interceptors consume and produce immutable data via the context map then adapters transform some external information (such as an HTTP request) to and from that pure data.
This way the majority of the request processing (including application logic) is unconcerned with the particular web server implementation, the adapter enqueues the necessary interceptor to transform incoming HTTP requests into data and outgoing data into HTTP responses.
Fugue provides a Twisted Web adapter in the form of an IResource, the effect of this adapter is to act as a leaf resource—meaning Twisted performs no child resource lookups on it—that converts a Twisted Web request into a context map, executes an interceptor chain, and converts the context map back into something Twisted Web can respond to the request with.
An adapter has no formal structure since the coupling will depend on what is being adapted.
A basic HTTP API example that returns a personal greeting based on a route:
from pyrsistent import m
from fugue.interceptors.http import route
from fugue.interceptors.http.route import GET
from fugue.adapters.twisted import twisted_adapter_resource
# Define a helper to construct HTTP 200 responses.
def ok(body):
return m(status=200, body=body.encode('utf-8'))
# Define the handler behaviour.
def greet(request):
name = request['path_params']['name']
return ok(u'Hello, {}!'.format(name))
# Declare the route.
interceptor = route.router(
(u'/greet/:name', GET, greet))
# Create a Twisted Web resource that will execute the interceptor chain.
resource = twisted_adapter_resource([interceptor])Executing the example:
# Run the script from a Fugue checkout.
$ twistd -n web --resource-script=examples/twisted_greet.py
# Use the service.
$ curl 'http://localhost:8080/greet/Bob'
Hello, Bob!pip install fugueSee CONTRIBUTING.rst.