rye

A simple library to support http services. Currently, rye provides a middleware handler which can be used to chain http handlers together while providing statsd metrics for use with DataDog or other logging aggregators. In addition, rye comes with various pre-built middleware handlers for enabling functionality such as CORS and rate/CIDR limiting.

Setup

In order to use rye, you should vendor it and the statsd client within your project.

govendor fetch github.com/InVisionApp/rye
govendor fetch github.com/cactus/go-statsd-client/statsd

Why another middleware lib?

• rye is tiny - the core lib is ~143 lines of code (including comments)!
• Each middleware gets statsd metrics tracking for free including an overall error counter
• We wanted to have an easy way to say “run these two middlewares on this endpoint, but only one middleware on this endpoint”
  • Of course, this is doable with negroni and gorilla-mux, but you’d have to use a subrouter with gorilla, which tends to end up in more code
• Bundled helper methods for standardising JSON response messages
• Unified way for handlers and middlewares to return more detailed responses via the rye.Response struct (if they chose to do so).
• Pre-built middlewares for things like CORS support

Example

You can run an example locally to give it a try. The code for the example is here!

cd example
go run rye_example.go

Writing custom middleware handlers

Begin by importing the required libraries:

import (
    "github.com/cactus/go-statsd-client/statsd"
    "github.com/InVisionApp/rye"
)

Create a statsd client (if desired) and create a rye Config in order to pass in optional dependencies:

config := &rye.Config{
    Statter:  statsdClient,
    StatRate: DEFAULT_STATSD_RATE,
}

Create a middleware handler. The purpose of the Handler is to keep Config and to provide an interface for chaining http handlers.

middlewareHandler := rye.NewMWHandler(config)

Set up any global handlers by using the Use() method. Global handlers get pre-pended to the list of your handlers for EVERY endpoint. They are bound to the MWHandler struct. Therefore, you could set up multiple MWHandler structs if you want to have different collections of global handlers.

middlewareHandler.Use(middleware_routelogger)

Build your http handlers using the Handler type from the rye package.

type Handler func(w http.ResponseWriter, r *http.Request) *rye.Response

Here are some example (custom) handlers:

func homeHandler(rw http.ResponseWriter, r *http.Request) *rye.Response {
    fmt.Fprint(rw, "Refer to README.md for auth-api API usage")
    return nil
}

func middlewareFirstHandler(rw http.ResponseWriter, r *http.Request) *rye.Response {
    fmt.Fprint(rw, "This handler fires first.")
    return nil
}

func errorHandler(rw http.ResponseWriter, r *http.Request) *rye.Response {
    return &rye.Response{
        StatusCode: http.StatusInternalServerError,
        Err:        errors.New(message),
    }
}

Finally, to setup your handlers in your API (Example shown using Gorilla):

routes := mux.NewRouter().StrictSlash(true)

routes.Handle("/", middlewareHandler.Handle([]rye.Handler{
    a.middlewareFirstHandler,
    a.homeHandler,
})).Methods("GET")

log.Infof("API server listening on %v", ListenAddress)

srv := &http.Server{
    Addr:    ListenAddress,
    Handler: routes,
}

srv.ListenAndServe()

Statsd Generated by Rye

Rye comes with built-in configurable statsd statistics that you could record to your favorite monitoring system. To configure that, you'll need to set up a Statter based on the github.com/cactus/go-statsd-client and set it in your instantiation of MWHandler through the rye.Config.

When a middleware is called, it's timing is recorded and a counter is recorded associated directly with the http status code returned during the call. Additionally, an errors counter is also sent to the statter which allows you to count any errors that occur with a code equaling or above 500.

Example: If you have a middleware handler you've created with a method named loginHandler, successful calls to that will be recorded to handlers.loginHandler.2xx. Additionally you'll receive stats such as handlers.loginHandler.400 or handlers.loginHandler.500. You also will receive an increase in the errors count.

If you're sending your logs into a system such as DataDog, be aware that your stats from Rye can have prefixes such as statsd.my-service.my-k8s-cluster.handlers.loginHandler.2xx or even statsd.my-service.my-k8s-cluster.errors. Just keep in mind your stats could end up in the destination sink system with prefixes.

Using with Golang 1.7 Context

With Golang 1.7, a new feature has been added that supports a request specific context. This is a great feature that Rye supports out-of-the-box. The tricky part of this is how the context is modified on the request. In Golang, the Context is always available on a Request through http.Request.Context(). Great! However, if you want to add key/value pairs to the context, you will have to add the context to the request before it gets passed to the next Middleware. To support this, the rye.Response has a property called Context. This property takes a properly created context (pulled from the request.Context() function. When you return a rye.Response which has Context, the rye library will craft a new Request and make sure that the next middleware receives that request.

Here's the details of creating a middleware with a proper Context. You must first pull from the current request Context. In the example below, you see ctx := r.Context(). That pulls the current context. Then, you create a NEW context with your additional context key/value. Finally, you return &rye.Response{Context:ctx}

func addContextVar(rw http.ResponseWriter, r *http.Request) *rye.Response {
    // Retrieve the request's context
    ctx := r.Context()

    // Create a NEW context
    ctx = context.WithValue(ctx,"CONTEXT_KEY","my context value")

    // Return that in the Rye response 
    // Rye will add it to the Request to 
    // pass to the next middleware
    return &rye.Response{Context:ctx}
}

Now in a later middleware, you can easily retrieve the value you set!

func getContextVar(rw http.ResponseWriter, r *http.Request) *rye.Response {
    // Retrieving the value is easy!
    myVal := r.Context().Value("CONTEXT_KEY")

    // Log it to the server log?
    log.Infof("Context Value: %v", myVal)

    return nil
}

For another simple example, look in the JWT middleware - it adds the JWT into the context for use by other middlewares. It uses the CONTEXT_JWT key to push the JWT token into the Context.

Using built-in middleware handlers

Rye comes with various pre-built middleware handlers. Pre-built middlewares source (and docs) can be found in the package dir following the pattern middleware_*.go.

To use them, specify the constructor of the middleware as one of the middleware handlers when you define your routes:

// example
routes.Handle("/", middlewareHandler.Handle([]rye.Handler{
    rye.MiddlewareCORS(), // to use the CORS middleware (with defaults)
    a.homeHandler,
})).Methods("GET")

OR 

routes.Handle("/", middlewareHandler.Handle([]rye.Handler{
    rye.NewMiddlewareCORS("*", "GET, POST", "X-Access-Token"), // to use specific config when instantiating the middleware handler
    a.homeHandler,
})).Methods("GET")

Serving Static Files

Rye has the ability to add serving static files in the chain. Two handlers have been provided: StaticFilesystem and StaticFile. These middlewares should always be used at the end of the chain. Their configuration is simply based on an absolute path on the server and possibly a skipped path prefix.

The use case here could be a powerful one. Rye allows you to serve a filesystem just as a whole or a single file. Used together you could facilitate an application which does both -> fulfilling the capability to provide a single page application. For example, if you had a webpack application which served static resources and artifacts, you would use the StaticFilesystem to serve those. Then you'd use StaticFile to serve the single page which refers to the single-page application through index.html.

A full sample is provided in the static-examples folder. Here's a snippet from the example using Gorilla:

    pwd, err := os.Getwd()
    if err != nil {
        log.Fatalf("NewStaticFile: Could not get working directory.")
    }

    routes.PathPrefix("/dist/").Handler(middlewareHandler.Handle([]rye.Handler{
        rye.MiddlewareRouteLogger(),
        rye.NewStaticFilesystem(pwd+"/dist/", "/dist/"),
    }))

    routes.PathPrefix("/ui/").Handler(middlewareHandler.Handle([]rye.Handler{
        rye.MiddlewareRouteLogger(),
        rye.NewStaticFile(pwd + "/dist/index.html"),
    }))

Middleware list

Name	Description
Access Token	Provide Access Token validation
CIDR	Provide request IP whitelisting
CORS	Provide CORS functionality for routes
Auth	Provide Authorization header validation (basic auth, JWT)
Route Logger	Provide basic logging for a specific route
Static File	Provides serving a single file
Static Filesystem	Provides serving a single file

A Note on the JWT Middleware

The JWT Middleware pushes the JWT token onto the Context for use by other middlewares in the chain. This is a convenience that allows any part of your middleware chain quick access to the JWT. Example usage might include a middleware that needs access to your user id or email address stored in the JWT. To access this Context variable, the code is very simple:

func getJWTfromContext(rw http.ResponseWriter, r *http.Request) *rye.Response {
    // Retrieving the value is easy!
    // Just reference the rye.CONTEXT_JWT const as a key
    myVal := r.Context().Value(rye.CONTEXT_JWT)

    // Log it to the server log?
    log.Infof("Context Value: %v", myVal)

    return nil
}

API

Config

This struct is configuration for the MWHandler. It holds references and config to dependencies such as the statsdClient.

type Config struct {
    Statter  statsd.Statter
    StatRate float32
}

MWHandler

This struct is the primary handler container. It holds references to the statsd client.

type MWHandler struct {
    Config Config
}

Constructor

func NewMWHandler(statter statsd.Statter, statrate float32) *MWHandler

Use

This method prepends a global handler for every Handle method you call. Use this multiple times to setup global handlers for every endpoint. Call Use() for each global handler before setting up additional routes.

func (m *MWHandler) Use(handlers Handler)

Handle

This method chains middleware handlers in order and returns a complete http.Handler.

func (m *MWHandler) Handle(handlers []Handler) http.Handler

rye.Response

This struct is utilized by middlewares as a way to share state; ie. a middleware can return a *rye.Response as a way to indicate that further middleware execution should stop (without an error) or return a hard error by setting Err + StatusCode or add to the request Context by returning a non-nil Context.

type Response struct {
    Err           error
    StatusCode    int
    StopExecution bool
    Context       context.Context
}

Handler

This type is used to define an http handler that can be chained using the MWHandler.Handle method. The rye.Response is from the rye package and has facilities to emit StatusCode, bubble up errors and/or stop further middleware execution chain.

type Handler func(w http.ResponseWriter, r *http.Request) *rye.Response

Test stuff

All interfacing with the project is done via make. Targets exist for all primary tasks such as:

• Testing: make test or make testv (for verbosity)
• Generate: make generate - this generates based on vendored libraries (from $GOPATH)
• All (test, build): make all
• .. and a few others. Run make help to see all available targets.
• You can also test the project in Docker (and Codeship) by running jet steps

Contributing

Fork the repository, write a PR and we'll consider it!

Special Thanks

Thanks go out to Justin Reyna (InVisionApp.com) for the awesome logo!