Introduction

WARNING: this project is in its infancy, test hard before using in production

Restana-express is a partial Express compatibility layer for Restana, which implements most of the (req, res) additions over native node.js http(s) server made in Express. It may also work with other frameworks, but this is not guaranteed and may require slight modification.

To achieve this, restana-express imports some methods and properties from Express (if possible) or reimplements them.

Restana-express-compatibility does not aim to offer a 100% compatibility with Express and offers only most of the methods and properties outlined on the following pages:

Express Res

Express Req

Despite this, testing is mostly done with modified Express tests. However, please see the table in Compatibility section

Version & Changes

Please see CHANGELOG.md

License

MIT License

Usage

Installation

npm i --save restana-express-compatibility

Setting up the restana and middleware

	const restana = require('restana')
	let restanaExpressCompatibilityMod = require('restana-express-compatibility')
	let compatibilityLayerSettings = {
		res: {
			toUse: ['all'], // you can specify which res components you want. However, do take note that you may run into issues, if the component you use depends on another one, which you haven't specified. Check dependencies table in Observations and known problems
			toDisable: [], // Array of methods and properties to disable. Overrides 'toUse'
			render: {  // res.render was completely reimplemented and is now setup differently
				viewsDir: path.resolve(__dirname + "/views/"),
				renderExt: '.pug',
				renderEngine: 'pug',
				renderFunction: "__express"
			}
		},
		req: {
			toUse: ['all'], // you can specify which req components you want. However, do take note that you may run into issues, if the component you use depends on another one, which you haven't specified. Check dependencies table in Observations and known problems 
			toDisable: [], // Array of methods and properties to disable. Overrides 'toUse'
			proxyTrust: true, // express proxyTrust-related functions have been reimplemented, so it is setup here. Accepts Number, function, String or Array of IPs, Boolean (see Express proxy settings)
			queryParser: true, // true, "simple" uses restana's url query parser, "extended" or undefined enables Express's extended query parser (Check express docs for explanation)
			subdomainsOffset: 0, // defaults to 2
			// OR
			subdomainOffset: 0, // subdomainsOffset is still prioritized
			propertiesAsFunctions: true, // default - undefined. False to disable, true to enable
			etag: {
				type: 'tiny', // default - tiny, true enables Express's weak ETag, otherwise uses the same options as Express (see - https://expressjs.com/en/api.html#etag.options.table)
				seed: 01234567890, // Number, for use with Tiny ETag. Should be syncronised between all instances of your server on all machines, otherwise useless, since all ETags will be different. However, will generate a pseudo-random number by default instead.
				maxSize: 1000 // Used with Tiny ETag, size of internal cache (number of entries). 1000 is the default. You should probably set closer towards the number of resources available on your service. 
			}
		}
	}
	
	var app = restana();

	let restanaExpressCompatibility = new restanaExpressCompatibilityMod(compatibilityLayerSettings)

	app.use(restanaExpressCompatibility.middleware)

	app.get('/hi/', (req, res) => {
		res.send({
			msg: 'Hello World!',
			query: req.query,
			subdomains: req.subdomains,
			ip: req.ip,
		})
	})

	let server = app.start(3002, '0.0.0.0')
	  

Compatibility

Res

Properties

Property or object	Implemented?	Notes
locals	Yes	N/A
headersSent	Native	Doesn't need reimplementation
app	No	Probably not going to be implemented

Methods

Method or object	Implemented?	Notes
append	Yes	N/A
attachment	Yes	N/A
cookie	Yes	cookieParser required for signed cookies, check req.cookie entry in req properties
clearCookie	Yes	cookieParser required for signed cookies, check req.cookie entry in req properties
download	Yes	N/A
end	Native	Doesn't need reimplementation
get	Yes	N/A
json and jsonp	Partial	Reimplemented, but very crudely. It only JSON stringifies the object supplied. Needs to be rewritten
links	Yes	N/A
location	Yes	N/A
redirect	Yes	N/A
render	Rewritten from scratch	Rewritten from scratch, compatible with using only one render engine. Workarounds might be possible. Please read section on res.render
send	Partial	Slightly modifies restana's res.send to make it respect change of http status code through res.status(code) or res.statusCode = code
sendFile	Partial	Not sufficiently covered by tests, not all Express tests are ported. Likely works just fine
sendStatus	Yes	N/A
set / header	Yes	N/A
status	Yes	N/A
type / contentType	Yes	N/A
vary	Yes	N/A
format	Yes	Not sufficiently covered by tests

res.render

restana-express does not import res.render, because it depends too much on express core. Therefore, this project partially reimplements the res.render itself.

Limitations

• Works only with one render engine, you can't specify many (workarounds might be possible by initializing this middleware several times differently in various places of your project)
• No express automagic :(

Settings

let compatibilityLayerSettings = {
	res: {
		toUse: ['all'], // should either be an Array with 'all' or at least include 'render'
		render: {  // res.render was completely reimplemented and is now setup differently
			viewsDir: path.resolve(__dirname + "/views/"), // ESSENTIAL parameter, without it res.render will send an error
			renderExt: '.pug', // essential, if you use something besides Pug. Dot at the beginning is needed. Defaults to '.pug'
			renderEngine: 'pug', // name of engine. restana-express will require it. Defaults to 'pug'
			renderFunction: "__express" // Internal function that works with express. Usually you should leave it unchanged. Defaults to '__express'
		}
	},
	...
}

Req

Properties

Property or object	Implemented?	Notes
app	No	Probably not going to be implemented
baseUrl	No	Help is welcome
headersSent	Native	Doesn't need reimplementation
body	No	You may use body-parser middleware, since this is what express uses internally. You can app.use (or abuse :D) it either before or immediately after restana-express
cookies	No, not in restana-express itself. Please, see notes	You should use cookie-parser middleware, as this property is not implemented in Express itself.
fresh	Yes	N/A
hostname	Yes	N/A
ip	Yes	N/A
ips	Yes	N/A
method	Native	Does not need reimplementation
originalUrl	Native	Does not need reimplementation
path	Native	Does not need reimplementation
protocol	Yes	N/A
query	Yes	Set up the type of parser through compatibilityLayerSettings.req.queryParser, before initializing and app.use'ing the middleware. Accepts same values as express for this setting
hostname	Yes	N/A
route	No	Help is welcome
secure	Yes	N/A
signedCookies	No, not in restana-express itself. Please, see notes	Check entry in this table on req.cookies
stale	Yes	N/A
subdomains	Yes	subdomain offset is set either through compatibilityLayerSettings.req.subdomainsOffset OR compatibilityLayerSettings.req.subdomainOffset
xhr	Yes	N/A

Methods

Method or object	Implemented?	Notes
accepts	Yes	N/A
acceptsCharsets	Yes	N/A
acceptsEncodings	Yes	N/A
acceptsLanguages	Yes	N/A
get / header	Yes	N/A
is	Yes	N/A
param	Yes	Deprecated by express. Please note that you need body-parser middleware
range	Yes	N/A

Serve static

For serving static content, see this article: https://thejs701816742.wordpress.com/2019/07/12/restana-static-serving-the-frontend-with-node-js-beyond-nginx/

It is recommended that you limit these middleware to some routes, while restana-express-compatibility to other routes.

Observations and known problems

Tests

Tests currently cover only the default options, at which restana-express-compatibility is the most compatible with Express, but also the slowest. Help is welcome.

Restana-express-compatibility and cookie- and body-parser

In order for this middleware to work correctly for routes or middleware that were designed for Express, you need to speicfy and app.use it before them. Do take note, though, that you need to enable cookie-parser middleware before restana-express, if you plan to use req.secureCookies and send secure cookies:

var cookieParser = require('cookie-parser');

...
app.use(cookieParser(process.env.COOKIE_PARSER_SECRET || "wonderland"));
app.use(restanaExpressCompatibility.middleware)

The same applies to body-parser:

var cookieParser = require('cookie-parser');
const bodyParser = require('body-parser')


...
// parse application/json
app.use(bodyParser.json())
app.use(cookieParser(process.env.COOKIE_PARSER_SECRET || "wonderland"));
app.use(restanaExpressCompatibility.middleware)

Performance

With the default settings, restana-express-compatibility is highly compatible with express. This compatibility comes at a great cost - the performance is higher than with express, but not THAT much higher (only by up to 60%, depending on task). In other words, restana-express-compatibility can handle 16-17k requests per second instead of 10-11k with express with /json/ route (see benchmarks section)

Here are the tips to increase performance:

Tip 1 - Disable req properties, if you don't use them [EASY]

Some express req properties are very convenient at times. These include:

	['protocol', 'ip', 'ips', 'hostname', 'subdomains', 'fresh', "stale", 'xhr', 'secure'] // the list is exhaustive

However, before continuing with your application logic, each of them has to be processed, EVEN IF you don't end up using them. Internally, there are functions, like req.protocolPropFn which is run in the middleware, before carrying on to your routes. In this example, req.protocolPropFn populates req.protocol. In the end, it means that middleware is forced to traverse through these functions and then run each one that is enabled.

As such, if performance is vital, you can either disable some properties (very modest performance increase) OR disable them altogether (great speed increase). Alternatively, you can turn them into functions instead:

let compatibilityLayerSettings = {
	req: {
		propertiesAsFunctions: true,
	}
}

This way, whenever you need to know the protocol, instead of accessing req.protocol property, you'll need to call req.protocol() function. No arguments are required. This measure will boost restana-express-compatibility from handling 16-17k req/s to handling 21-23 req/s, which is a further 35% speed increase. You should be able to easily replace calls to req properties with calls to the new functions with your IDE or shell commands.

Tip 2 - Don't enable extended query parsing or weak or strong ETag [MEDIUM]

By default, restana-express-compatibility uses a different implementation of ETag, which should work just fine. Don't change the ETag setting, unless you know what you are doing. The same applies to query parsing - by default, Express.JS extended query parsing is used for Express compatibility reasons. We advise you to disable it and use the simple parser, as it is built-in, unless you REALLY need the extended parser. But in any case, you should avoid passing data as a query due to its inherent limitations.

You can also consider benchmarking with ETag disabled on specific routes with setting res.locals.NO_ETAG = true in your route or middleware. Beware that it may affect caching, if it is used.

Tip 3 - disable what you don't need [HARD]

By disabling things you don't need, you make the server use less RAM, which is beneficial in the long term. Additionally, the middleware merges the req and res objects with objects containing functions and properties taken from Express each time a route after compatibility middleware is requested by a client. This means that for every unnecessary method or property activated, the server spends more time merging the object with Object.assign.

For instance, with no methods or properties activated, the merge takes only ~5000 nanoseconds. However, with all methods or properties, it takes ~60000 nanoseconds. That is more than a third of the total middleware working time (with req.properties converted into functions). This means that keeping bare minimum of methods and properties you need will make the middleware work faster. Therefore, your own code will execute faster.

For your convenience, there are dependency tables provided after the tips. Take note that number of dependencies may be reduced in the next minor release.

Tip 4 - gradually stop using this module [POSSIBLY HARD]

By disabling this module and finding alternatives for your needs (if possible), you are going to have a much faster application. It is highly likely you do not need all of the methods and properties offered by express and only need a subset. Find appropriate middleware for your needs or reimplement the required methods to suit your needs. Your app performance is gonna thank you. Moreover, you can limit the middleware only to specific routes, so that it is not run, where it is not needed.

Dependency tables

Please take note that these tables serve as an indication only. It is highly possible some mistakes could have been made, especially with Dependents column

Res properties

Property or object	Can be disabled?	Dependencies
locals	No	None

Res methods

Method or object	Can be disabled?	Dependencies	Dependents
send	No	None	Irrelevant
append	Yes	res.get, res.set	res.cookie
attachment	Yes	res.type, res.set	None
cookie	Yes	res.append, cookie-parse middleware (for signed cookies, see Compatibility and Observations and known problems sections )	res.clearCookie
clearCookie	Yes	res.cookie and its dependencies	None
download	Yes	res.sendFile	None
end	No, native	None	Irrelevant
get	Yes	None	res.append, REQ.fresh, REQ.hostname
json and jsonp	Yes	None	res.render
links	Yes	res.set	None
location	Yes	REQ.get && res.set	res.redirect
redirect	Yes	res.location, res.format, res.set	None
render	Yes	res.status, res.json, res.set	None
sendFile	Yes	None	None
sendStatus	Yes	None	None
set / header	Yes	None	res.append, res.attachment, res.links, res.location, res.redirect, res.render, res.type/res.contentType, res.format
status	Yes	None	res.render
type / contentType	Yes	res.set	res.attachment
vary	Yes	None	res.format
format	Yes	res.vary, res.set	res.redirect

Req properties

Property or object	Can be disabled?	Dependencies	Dependents
headersSent	No, Native	None	Irrelevant
fresh	Yes	RES.get	req.stale, RES.send (soft-fail, ETag functionality gets disabled)
ip	Yes	None	None
ips	Yes	None	None
method	No, Native	None	Irrelevant
originalUrl	No, Native	None	Irrelevant
params	No, Native	None	Irrelevant
path	No, Native	None	Irrelevant
protocol	Yes	req.get	req.secure
query	No	None	Irrelevant
hostname	Yes	req.get	req.subdomains
secure	Yes	req.protocol	None
stale	Yes	req.fresh	None
subdomains	Yes	req.hostname	None
xhr	Yes	req.get	None

Req methods

Method or object	Can be disabled?	Dependencies	Dependents
accepts	Yes	None	None
acceptsCharsets	Yes	None	None
acceptsEncodings	Yes	None	None
acceptsLanguages	Yes	None	None
get / header	Yes	None	req.range, RES.location, req.protocol, req.hostname, req.xhr
is	Yes	None	None
param	Yes	req.params, req.body, req.query	None
range	Yes	req.get	None

Benchmarks (not scientific)

Route setup

The following route is used:

app.get('/hi/', async (req, res) => {
	res.send({
	  msg: 'Hello World!',
	  query: req.query

	})
})

For restana without restana-express, the following middleware is added:

app.use((req,res,next) => {
	res.json = function(inp) {
		res.setHeader('Content-Type', 'application/json');
		if (inp == null) return res.send(null)
		return res.send(JSON.stringify(inp))
	}
	return next()
})

Tool used

node ./performance/APP.js
sleep 5
wrk -t8 -c64 -d5s http://localhost:PORT/hi/

, where PORT and APP are changed to:

• 3001 and restana for restana
• 3002 and restanaExpressHighPerf for restana + restana-express (performance conscious, query-parser set to simple, req.properties turned into methods)
• 3003 and restanaExpress for restana + restana-express (default)
• 3004 and express for express

Hardware used

MacBook Pro 2019, 2,4 GHz Intel Core i9, 64 GB 2667 MHz DDR4

Results

Server	Result, req/s	Gains over express
Restana	48866.22	315%
Restana + restana-express (performance conscious)	23014.65	96%
restana + restana-express (default)	19087.31	62%
Express	11752.61	0%