nbb on AWS Lambda

[vharmain]

nbb on AWS Lambda

I wanted to play with idea of having a simple and easy way to run cljs code on AWS Lambda. Ideally the goal is that it would be possible to write cljs code directly in the Lambda console and run it. Another goal is to use existing battle tested runtime code from AWS as much as possible.

Approach

• Quick & dirty PoC
• Copy Official Lambda Node14 Docker image and see what's in there
• Extend the image by installing nbb via npm
• Patch necessary files to load userland code from a cljs file with nbb API

Files

Dockerfile

FROM public.ecr.aws/lambda/nodejs:14
# Alternatively, you can pull the base image from Docker Hub: amazon/aws-lambda-nodejs:12

# Assumes your function is named "app.js", and there is a package.json file in the app directory 
COPY test.cljs ${LAMBDA_TASK_ROOT}

# Replace original bootstrap file with our own
COPY ./runtime/bootstrap /var/runtime/bootstrap
RUN chmod +x /var/runtime/bootstrap

# Replace original runtime/index.js with our own mjs file
COPY ./runtime/index.mjs /var/runtime/index.mjs

# Install nbb into the runtime
RUN npm --prefix /var/runtime install nbb

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "test.handler" ]

runtime/bootstrap

This file is originally copied from /var/runtime/bootstrap.

The only change is that /var/runtime/index.js file extension was changed to .mjs so we can import nbb.

#!/bin/sh
# Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.

if [ -z "$NODE_PATH" ];
then
  nodejs_mods="/opt/nodejs/node_modules"
  nodejs14_mods="/opt/nodejs/node14/node_modules"
  runtime_mods="/var/runtime/node_modules"
  task="/var/runtime:/var/task"
  export NODE_PATH="$nodejs14_mods:$nodejs_mods:$runtime_mods:$task"
fi

if [ -n "$AWS_LAMBDA_FUNCTION_MEMORY_SIZE" ];
then
  new_space=$(expr $AWS_LAMBDA_FUNCTION_MEMORY_SIZE / 10)
  semi_space=$(expr $new_space / 2)
  old_space=$(expr $AWS_LAMBDA_FUNCTION_MEMORY_SIZE - $new_space)
  MEMORY_ARGS=(
    "--max-semi-space-size=$semi_space"
    "--max-old-space-size=$old_space"
  )
fi

if [[ $AWS_DEFAULT_REGION == us-iso* ]]
then export NODE_EXTRA_CA_CERTS="/etc/pki/tls/certs/ca-bundle.crt"
fi

export AWS_EXECUTION_ENV=AWS_Lambda_nodejs14.x

NODE_ARGS=(
    --expose-gc
    --max-http-header-size 81920
    "${MEMORY_ARGS[@]}"
    /var/runtime/index.mjs
    )

if [ -z "$AWS_LAMBDA_EXEC_WRAPPER" ]; then
  exec /var/lang/bin/node "${NODE_ARGS[@]}"
else
  wrapper="$AWS_LAMBDA_EXEC_WRAPPER"
  if [ ! -f "$wrapper" ]; then
    echo "$wrapper: does not exist"
    exit 127
  fi
  if [ ! -x "$wrapper" ]; then
    echo "$wrapper: is not an executable"
    exit 126
  fi
    exec -- "$wrapper" /var/lang/bin/node "${NODE_ARGS[@]}"
fi

runtime/index.mjs

This file is originally copied from /var/runtime/index.js. Since the original code uses require (cjs) and nbb uses import (mjs) I made a quick hack to support both (createRequire).

Instead of using UserFunction.load to load userland code and resolve the handler function, I use loadFile from nbb API to do the same. Here the file and function names are hardcoded but they could easily be resolved from LAMBDA_TASK_ROOT and _HANDLER env vars.

/**
 * Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 *
 * This module is the bootstrap entrypoint. It establishes the top-level event
 * listeners and loads the user's code.
 */

"use strict";
import { createRequire } from "module";
const require = createRequire(import.meta.url);

import { loadFile } from "nbb";
const path = require("path");

const RAPIDClient = require("./RAPIDClient.js");
const Runtime = require("./Runtime.js");
const UserFunction = require("./UserFunction.js");
const Errors = require("./Errors.js");
const BeforeExitListener = require("./BeforeExitListener.js");

require("./LogPatch").patchConsole();

const client = new RAPIDClient(process.env.AWS_LAMBDA_RUNTIME_API);

let errorCallbacks = {
  uncaughtException: (error) => {
    client.postInitError(error, () => process.exit(129));
  },
  unhandledRejection: (error) => {
    client.postInitError(error, () => process.exit(128));
  },
};

process.on("uncaughtException", (error) => {
  console.error("Uncaught Exception", Errors.toFormatted(error));
  errorCallbacks.uncaughtException(error);
});

process.on("unhandledRejection", (reason, promise) => {
  let error = new Errors.UnhandledPromiseRejection(reason, promise);
  console.error("Unhandled Promise Rejection", Errors.toFormatted(error));
  errorCallbacks.unhandledRejection(error);
});

BeforeExitListener.reset();
process.on("beforeExit", BeforeExitListener.invoke);

// const handler = UserFunction.load(
//   process.env.LAMBDA_TASK_ROOT,
//   process.env._HANDLER
// );

const filePath = path.join(process.env.LAMBDA_TASK_ROOT, "test.cljs");

loadFile(filePath).then((res) => {
  const handler = res["handler"];
  new Runtime(client, handler, errorCallbacks).scheduleIteration();
});

test.cljs

(ns script
  (:require [clojure.string]))

(defn handler [event ctx callback]
  (js/console.log event)
  (callback nil #js{:hello "world"}))

;; Exports
#js {:handler handler}

Build and deploy

I built the Docker image locally and pushed it to a private ecr container repository on my personal AWS account. Then I created a new lambda function from the console and told lambda to use the image from the repo.

Benchmarks

NOTE benchmarks are naive and meant to just provide a "ballpark"

Benchmarks were made from the AWS Console by clicking the "test" button.

NodeJS 14 x86 AWS Default (baseline)

Cold start

Duration: 18.23 ms
Billed Duration: 19 ms
Memory Size: 128 MB
Max Memory Used: 53 MB
Init Duration: 135.35 ms

2nd run

Duration: 11.34 ms
Billed Duration: 12 ms
Memory Size: 128 MB	
Max Memory Used: 53 MB
Init Duration: 171.17 ms

3rd run

Duration: 112.64 ms
Billed Duration: 113 ms
Memory Size: 128 MB
Max Memory Used: 53 MB

Several subsequent runs

Duration: 1.15 ms
Billed Duration: 2 ms
Memory Size: 128 MB
Max Memory Used: 53 MB	

NodeJS 14 x86 Docker nbb as lib, code loaded on init

Cold start

Duration: 63.60 ms
Billed Duration: 724 ms
Memory Size: 128 MB
Max Memory Used: 74 MB
Init Duration: 659.93 ms	

2nd run

Duration: 246.63 ms
Billed Duration: 247 ms
Memory Size: 128 MB
Max Memory Used: 74 MB

3rd run

Duration: 1.65 ms
Billed Duration: 2 ms
Memory Size: 128 MB
Max Memory Used: 74 MB	

Several subsequent runs

Duration: 1.36 ms
Billed Duration: 2 ms
Memory Size: 128 MB
Max Memory Used: 75 MB	

Conclusion

• It works!
• I originally hoped that I could've tricked AWS console to think that this is a NodeJS runtime and get the code editor visible. However I couldn't achieve this

Next steps

• Try to think if there's a convenient/fast way to do iterative cljs development on Lambda. Now the code is baked into the Docker image. Local development is possible with these instructions. Perhaps we could run a nrepl server locally?
• Investigate if init perf can be improved
• Compare with a custom runtime that runs cljs code with the nbb executable
• Think what are the wins over "normal CLJS" compiled to JS and deployed to vanilla NodeJS runtime

[viesti]

I made a trial also, using the nodejs14x runtime: https://github.com/viesti/nbb-lambda

The gist was to load nbb inside the handler function, since the async import that nbb uses seems to make it hard to load the libraty in a "top level" expression in nodejs, in the way that the default Lambda runtime works (seems that it looks for index.js, so misses index.mjs).

I used the Lambda console to then modify the core.cljs code so that it loades the bundled AWS JavaScript library and used functionality from that library. Next thing wouöd be to test the v3 of AWS JS libs, since the nodejs runtimes bundle only v2.

[vharmain]

@borkdude I got mjs to work by forking the default NodeJS runtime and creating my own. In the original post I did it with Docker but it turned out to be un-optimal for quick iterative stuff. For quick & fun I think a custom runtime is a good solution.

I guess loading nbb on top level with the default runtime is hard because it's async and Node supports await on top-level only in most recent version (16?).

[viesti]

Right, so this would be similar as with babashka lambda layer, which also allows to edit code in the Lambda console.

On mjs support on current AWS default Nodejs runtimes: aws/aws-cdk#13274

But I think it would not be bad to have a custom runtime for nbb, could lead to ability to bundle dependencies better even.

[borkdude]

Node supports await on top-level only in most recent version

You can solve this by writing an async function that calls itself on the top level.

[viesti]

I tried to quickly do that, but my JavaScript chops aren't that good :D Can such invocation also set exports?

[vharmain]

Update: this discussion among with other discussions on Clojurians Slack converged into releasing an npm lib that enables rapid cljs development on the vanilla NodeJS lambda runtime: https://github.com/vharmain/nbb-lambda-adapter

We ended up with this approach because it provides low-barrier entry to start hacking. I still think that custom nbb runtime is an interesting area to explore more.