GitHub - awslabs/llrt: LLRT (Low Latency Runtime) is an experimental, lightweight JavaScript runtime designed to address the growing demand for fast and efficient Serverless applications.

LLRT (Low Latency Runtime) is a lightweight JavaScript runtime designed to address the growing demand for fast and efficient Serverless applications. LLRT offers up to over 10x faster startup and up to 2x overall lower cost compared to other JavaScript runtimes running on AWS Lambda

It's built in Rust, utilizing QuickJS as JavaScript engine, ensuring efficient memory usage and swift startup.

Warning

LLRT is an experimental package. It is subject to change and intended only for evaluation purposes.

_{LLRT - DynamoDB Put, ARM, 128MB:}

_{Node.js 20 - DynamoDB Put, ARM, 128MB:}

HTTP benchmarks measured in round trip time for a cold start (why?)

Configure Lambda functions to use LLRT

Download the last LLRT release from https://github.com/awslabs/llrt/releases

Option 1: Custom runtime (recommended)

Choose Custom Runtime on Amazon Linux 2023 and package the LLRT bootstrap binary together with your JS code.

Option 2: Use a layer

Choose Custom Runtime on Amazon Linux 2023, upload llrt-lambda-arm64.zip or llrt-lambda-x64.zip as a layer and add to your function

Option 3: Package LLRT in a container image

See our AWS SAM example or:

FROM --platform=arm64 busybox
WORKDIR /var/task/
COPY app.mjs ./
ADD https://github.com/awslabs/llrt/releases/latest/download/llrt-container-arm64 /usr/bin/llrt
RUN chmod +x /usr/bin/llrt

ENV LAMBDA_HANDLER "app.handler"

CMD [ "llrt" ]

Option 4: AWS SAM

The following example project sets up a lambda instrumented with a layer containing the llrt runtime.

Option 5: AWS CDK

You can use cdk-lambda-llrt construct library to deploy LLRT Lambda functions with AWS CDK.

import { LlrtFunction } from "cdk-lambda-llrt";

const handler = new LlrtFunction(this, "Handler", {
  entry: "lambda/index.ts",
});

See Construct Hub and its examples for more details.

That's it 🎉

Important

Even though LLRT supports ES2023 it's NOT a drop in replacement for Node.js. Consult Compatibility matrix and API for more details. All dependencies should be bundled for a browser platform and mark included @aws-sdk packages as external.

Testing & ensuring compatibility

The best way to ensure your code is compatible with LLRT is to write tests and execute them using the built-in test runner. The test runner currently supports Jest/Chai assertions. There are two main types of tests you can create:

Unit Tests

Useful for validating specific modules and functions in isolation
Allow focused testing of individual components

End-to-End (E2E) Tests

Validate overall compatibility with AWS SDK and WinterCG compliance
Test the integration between all components
Confirm expected behavior from end-user perspective

For more information about the E2E Tests and how to run them, see here.

Test runner

Test runner uses a lightweight Jest-like API and supports Jest/Chai assertions. For examples on how to implement tests for LLRT see the /tests folder of this repository.

To run tests, execute the llrt test command. LLRT scans the current directory and sub-directories for files that ends with *.test.js or *.test.mjs. You can also provide a specific test directory to scan by using the llrt test -d <directory> option.

The test runner also has support for filters. Using filters is as simple as adding additional command line arguments, i.e: llrt test crypto will only run tests that match the filename containing crypto.

Compatibility matrix

Note

LLRT only support a fraction of the Node.js APIs. It is NOT a drop in replacement for Node.js, nor will it ever be. Below is a high level overview of partially supported APIs and modules. For more details consult the API documentation

Modules	Node.js	LLRT ⚠️
assert	✔︎	✔︎️
buffer	✔︎	✔︎️
child_process	✔︎	✔︎⏱
console	✔︎	✔︎
crypto	✔︎	✔︎
events	✔︎	✔︎
fs/promises	✔︎	✔︎
fs	✔︎	✘⏱
http	✔︎	✘⏱**
https	✔︎	✘⏱**
net:sockets	✔︎	✔︎⏱
net:server	✔︎	✔︎
os	✔︎	✔︎
path	✔︎	✔︎
perf_hooks	✔︎	✔︎
process	✔︎	✔︎
streams	✔︎	✔︎*
timers	✔︎	✔︎
url	✔︎	✔︎
tls	✔︎	✘⏱
zlib	✔︎	✔︎
Other modules	✔︎	✘

Features	Node.js	LLRT ⚠️
async/await	✔︎	✔︎
encoding	✔︎	✔︎
fetch	✔︎	✔︎
ESM	✔︎	✔︎
CJS	✔︎	✔︎

⚠️ = partially supported in LLRT
⏱ = planned partial support
* = Not native
** = Use fetch instead

Using node_modules (dependencies) with LLRT

Since LLRT is meant for performance critical application it's not recommended to deploy node_modules without bundling, minification and tree-shaking.

LLRT can work with any bundler of your choice. Below are some configurations for popular bundlers:

Warning

LLRT implements native modules that are largely compatible with the following external packages. By implementing the following conversions in the bundler's alias function, your application may be faster, but we recommend that you test thoroughly as they are not fully compatible.

Node.js	LLRT
fast-xml-parser	llrt:xml
uuid	llrt:uuid

ESBuild

esbuild index.js --platform=browser --target=es2023 --format=esm --bundle --minify --external:@aws-sdk --external:@smithy

Rollup

import resolve from "@rollup/plugin-node-resolve";
import commonjs from "@rollup/plugin-commonjs";
import terser from "@rollup/plugin-terser";

export default {
  input: "index.js",
  output: {
    file: "dist/bundle.js",
    format: "esm",
    sourcemap: true,
    target: "es2023",
  },
  plugins: [resolve(), commonjs(), terser()],
  external: ["@aws-sdk", "@smithy"],
};

Webpack

import TerserPlugin from "terser-webpack-plugin";
import nodeExternals from "webpack-node-externals";

export default {
  entry: "./index.js",
  output: {
    path: "dist",
    filename: "bundle.js",
    libraryTarget: "module",
  },
  target: "web",
  mode: "production",
  resolve: {
    extensions: [".js"],
  },
  externals: [nodeExternals(), "@aws-sdk", "@smithy"],
  optimization: {
    minimize: true,
    minimizer: [
      new TerserPlugin({
        terserOptions: {
          ecma: 2023,
        },
      }),
    ],
  },
};

Using AWS SDK (v3) with LLRT

LLRT includes many AWS SDK clients and utils as part of the runtime, built into the executable. These SDK Clients have been specifically fine-tuned to offer best performance while not compromising on compatibility. LLRT replaces some JavaScript dependencies used by the AWS SDK by native ones such as Hash calculations and XML parsing. V3 SDK packages not included in the list below have to be bundled with your source code. For an example on how to use a non-included SDK, see this example build script (buildExternalSdkFunction)

LLRT supports the following three bundles by default. Bundle types and suffixes are as follows.

Bundle Type	Suffix	Purpose of Use
no-sdk	*-no-sdk	Suitable for workloads that do not use `@aws-sdk`.
std-sdk	(none)	Suitable for workloads that utilize the major `@aws-sdk`.
full-sdk	*-full-sdk	Suitable for workloads that utilize any `@aws-sdk`.

The relationship between the supported packages for each bundle type is as follows.

Analytics	no-sdk	std-sdk	full-sdk
@aws-sdk/client-athena			✔︎
@aws-sdk/client-firehose			✔︎
@aws-sdk/client-glue			✔︎
@aws-sdk/client-kinesis			✔︎
@aws-sdk/client-opensearch			✔︎
@aws-sdk/client-opensearchserverless			✔︎

Application integration	std-sdk	full-sdk
@aws-sdk/client-eventbridge	✔︎	✔︎
@aws-sdk/client-scheduler		✔︎
@aws-sdk/client-sfn	✔︎	✔︎
@aws-sdk/client-sns	✔︎	✔︎
@aws-sdk/client-sqs	✔︎	✔︎

Business applications	no-sdk	std-sdk	full-sdk
@aws-sdk/client-ses		✔︎	✔︎
@aws-sdk/client-sesv2			✔︎

Compute services	no-sdk	std-sdk	full-sdk
@aws-sdk/client-auto-scaling			✔︎
@aws-sdk/client-batch			✔︎
@aws-sdk/client-ec2			✔︎
@aws-sdk/client-lambda			✔︎

Containers	no-sdk	std-sdk	full-sdk
@aws-sdk/client-ecr			✔︎
@aws-sdk/client-ecs			✔︎
@aws-sdk/client-eks			✔︎
@aws-sdk/client-servicediscovery			✔︎

Databases	std-sdk	full-sdk
@aws-sdk/client-dynamodb	✔︎	✔︎
@aws-sdk/client-dynamodb-streams		✔︎
@aws-sdk/client-elasticache		✔︎
@aws-sdk/client-rds		✔︎
@aws-sdk/client-rds-data		✔︎

Developer tools	no-sdk	std-sdk	full-sdk
@aws-sdk/client-xray		✔︎	✔︎

Front-end web and mobile services	no-sdk	std-sdk	full-sdk
@aws-sdk/client-amplify			✔︎
@aws-sdk/client-appsync			✔︎
@aws-sdk/client-location			✔︎

Machine Learning (ML) and Artificial Intelligence (AI)	no-sdk	std-sdk	full-sdk
@aws-sdk/client-bedrock			✔︎
@aws-sdk/client-bedrock-runtime			✔︎
@aws-sdk/client-bedrock-agent			✔︎
@aws-sdk/client-bedrock-agent-runtime			✔︎
@aws-sdk/client-polly			✔︎
@aws-sdk/client-rekognition			✔︎
@aws-sdk/client-textract			✔︎
@aws-sdk/client-translate			✔︎

Management and governance	std-sdk	full-sdk
@aws-sdk/client-appconfig		✔︎
@aws-sdk/client-appconfigdata		✔︎
@aws-sdk/client-cloudformation		✔︎
@aws-sdk/client-cloudwatch		✔︎
@aws-sdk/client-cloudwatch-events	✔︎	✔︎
@aws-sdk/client-cloudwatch-logs	✔︎	✔︎
@aws-sdk/client-service-catalog		✔︎
@aws-sdk/client-ssm	✔︎	✔︎

Media	no-sdk	std-sdk	full-sdk
@aws-sdk/client-mediaconvert			✔︎

Networking and content delivery	no-sdk	std-sdk	full-sdk
@aws-sdk/client-api-gateway			✔︎
@aws-sdk/client-apigatewayv2			✔︎
@aws-sdk/client-elastic-load-balancing-v2			✔︎

Security, identity, and compliance	std-sdk	full-sdk
@aws-sdk/client-acm		✔︎
@aws-sdk/client-cognito-identity	✔︎	✔︎
@aws-sdk/client-cognito-identity-provider	✔︎	✔︎
@aws-sdk/client-iam		✔︎
@aws-sdk/client-kms	✔︎	✔︎
@aws-sdk/client-secrets-manager	✔︎	✔︎
@aws-sdk/client-sso		✔︎
@aws-sdk/client-sso-admin		✔︎
@aws-sdk/client-sso-oidc		✔︎
@aws-sdk/client-sts	✔︎	✔︎

Storage	no-sdk	std-sdk	full-sdk
@aws-sdk/client-efs			✔︎
@aws-sdk/client-s3		✔︎	✔︎

Other bundled packages	std-sdk	full-sdk
@aws-crypto	✔︎	✔︎
@aws-sdk/credential-providers	✔︎	✔︎
@aws-sdk/lib-dynamodb	✔︎	✔︎
@aws-sdk/lib-storage	✔︎	✔︎
@aws-sdk/s3-presigned-post	✔︎	✔︎
@aws-sdk/s3-request-presigner	✔︎	✔︎
@aws-sdk/util-dynamodb	✔︎	✔︎
@aws-sdk/util-user-agent-browser	✔︎	✔︎
@smithy	✔︎	✔︎

Important

LLRT currently does not support returning streams from SDK responses. Use response.Body.transformToString(); or response.Body.transformToByteArray(); as shown below.

const response = await client.send(command);
// or 'transformToByteArray()'
const str = await response.Body.transformToString();

Running TypeScript with LLRT

Same principle as dependencies applies when using TypeScript. TypeScript must be bundled and transpiled into ES2023 JavaScript.

Note

LLRT will not support running TypeScript without transpilation. This is by design for performance reasons. Transpiling requires CPU and memory that adds latency and cost during execution. This can be avoided if done ahead of time during deployment.

Rationale

What justifies the introduction of another JavaScript runtime in light of existing options such as Node.js, Bun & Deno?

Node.js, Bun, and Deno represent highly proficient JavaScript runtimes. However, they are designed with general-purpose applications in mind. These runtimes were not specifically tailored for the demands of a Serverless environment, characterized by short-lived runtime instances. They each depend on a (Just-In-Time compiler (JIT) for dynamic code compilation and optimization during execution. While JIT compilation offers substantial long-term performance advantages, it carries a computational and memory overhead.

In contrast, LLRT distinguishes itself by not incorporating a JIT compiler, a strategic decision that yields two significant advantages:

A) JIT compilation is a notably sophisticated technological component, introducing increased system complexity and contributing substantially to the runtime's overall size.

B) Without the JIT overhead, LLRT conserves both CPU and memory resources that can be more efficiently allocated to code execution tasks, thereby reducing application startup times.

Limitations

There are many cases where LLRT shows notable performance drawbacks compared with JIT-powered runtimes, such as large data processing, Monte Carlo simulations or performing tasks with hundreds of thousands or millions of iterations. LLRT is most effective when applied to smaller Serverless functions dedicated to tasks such as data transformation, real time processing, AWS service integrations, authorization, validation etc. It is designed to complement existing components rather than serve as a comprehensive replacement for everything. Notably, given its supported APIs are based on Node.js specification, transitioning back to alternative solutions requires minimal code adjustments.

Building from source

Clone code and cd to directory

git clone [email protected]:awslabs/llrt.git
cd llrt

Install git submodules

git submodule update --init --checkout

Install rust

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | bash -s -- -y
source "$HOME/.cargo/env"

Install dependencies

# MacOS
brew install zig make cmake zstd node corepack

# Ubuntu
sudo apt -y install make zstd
sudo snap install zig --classic --beta

# Windows WSL2
sudo apt -y install cmake g++ gcc make zip zstd
sudo snap install zig --classic --beta

# Windows WSL2 (If Node.js is not yet installed)
sudo curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/master/install.sh | bash
nvm install --lts

Install Node.js packages

corepack enable
yarn

Install generate libs and setup rust targets & toolchains

make stdlib && make libs

Note

If these commands exit with an error that says can't cd to zstd/lib, you've not cloned this repository recursively. Run git submodule update --init to download the submodules and run the commands above again.

Build binaries for Lambda (Per bundle type and architecture desired)

# for arm64, use
make llrt-lambda-arm64.zip
make llrt-lambda-arm64-no-sdk.zip
make llrt-lambda-arm64-full-sdk.zip
# or for x86-64, use
make llrt-lambda-x64.zip
make llrt-lambda-x64-no-sdk.zip
make llrt-lambda-x64-full-sdk.zip

Build binaries for Container (Per bundle type and architecture desired)

# for arm64, use
make llrt-container-arm64
make llrt-container-arm64-no-sdk
make llrt-container-arm64-full-sdk
# or for x86-64, use
make llrt-container-x64
make llrt-container-x64-no-sdk
make llrt-container-x64-full-sdk

Optionally build for your local machine (Mac or Linux)

make release
make release-no-sdk
make release-full-sdk

You should now have a llrt-lambda-arm64*.zip or llrt-lambda-x64*.zip. You can manually upload this as a Lambda layer or use it via your Infrastructure-as-code pipeline

Running Lambda emulator

Please note that in order to run the example you will need:

Valid AWS credentials via a ~/.aws/credentials or via environment variables.

export AWS_ACCESS_KEY_ID=XXX
export AWS_SECRET_ACCESS_KEY=YYY
export AWS_REGION=us-east-1

A DynamoDB table (with id as the partition key) on us-east-1
The dynamodb:PutItem IAM permission on this table. You can use this policy (don't forget to modify <YOUR_ACCOUNT_ID>):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "putItem",
      "Effect": "Allow",
      "Action": "dynamodb:PutItem",
      "Resource": "arn:aws:dynamodb:us-east-1:<YOUR_ACCOUNT_ID>:table/quickjs-table"
    }
  ]
}

Start the lambda-server.js in a separate terminal

node lambda-server.js

Then run llrt:

make run

Environment Variables

`LLRT_EXTRA_CA_CERTS=file`

Load extra certificate authorities from a PEM encoded file

`LLRT_GC_THRESHOLD_MB=value`

Set a memory threshold in MB for garbage collection. Default threshold is 20MB

`LLRT_HTTP_VERSION=value`

Restrict HTTP requests to use a specific version. By default HTTP 1.1 and 2 are enabled. Set this variable to 1.1 to only use HTTP 1.1

`LLRT_LOG=[target][=][level][,...]`

Filter the log output by target module, level, or both (using =). Log levels are case-insensitive and will also enable any higher priority logs.

Log levels in descending priority order:

Error
Warn | Warning
Info
Debug
Trace

Example filters:

warn will enable all warning and error logs
llrt_core::vm=trace will enable all logs in the llrt_core::vm module
warn,llrt_core::vm=trace will enable all logs in the llrt_core::vm module and all warning and error logs in other modules

`LLRT_NET_ALLOW="host[ ...]"`

Space-delimited list of hosts or socket paths which should be allowed for network connections. Network connections will be denied for any host or socket path missing from this list. Set an empty list to deny all connections

`LLRT_NET_DENY="host[ ...]"`

Space-delimited list of hosts or socket paths which should be denied for network connections

`LLRT_NET_POOL_IDLE_TIMEOUT=value`

Set a timeout in seconds for idle sockets being kept-alive. Default timeout is 15 seconds

`LLRT_PLATFORM=value`

Used to explicitly specify a preferred platform for the Node.js package resolver. The default is browser. If node is specified, "node" takes precedence in the search path. If a value other than browser or node is specified, it will behave as if "browser" was specified.

`LLRT_TLS_VERSION=value`

Set the TLS version to be used for network connections. By default only TLS 1.2 is enabled. TLS 1.3 can also be enabled by setting this variable to 1.3

Benchmark Methodology

Although Init Duration reported by Lambda is commonly used to understand cold start impact on overall request latency, this metric does not include the time needed to copy code into the Lambda sandbox.

The technical definition of Init Duration (source):

For the first request served, the amount of time it took the runtime to load the function and run code outside of the handler method.

Measuring round-trip request duration provides a more complete picture of user facing cold-start latency.

Lambda invocation results (λ-labeled row) report the sum total of Init Duration + Function Duration.

Security

See CONTRIBUTING for more information.

License

This library is licensed under the Apache-2.0 License. See the LICENSE file.

Name		Name	Last commit message	Last commit date
Latest commit History 576 Commits
.cargo		.cargo
.fleet		.fleet
.github		.github
.vscode		.vscode
benchmarks		benchmarks
example		example
fixtures		fixtures
libs		libs
linker		linker
llrt		llrt
llrt_core		llrt_core
llrt_modules		llrt_modules
modules		modules
shims		shims
tests		tests
types		types
zstd @ 794ea1b		zstd @ 794ea1b
.gitignore		.gitignore
.gitmodules		.gitmodules
.prettierignore		.prettierignore
.yarnrc.yml		.yarnrc.yml
API.md		API.md
CHANGELOG.md		CHANGELOG.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
GOVERNANCE.md		GOVERNANCE.md
LICENSE		LICENSE
MAINTAINERS.md		MAINTAINERS.md
Makefile		Makefile
NOTICE		NOTICE
README.md		README.md
THIRD_PARTY_LICENSES		THIRD_PARTY_LICENSES
build.mjs		build.mjs
index.mjs		index.mjs
lambda-server.js		lambda-server.js
pack		pack
package.json		package.json
rustfmt.toml		rustfmt.toml
sdk.cfg		sdk.cfg
tsconfig.json		tsconfig.json
vitest.config.mjs		vitest.config.mjs
yarn.lock		yarn.lock

License

awslabs/llrt

Folders and files

Latest commit

History

Repository files navigation

Configure Lambda functions to use LLRT

Option 1: Custom runtime (recommended)

Option 2: Use a layer

Option 3: Package LLRT in a container image

Option 4: AWS SAM

Option 5: AWS CDK

Testing & ensuring compatibility

Test runner

Compatibility matrix

Using node_modules (dependencies) with LLRT

ESBuild

Rollup

Webpack

Using AWS SDK (v3) with LLRT

Running TypeScript with LLRT

Rationale

Limitations

Building from source

Running Lambda emulator

Environment Variables

LLRT_EXTRA_CA_CERTS=file

LLRT_GC_THRESHOLD_MB=value

LLRT_HTTP_VERSION=value

LLRT_LOG=[target][=][level][,...]

LLRT_NET_ALLOW="host[ ...]"

LLRT_NET_DENY="host[ ...]"

LLRT_NET_POOL_IDLE_TIMEOUT=value

LLRT_PLATFORM=value

LLRT_TLS_VERSION=value

Benchmark Methodology

Security

License

About

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases 20

Packages 0

Contributors 46

Languages

`LLRT_EXTRA_CA_CERTS=file`

`LLRT_GC_THRESHOLD_MB=value`

`LLRT_HTTP_VERSION=value`

`LLRT_LOG=[target][=][level][,...]`

`LLRT_NET_ALLOW="host[ ...]"`

`LLRT_NET_DENY="host[ ...]"`

`LLRT_NET_POOL_IDLE_TIMEOUT=value`

`LLRT_PLATFORM=value`

`LLRT_TLS_VERSION=value`

Packages