Support AWS Lambda layers

[eladb]

Use Case

I'd like to be able to define a set of AWS Lambda layers that will be loaded into my cloud.Functions:

new cloud.Function(inflight () => {
 // code that depends on some lambda layer!
});

This should also for functions that are implicitly created (e.g. as cloud.Api handlers or used by some method:

class MyClass {
  pub inflight foo() {
    // do something that depends on a lambda layer
  }
}

let c = new MyClass();

let api = new cloud.Api();

api.get("/", inflight () => {
  c.foo();
});

Proposed Solution

No response

Implementation Notes

No response

Component

No response

Community Notes

• Please vote by adding a 👍 reaction to the issue to help us prioritize.
• If you are interested to work on this issue, please leave a comment.
• If this issue is labeled needs-discussion, it means the spec has not been finalized yet. Please reach out on the #dev channel in the Wing Discord.

[eladb]

I'd like to propose that we solve this for now via @meta tags: #6762

[Chriscbr]

We can support this initially without making adding surface area to the language by exposing an appropriate API for cloud.Function, e.g.

let fn = new cloud.Function(inflight () => { ... });
aws.Function.from(fn)?.addLambdaLayer("acme.lambda_layer");

To me this feels more robust from an API design perspective since lambda layers are an AWS Lambda specific concept. If you added some kind of metadata to a block of inflight code, you're now pushing responsibility onto other inflight host types (like cloud.Service etc.) to either understand Lambda layers, or to "reject" inflights which contain metadata it doesn't understand/support.

[eladb]

That's not going to work because there are many functions that are created implicitly. For example, the function that hosts the handler of a cloud.Api or a function created implicitly via the ui.Field, etc.

The decision to take a dependency on the layer is an attribute of the inflight closure code.

[Chriscbr]

That's not going to work because there are many functions that are created implicitly. For example, the function that hosts the handler of a cloud.Api or a function created implicitly via the ui.Field, etc.

While these functions are created implicitly, I don't think they're meant to be hidden. It wouldn't be too hard to provide access to these resources:

let api = new cloud.Api();
api.get("/hello", inflight () => {
  c.foo();
});
let fn = aws.Api.from(api)?.getFn("/hello"); // aws.Function?

We also have platform providers, right?

export class MyPlatform {
  preSynth(app) {
    for (const c of app.node.findAll()) {
      if (c instanceof cloud.Function) {
        c.addLambdaLayer("acme.lambda_layer");
      }
    }
  }
}

I think these mechanisms could be a good incremental step towards addressing the base use case of "I would like to add lambda layers to new and existing cloud.Function resources in my app."

[hasanaburayyan]

We also have platform providers, right?

Platform providers will work well for blanket type rules (i.e. all lambdas need the layer) but if adding a layer had performance implications, then its challenging to to just specify which ones should get it.

[eladb]

let fn = aws.Api.from(api)?.getFn("/hello");

@Chriscbr how would this work if a layer is needed only by some inflight method implemented by a class? (the foo() method in the example in the description).

[eladb]

For the time being, @meta tags will unblock this use case without spiraling into a deep design process. Let's see how this use case evolves and we can "elevate" the experience over time.

[Chriscbr]

@Chriscbr how would this work if a layer is needed only by some inflight method implemented by a class? (the foo() method in the example in the description).

For those kinds of use cases you can use the onLift hook. For example, the platform team or library author would write:

bring aws;

pub class DataDog {
  pub inflight publishMetric() {
    // implementation
  }

  pub onLift(host: std.IInflightHost, ops: Array<str>) {
    if let fn = aws.Function.from(host) {
      fn.addLambdaLayer("datadog-1.2");
    }
  }
}

And then when you are using the class...

bring mylib;

let datadog = new Datadog();

pub class MyResource {
  pub inflight fly() {
    datadog.publishMetric();
  }
}

... and you'd be guaranteed that any cloud.Function created directly, or indirectly through cloud.Api or ui.Field, would have the layer added. No extra syntax needed. Pretty elegant, right? You can even add logic so that if the inflight code is lifted by a non-Lambda host, a compilation error is thrown -- so it's actually a safer abstraction.

[eladb]

Love it! Great solution.

We need to add support for adding lambda layers the aws.Function thingy, no?

And also document this!

[eladb]

Follow ups:

1. What happens if I want only some methods in my class to use a layer?
2. How would that look like for an inflight closure? (not method)

[Chriscbr]

What happens if I want only some methods in my class to use a layer?

For these cases you can add an if-condition to only call addLambdaLayer() when an appropriate method is used:

pub class DataDog {
  // ...
  pub onLift(host: std.IInflightHost, ops: Array<str>) {
    if ops.includes("myMethod") {
      if let fn = aws.Function.from(host) {
        fn.addLambdaLayer("datadog-1.2");
      }
    }
  }
}

How would that look like for an inflight closure? (not method)

The most straightforward way to do this would be to define a helper class once for each lambda layer you want to use, and then you can call it freely in any of your inflight closures.

pub class DatadogLayer {
  pub static inflight load() {}
  pub static onLiftType(host: std.IInflightHost) {
    if let fn = aws.Function.from(host) {
      fn.addLambdaLayer("datadog-1.2");
    }
  }
}

let api = new cloud.Api();
api.get("/hello", inflight () => {
  DatadogLayer.load();
});

The class is about 8 lines of code, but that's pretty small and easy to maintain. The fact that this didn't require any new language capabilities to support is pretty sweet to be honest (and it's a huge win maintenance-wise).

I think the @meta suggestion is cool BTW - though it could seriously benefit from some more bake time / there are several design issues that still have to be worked out.

It's also worth weighing the level of the investment we want to put into this feature against the frequency of the use case. It's the first time the issue of supporting lambda layers has popped up as an issue if I understand correctly. There are also credible sources recommending against use of lambda layers which are worth considering.

[eladb]

The onLift and onLiftType approaches are cool, but they become very cumbersome very quickly and require quite deep understanding on how Wing works. They require platform teams to provide wrappers to these APIs because it's hard to expect devs to use these directly.

I'd like us to go with the @meta approach as an immediate solution for this requirement. I believe it is a relatively simple and powerful low-level mechanism that we can add to the language/framework and we could use it to unblock these types of use cases before they have full support.

[monadabot]

Congrats! 🚀 This was released in Wing 0.75.12.