Cloud Assembly Specification

[RomainMuller]

In order to confidently design a CI/CD deployment experience that supports assets and multiple stack ordering constraints, the Cloud Assembly document format needs to be defined, at least in an initial version.

Use-cases

1. An arbitrary number of stacks, some of which may be related
   • Nested stacks, too
2. Embedded support files (for example: Runtime code for Lambda; Static assets to be loaded to S3; Docker Images; CloudFormation documents for nested stacks; AMI baking instructions; ...)
   • Some of the support files may be too large to "bundle in", so support for external storage (S3, ...) may be relevant
   • Managed build steps (specific build steps for specific asset types (e.g: Python lambda bundle, NodeJS lambda bundle, Docker image, ...).
     • Need to consider CodePipeline limits (can be raised, but defaults are quite low & would come in the way of un-capped step count) as well as CodeBuild billing model (aka cost implications of many fast builds versus one slower build).
   • Self-contained means it can be digitally signed, and provisioning mechanisms can be configured to refuse handling un-signed or tampered with assemblies.
3. Phasing of deployments to honor dependencies between stacks & support files
   • Through use of the CDK CLI tools
   • Using a CI/CD pipeline
4. Re-use of the exact same artifacts in several stacks (in a CI/CD environment, deploy to QA environment, run integration tests, roll forward to production environment)
5. Interaction with context (in particular, when doing CI/CD, need to handle missing context correctly).
6. Workflows similar to:
   1. Build assembly, send to QA
   2. QA tests assembly, signs it if it passes, send to release management
   3. Release management verifies assembly is signed by QA key, deploys it to production

Remarks

• Phases: build then synthesize then package then deploy.
• The output of a CDK App execution is not the Cloud Assembly, but would describe instructions on how to piece it together.

[eladb]

A Cloud Assembly is the result of synthesizing a CDK App (or a subset thereof). It is a self-sufficient set of artifacts that can be used to deploy the application to AWS accounts without using the CDK App or toolkit.

I don't think there is a need to support synthesis of subsets of applications. I think a cloud assembly is always 1:1 with an application. Keep it simple.

Deploying a Cloud Assembly

I would start with "Requirements". A list of use cases and scenarios that we want to support. Perhaps even prioritized or at least define which ones are supported in the current version of the spec and which are planned to be supported, and also which use cases are "non requirements" (i.e. not covered by this standard).

See #233 for list of requirements.

Specification

A few things come to mind:

• I don't think we need to define the file system structure inside the assembly. We can leave it to the app to decide how to organize the files. Let's define the manifest as a single entrypoint which includes all the information and relatively reference files within the assembly.
• I wonder if we can actually treat assets and stacks as the same "thing". Effectively an app includes various "things" that reference each other and have some dependencies between them. Then, our tools (toolkit, CI/CD) can deploy a set of "thing types" (very similar to asset handlers). Try to see if you can generalize it like this.
• Dependencies are related to references. Ideally, dependencies can be deduced based on references and so the relationship between a stack and its assets is an implicit dependency that stems from the fact that the stack references the assets.

[RomainMuller]

• Dependencies are related to references. Ideally, dependencies can be deduced based on references and so the relationship between a stack and its assets is an implicit dependency that stems from the fact that the stack references the assets.

If you consider "assets" and "stacks" to be "things" (which I read as opaque data), then how are you supposed to discover dependencies between all those? This makes a case of each of them declaring their dependencies at assembly time, and the dependencies being expressed directly in the manifest.

[RomainMuller]

• I don't think we need to define the file system structure inside the assembly. We can leave it to the app to decide how to organize the files. Let's define the manifest as a single entrypoint which includes all the information and relatively reference files within the assembly.

Yeah I kinda intended the paths to be more akin to examples than a hard specification of what it should be like. I've changed the table heading to clear things up a bit there.

• I wonder if we can actually treat assets and stacks as the same "thing". Effectively an app includes various "things" that reference each other and have some dependencies between them. Then, our tools (toolkit, CI/CD) can deploy a set of "thing types" (very similar to asset handlers). Try to see if you can generalize it like this.

Yeah I like the idea. There is no reason why there should be a different handling there.

[rix0rrr]

It is a YAML 1.2 document

Why not JSON? No human will ever need to write it by hand, so why not lowest common denominator data structure?

The type of the artifact, referring to an entry of the artifactTypes attribute of the manifest

Why this level of indirection? Could put the handler directly on the asset type.

• Topologically sort the assets in the document

Need to declare the dependencies then on the Artifact type.

---

Are we provisioning for negotiation between Cloud Assembly producer and consumer? I'm thinking context parameters between toolkit and CDK app. Seems to be out of scope for this document, but it's going to have to be implemented anyway. Will it be some out-of-band mechanism? Also, a method to report errors and other metadata? Will that be out-of-band with this as well?

If so, it seems like this describes what the output will be if synthesis succeeds, but we also need to have a standardized method to describe failing synthesis. Now it might be called something different, but it needs to be standardized as well because multiple tools are going to have to implement it.

[rix0rrr]

Also, are handlers only called once? Or might there be multiple phases for asset handling.

For example: build/package/publish/deploy?

[RomainMuller]

Why not JSON? No human will ever need to write it by hand, so why not lowest common denominator data structure?

I like computer-generated documents to be human-readable nonetheless, as it makes it easier for people to dive deep into the document in the event they have to. I'm not willing to die on this hill, though...

Why this level of indirection? Could put the handler directly on the asset type.

Handlers may have more configuration than just the path to the command, and they could be used by multiple assets. Call me old-school but I like to avoid repeating data.

Need to declare the dependencies then on the Artifact type.

This is what Artifact#references is for.

Are we provisioning for negotiation between Cloud Assembly producer and consumer?

That's a good point. I need t add this.

but we also need to have a standardized method to describe failing synthesis.

I'm not sure I understand the need for standardization beyond "returns in error when not successful"?

Also, are handlers only called once?

That's a part I haven't gone over entirely just yet, but yes, there may be multiple phases involved, and the handlers would somehow be told which phase they're invoked for.

[rix0rrr]

Currently "missing context" is cause for a reinvocation, for example.

That's not a success (yet) but also not a failure.

And reporting warnings is something that we have, as are reporting errors via stack output (even though the called program does not fail)

[RomainMuller]

I guess the missing context & warnings are consequences of how one generates the Cloud Assembly, and not features of the assembly itself...

[rix0rrr]

I know. But we need a standard for those as well, and right now we have none. We might be tempted to say today:

"A CDK app is a program that generates a Cloud Assembly."

But if we did that'd be incomplete. Instead, we have to say something like:

"A CDK app is a program that adheres to the XXX protocol."

And in the XXX protocol it is written somewhere that under some conditions some output somewhere is a Cloud Assembly.

But without XXX protocol being written down, how do we do that?

[eladb]

I think I agree with @rix0rrr that the protocol needs to allow the app to return an error that says "missing context".

[ejholmes]

/cc @phobologic

Would be really cool if we can define a general spec here that would work for both CDK and stacker.

The example manifest above looks really similar to how the internal graph is built within stacker, so going from that to a compiled "Cloud Assembly" would be relatively straight forward.

[RomainMuller]

I'm moving the document building to a PR (#1119), to make it easier to track the revision history of the document and to facilitate conversation & commenting inline.

[RomainMuller]

The CNAB specification provides a mechanism for bundling arbitrary provisioning logic and configuration, but does not quite specify what that logic should look like. We'd have freedom to design the orchestration in whatever way we want.

An interesting work form Microsoft on that topic is deislabs/porter, which aims at providing composable CNAB bundle fragments.

The elements we have to figure out is how to instrument deployments externally to the invocation image (how to pass data in/out of the CNAB, so we can for example orchestrate deployment phases using CodePipelines).

[eladb]

@hjander asks:

What will be the future direction regarding the Cloudlet/CNAB topic ?

[eladb]

@hjander we are still thinking what is the best approach. At the moment, the protocol between CDK apps and the CDK CLI is a proprietary (and ugly) protocol we call the cx-api. It will serve in the short term, but we have a desire to standardize this interface and the cloud assembly was an attempt.

From our conversations with the CNAB folks, it seems like there might be an opportunity to collaborate and define a higher level standard on top of CNAB that will address the requirements of the cloud assembly.

Bottom line: at the moment, we are putting this work on hold, and we will get back to it later, most probably when we do some work on the CLI to clean it up and modularize it.