Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): preview of cdk import #17666

Merged
merged 16 commits into from
Apr 5, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions packages/aws-cdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Command | Description
[`cdk diff`](#cdk-diff) | Diff stacks against current state
[`cdk deploy`](#cdk-deploy) | Deploy a stack into an AWS account
[`cdk destroy`](#cdk-destroy) | Deletes a stack from an AWS account
[`cdk import`](#cdk-import) | Import existing AWS resources into a CDK stack
[`cdk bootstrap`](#cdk-bootstrap) | Deploy a toolkit stack to support deploying large stacks & artifacts
[`cdk doctor`](#cdk-doctor) | Inspect the environment and produce information useful for troubleshooting

Expand Down Expand Up @@ -436,6 +437,31 @@ to turn them off, pass the `--no-hotswap` option when invoking it.
**Note**: This command is considered experimental,
and might have breaking changes in the future.

### `cdk import`

Sometimes, it is beneficial to import (enroll/adopt/...) AWS resources, that were
created manually (or by different means), into a CDK stack. Some resources can simply be
deleted and recreated by CDK, but for others, this is not convenient: Typically stateful
resources like S3 Buckets, DynamoDB tables, etc., cannot be easily deleted without an
impact on the service.

To import an existing resource to a CDK stack:

- run a `cdk diff` to ensure there are no pending changes to the CDK stack you want to
import resources into - if there are, apply/discard them first
- add corresponding constructs for the resources to be added in your stack - for example,
for an S3 bucket, add something like `new s3.Bucket(this, 'ImportedS3Bucket', {});` -
**no other changes must be done to the stack before the import is completed**
- run `cdk import` command - if there are multiple stacks in the CDK app, pass a specific
stack name as an argument
- if the resource definition contains all information needed for the import, this happens
automatically (e.g. an `s3.Bucket` construct has an explicit `bucketName` set),
otherwise, CDK will prompt user to provide neccessary identification information (e.g.
the bucket name)
- after cdk import reports success, the resource is managed by CDK. Any subsequent
changes in the construct configuration will be reflected on the resource


### `cdk destroy`

Deletes a stack from it's environment. This will cause the resources in the stack to be destroyed (unless they were
Expand Down
36 changes: 36 additions & 0 deletions packages/aws-cdk/bin/cdk.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,29 @@ async function parseCommandLineArguments() {
'Implies --hotswap by default',
}),
)
.command('import [STACK]', 'Import existing resource(s) into the given STACK', yargs => yargs
.option('execute', { type: 'boolean', desc: 'Whether to execute ChangeSet (--no-execute will NOT execute the ChangeSet)', default: true })
.option('change-set-name', { type: 'string', desc: 'Name of the CloudFormation change set to create' })
.option('rollback', {
type: 'boolean',
desc: "Rollback stack to stable state on failure. Defaults to 'true', iterate more rapidly with --no-rollback or -R. " +
'Note: do **not** disable this flag for deployments with resource replacements, as that will always fail',
})
.option('create-resource-mapping', {
type: 'string',
alias: 'o',
requiresArg: true,
desc: 'If specified, CDK will generate a mapping of existing physical resources to CDK resources to be imported as. The mapping ' +
'will be written in the given file path. No actual import operation will be performed',
})
.option('resource-mapping', {
type: 'string',
alias: 'm',
requiresArg: true,
desc: 'If specified, CDK will use the given file to map physical resources to CDK resources for import, instead of interactively ' +
'asking the user. Can be run from scripts',
}),
)
.command('watch [STACKS..]', "Shortcut for 'deploy --watch'", yargs => yargs
// I'm fairly certain none of these options, present for 'deploy', make sense for 'watch':
// .option('all', { type: 'boolean', default: false, desc: 'Deploy all available stacks' })
Expand Down Expand Up @@ -377,6 +400,19 @@ async function initCommandLine() {
watch: args.watch,
});

case 'import':
return cli.import({
selector,
toolkitStackName,
roleArn: args.roleArn,
execute: args.execute,
changeSetName: args.changeSetName,
progress: configuration.settings.get(['progress']),
rollback: configuration.settings.get(['rollback']),
createResourceMapping: args['create-resource-mapping']?.length > 0,
resourceMappingFile: args['create-resource-mapping'] ? args['create-resource-mapping'] : args['resource-mapping'],
});

case 'watch':
return cli.watch({
selector,
Expand Down
22 changes: 19 additions & 3 deletions packages/aws-cdk/lib/api/cloudformation-deployments.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { publishAssets } from '../util/asset-publishing';
import { Mode, SdkProvider } from './aws-auth';
import { deployStack, DeployStackResult, destroyStack } from './deploy-stack';
import { ToolkitInfo } from './toolkit-info';
import { CloudFormationStack, Template } from './util/cloudformation';
import { CloudFormationStack, Template, ResourcesToImport, ResourceIdentifierSummaries } from './util/cloudformation';
import { StackActivityProgress } from './util/cloudformation/stack-activity-monitor';

/**
Expand Down Expand Up @@ -152,6 +152,11 @@ export interface DeployStackOptions {
* @default - nothing extra is appended to the User-Agent header
*/
readonly extraUserAgent?: string;

/**
* List of existing resources to be IMPORTED into the stack, instead of being CREATED
*/
readonly resourcesToImport?: ResourcesToImport;
}

export interface DestroyStackOptions {
Expand Down Expand Up @@ -193,13 +198,23 @@ export class CloudFormationDeployments {
return stack.template();
}

public async getTemplateSummary(stackArtifact: cxapi.CloudFormationStackArtifact): Promise<ResourceIdentifierSummaries> {
debug(`Retrieving template summary for stack ${stackArtifact.displayName}.`);
const { stackSdk } = await this.prepareSdkFor(stackArtifact, undefined, Mode.ForReading);
const cfn = stackSdk.cloudFormation();

return CloudFormationStack.templateSummary(cfn, stackArtifact.template);
}

public async deployStack(options: DeployStackOptions): Promise<DeployStackResult> {
const { stackSdk, resolvedEnvironment, cloudFormationRoleArn } = await this.prepareSdkFor(options.stack, options.roleArn);

const toolkitInfo = await ToolkitInfo.lookup(resolvedEnvironment, stackSdk, options.toolkitStackName);

// Publish any assets before doing the actual deploy
await this.publishStackAssets(options.stack, toolkitInfo);
// Publish any assets before doing the actual deploy (do not publish any assets on import operation)
if (options.resourcesToImport === undefined) {
await this.publishStackAssets(options.stack, toolkitInfo);
}

// Do a verification of the bootstrap stack version
await this.validateBootstrapStackVersion(
Expand Down Expand Up @@ -230,6 +245,7 @@ export class CloudFormationDeployments {
rollback: options.rollback,
hotswap: options.hotswap,
extraUserAgent: options.extraUserAgent,
resourcesToImport: options.resourcesToImport,
});
}

Expand Down
11 changes: 9 additions & 2 deletions packages/aws-cdk/lib/api/deploy-stack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import { CfnEvaluationException } from './hotswap/evaluate-cloudformation-templa
import { ToolkitInfo } from './toolkit-info';
import {
changeSetHasNoChanges, CloudFormationStack, TemplateParameters, waitForChangeSet,
waitForStackDeploy, waitForStackDelete, ParameterValues, ParameterChanges,
waitForStackDeploy, waitForStackDelete, ParameterValues, ParameterChanges, ResourcesToImport,
} from './util/cloudformation';
import { StackActivityMonitor, StackActivityProgress } from './util/cloudformation/stack-activity-monitor';

Expand Down Expand Up @@ -189,6 +189,12 @@ export interface DeployStackOptions {
* @default - nothing extra is appended to the User-Agent header
*/
readonly extraUserAgent?: string;

/**
* If set, change set of type IMPORT will be created, and resourcesToImport
* passed to it.
*/
readonly resourcesToImport?: ResourcesToImport;
}

const LARGE_TEMPLATE_SIZE_KB = 50;
Expand Down Expand Up @@ -294,7 +300,8 @@ async function prepareAndExecuteChangeSet(
const changeSet = await cfn.createChangeSet({
StackName: deployName,
ChangeSetName: changeSetName,
ChangeSetType: update ? 'UPDATE' : 'CREATE',
ChangeSetType: options.resourcesToImport ? 'IMPORT' : update ? 'UPDATE' : 'CREATE',
ResourcesToImport: options.resourcesToImport,
Description: `CDK Changeset for execution ${executionId}`,
TemplateBody: bodyParameter.TemplateBody,
TemplateURL: bodyParameter.TemplateURL,
Expand Down
15 changes: 15 additions & 0 deletions packages/aws-cdk/lib/api/util/cloudformation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ interface TemplateParameter {
[key: string]: any;
}

export type ResourceIdentifierProperties = CloudFormation.ResourceIdentifierProperties;
export type ResourceIdentifierSummaries = CloudFormation.ResourceIdentifierSummaries;
export type ResourcesToImport = CloudFormation.ResourcesToImport;

/**
* Represents an (existing) Stack in CloudFormation
*
Expand All @@ -35,6 +39,17 @@ export class CloudFormationStack {
}
}

/**
* Retrieve the stack template's summary with the information about resource import identifiers
*/
public static async templateSummary(cfn: CloudFormation, template: any): Promise<ResourceIdentifierSummaries> {
const response = await cfn.getTemplateSummary({ TemplateBody: JSON.stringify(template) }).promise();
if (!response.ResourceIdentifierSummaries) {
debug('GetTemplateSummary API call did not return "ReousrceIdentifierSummaries"');
}
return response.ResourceIdentifierSummaries ?? [];
}

/**
* Return a copy of the given stack that does not exist
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ export class StackStatus {
}

get isDeploySuccess(): boolean {
return !this.isNotFound && (this.name === 'CREATE_COMPLETE' || this.name === 'UPDATE_COMPLETE');
return !this.isNotFound && (this.name === 'CREATE_COMPLETE' || this.name === 'UPDATE_COMPLETE' || this.name === 'IMPORT_COMPLETE');
}

public toString(): string {
Expand Down
Loading