Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stepfunctions_tasks: Upgrade the volume type when creating EMR cluster with the EmrCreateCluster task #28874

Closed
lafeuil opened this issue Jan 26, 2024 · 3 comments · Fixed by #28911
Labels
@aws-cdk/aws-stepfunctions-tasks bug This issue is a bug. effort/small Small work item – less than a day of effort p2

Comments

@lafeuil
Copy link

lafeuil commented Jan 26, 2024

Describe the bug

In the step function, when creating an EMR cluster with the EmrCreateCluster task, the EBS volume of each instance can be configured. The volume is described by the VolumeSpecificationProperty interface and the type of each volume by the volumeType property and the EbsBlockDeviceVolumeType enum.

But in the EbsBlockDeviceVolumeType, there are only the gp2, io1 and standard type. The EMR specification of the RunJobFlow action accepts for the VolumeType property : gp3, gp2, io1, st1, sc1, and standard.

Expected Behavior

The EmrCreateCluster task should accept all supported volume type of the RunJobFlow action : gp3, gp2, io1, st1, sc1, and standard.

Current Behavior

The EmrCreateCluster task only accepts gp2, io1 and standard EBS volume type.

Reproduction Steps

new EmrCreateCluster(scope, 'Create Cluster', {
  name: 'MyCluster',
  instances: {
    instanceFleets: [
      {
        instanceFleetType: EmrCreateCluster.InstanceRoleType.CORE,
        instanceTypeConfigs: [
          {
            instanceType: 'r6g.xlarge',
            ebsConfiguration: {
              ebsBlockDeviceConfigs: [
                {
                  volumeSpecification: {
                    volumeSize: Size.gibibytes(256),
                    volumeType: EmrCreateCluster.EbsBlockDeviceVolumeType.GP3,
                  },
                },
              ],
            },
          },
        ],
      },
    ],
  },
});

Possible Solution

Upgrade the EbsBlockDeviceVolumeType enum with the missing value : gp3, st1, and sc1.

Additional Information/Context

No response

CDK CLI Version

2.123.0

Framework Version

No response

Node.js Version

20

OS

linux

Language

TypeScript

Language Version

No response

Other information

Some comments in other issue :

@lafeuil lafeuil added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jan 26, 2024
@pahud
Copy link
Contributor

pahud commented Jan 29, 2024

Yes we can submit a tiny PR to add all supported types in the enum class:

export enum EbsBlockDeviceVolumeType {
/**
* gp2 Volume Type
*/
GP2 = 'gp2',
/**
* io1 Volume Type
*/
IO1 = 'io1',
/**
* Standard Volume Type
*/
STANDARD = 'standard',
}

@pahud pahud added p2 effort/small Small work item – less than a day of effort and removed needs-triage This issue or PR still needs to be triaged. labels Jan 29, 2024
@mergify mergify bot closed this as completed in #28911 Jan 30, 2024
@mergify mergify bot closed this as completed in 41bb188 Jan 30, 2024
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

1 similar comment
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

SankyRed pushed a commit that referenced this issue Feb 8, 2024
…28911)

Support additional volume types for `EbsBlockDeviceVolumeType`.

Closes #28874

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-stepfunctions-tasks bug This issue is a bug. effort/small Small work item – less than a day of effort p2
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants