ec2-karpenter-agent: improve cloudformation and nodepool templates

[gthao313]

Issue number:

Description of changes:
Karpenter tests failed on bottlerocket 1.30 nodes. The root cause was The provided credentials do not have permission to create the service-linked role for EC2 Spot Instances. Adding on-demand configuration to nodepool template can solve the issue.

Meanwhile I update cloudformation.yaml file base on curl https://raw.githubusercontent.com/aws/karpenter-provider-aws/v0.37.0/website/content/en/preview/getting-started/getting-started-with-karpenter/cloudformation.yaml > cloudformation.yaml

Testing done:

• karpenter tests

 x86-64-aws-k8s-125-ipv6-quick                               Test                          passed                                                4                          0                        7065   fcf71a47                     2024-06-20T18:52:45Z
 x86-64-aws-k8s-125-quick                                    Test                          passed                                                4                          0                        7065   fcf71a47                     2024-06-20T18:57:06Z
 x86-64-aws-k8s-126-ipv6-quick                               Test                          passed                                                4                          0                        7068   fcf71a47                     2024-06-20T18:54:00Z
 x86-64-aws-k8s-126-quick                                    Test                          passed                                                4                          0                        7068   fcf71a47                     2024-06-20T18:54:45Z
 x86-64-aws-k8s-127-ipv6-quick                               Test                          passed                                                5                          0                        7208   fcf71a47                     2024-06-20T18:54:50Z
 x86-64-aws-k8s-128-ipv6-quick                               Test                          passed                                                5                          0                        7388   fcf71a47                     2024-06-20T18:54:11Z
 x86-64-aws-k8s-128-quick                                    Test                          passed                                                5                          0                        7388   fcf71a47                     2024-06-20T18:55:28Z
 x86-64-aws-k8s-129-ipv6-quick                               Test                          passed                                                5                          0                        7406   fcf71a47                     2024-06-20T18:55:49Z
 x86-64-aws-k8s-130-ipv6-quick                               Test                          passed                                                5                          0                        7196   fcf71a47                     2024-06-20T18:54:00Z
 x86-64-aws-k8s-130-quick                                    Test                          passed                                                5                          0                        7196   fcf71a47                     2024-06-20T18:52:50Z

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.