Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move pod-infra-container-image generation to schnauzer template helpers #1551

Merged
merged 2 commits into from
May 3, 2021

Conversation

tjkirch
Copy link
Contributor

@tjkirch tjkirch commented Apr 30, 2021

Description of changes:

Move pod-infra-container-image generation to schnauzer template helpers

Previously, pluto talked to IMDS to get the current region so it could generate
the right pause container URL.  This was added before we had the
settings.aws.region setting, so that was the only way.  However, it means extra
IMDS traffic, and it means you can't override the region of the image, which is
useful when testing new regions.  (It also means you can't override the image
arch, though that's less useful.)

This creates a 'pause-prefix' helper, like the existing ecr-prefix helper, that
can be used in schnauzer templates.

This also adds a 'goarch' helper, to convert Bottlerocket-standard architecture
names like x86_64 to "Go-like" architecture names like amd64, as used in the
pause container.
Add migration for pluto -> schnauzer migration for pause container URL

I made the pause-prefix code very similar to the existing ecr-prefix code, so if you expand the diff a little you'll see a good example. The regions/accounts were just a plain move, of course.

Testing done:

Before, the pause container ignored settings.aws.region. I launched a us-west-2 instance with settings.aws.region=ap-northeast-3 user data, and it still pulled from us-west-2.

host-ctr[3545]: time="2021-04-26T21:05:29Z" level=info msg="pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause-amd64:3.1"
host-ctr[3545]: time="2021-04-26T21:05:30Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause-amd64:3.1"
host-ctr[3545]: time="2021-04-26T21:05:30Z" level=info msg="unpacking image..." img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause-amd64:3.1"
host-ctr[3545]: time="2021-04-26T21:05:30Z" level=info msg="tagging image" img="602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause-amd64:3.1"

After, it used my setting for ap-northeast-3, even though the instance was in us-west-2:

host-ctr[3719]: time="2021-04-30T23:41:49Z" level=info msg="pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:ap-northeast-3:602401143452:repository/eks/pause-amd64:3.1"
host-ctr[3719]: time="2021-04-30T23:41:51Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:ap-northeast-3:602401143452:repository/eks/pause-amd64:3.1"
host-ctr[3719]: time="2021-04-30T23:41:51Z" level=info msg="unpacking image..." img="ecr.aws/arn:aws:ecr:ap-northeast-3:602401143452:repository/eks/pause-amd64:3.1"
host-ctr[3719]: time="2021-04-30T23:41:51Z" level=info msg="tagging image" img="602401143452.dkr.ecr.ap-northeast-3.amazonaws.com/eks/pause-amd64:3.1"

You can see that it generated the value correctly, and can see the new setting-generator and template:

# apiclient -u /settings?prefix=kubernetes.pod
{"kubernetes":{"pod-infra-container-image":"602401143452.dkr.ecr.ap-northeast-3.amazonaws.com/eks/pause-amd64:3.1"}}
# apiclient -u /metadata/setting-generators
{"settings.kubernetes.pod-infra-container-image":"schnauzer settings.kubernetes.pod-infra-container-image",...}
# apiclient -u /metadata/templates?keys=settings.kubernetes.pod-infra-container-image
{"settings.kubernetes.pod-infra-container-image":"{{ pause-prefix settings.aws.region }}/eks/pause-{{ goarch os.arch }}:3.1"}

(also tested normal health of instance)

Migration:

To test the migration, I started with a v1.0.8 instance built from a recent commit. (I didn't use the ap-northeast-3 user data when testing the migration.) You can see the original pause container, plus the setting-generator and (lack of) template:

$ apiclient -u /settings?prefix=kubernetes.pod
{"kubernetes":{"pod-infra-container-image":"602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause-amd64:3.1"}}
$ apiclient -u /metadata/setting-generators
{...,"settings.kubernetes.pod-infra-container-image":"pluto pod-infra-container-image",...}
$ apiclient -u /metadata/templates?keys=settings.kubernetes.pod-infra-container-image
{}

I updated to a v1.1.0 build that included this change. The migration worked, fixing the setting-generator and adding the template:

# apiclient -u /settings?prefix=kubernetes.pod
{"kubernetes":{"pod-infra-container-image":"602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause-amd64:3.1"}}
# apiclient -u /metadata/setting-generators
{...,"settings.kubernetes.pod-infra-container-image":"schnauzer settings.kubernetes.pod-infra-container-image",...}
# apiclient -u /metadata/templates?keys=settings.kubernetes.pod-infra-container-image
{"settings.kubernetes.pod-infra-container-image":"{{ pause-prefix settings.aws.region }}/eks/pause-{{ goarch os.arch }}:3.1"}

I downgraded back to v1.0.8 and you can see the setting-generator was reverted and the template removed:

# apiclient -u /settings?prefix=kubernetes.pod
{"kubernetes":{"pod-infra-container-image":"602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause-amd64:3.1"}}
# apiclient -u /metadata/setting-generators
{...,"settings.kubernetes.pod-infra-container-image":"pluto pod-infra-container-image",...}
# apiclient -u /metadata/templates?keys=settings.kubernetes.pod-infra-container-image
{}

(I tested the health of the instance at all three points)

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@webern
Copy link
Contributor

webern commented May 1, 2021

In #1547 I'm using Pluto's get_region logic before calling EKS.

Edit: I can get it from the API if that's what we prefer.

@tjkirch
Copy link
Contributor Author

tjkirch commented May 3, 2021

In #1547 I'm using Pluto's get_region logic before calling EKS.

Edit: I can get it from the API if that's what we prefer.

Yeah, I think we should use the existing region setting in #1547, or we'll have the same problem that resulted in this PR - no ability to consistently test new regions / region overrides.

Previously, pluto talked to IMDS to get the current region so it could generate
the right pause container URL.  This was added before we had the
settings.aws.region setting, so that was the only way.  However, it means extra
IMDS traffic, and it means you can't override the region of the image, which is
useful when testing new regions.  (It also means you can't override the image
arch, though that's less useful.)

This creates a 'pause-prefix' helper, like the existing ecr-prefix helper, that
can be used in schnauzer templates.

This also adds a 'goarch' helper, to convert Bottlerocket-standard architecture
names like x86_64 to "Go-like" architecture names like amd64, as used in the
pause container.
@tjkirch
Copy link
Contributor Author

tjkirch commented May 3, 2021

^ Rebase on develop.

@tjkirch tjkirch merged commit aefc551 into bottlerocket-os:develop May 3, 2021
@tjkirch tjkirch deleted the schnauzer-paws branch May 3, 2021 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants