Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create latest_container resolver #250

Merged
merged 10 commits into from
Aug 24, 2018
Merged

Conversation

patrobinson
Copy link
Contributor

While we generally don't recommend deploying containers with CloudFormation, see #237, it can sometimes be useful.

This PR creates a parameter resolver that finds the latest image in an ECR repository and returns the full image path to that image.

Air quotes save the day
Copy link
Contributor

@mipearson mipearson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Appreciate the spec for the semi-edge-case of multiple tags.

README.md Outdated
### Latest Container from Repository

Looks up the latest Container Image from an ECR repository. We use this instead of a "latest" tag, because if the tag is latest a CloudFormation update will not trigger a deployment of the container, since nothing has changed.
As such this resolver will never return the latest tag for a container.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like a fun thing to find out the hard way.

latest_image = images.first
latest_tag = latest_image.image_tags.delete_if { |tag| tag == "latest" }.first
# aws_account_id.dkr.ecr.region.amazonaws.com
return "#{latest_image.registry_id}.dkr.ecr.#{@region}.amazonaws.com/#{parameters['repository_name']}:#{latest_tag}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the tag safe to use here? Wouldn't we be better with image_digest?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only an issue if you use mutable tags like "production". I don't think that's good practice so while I wouldn't object to such a change I don't think we should prioritise it

Copy link
Contributor Author

@patrobinson patrobinson Aug 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, recent events have made me question this view.
It seems as though we use our repositories not just to store the latest images but also as a cache for our CI pipelines. It's quite common for pipelines to push up images from a test environment. So finding the latest image is probably a terrible idea and what we should be doing instead is tagging latest (or production or staging), but fetching the sha for that tag and returning it.

if resp.next_token.nil?
break
end
end
Copy link
Contributor

@stevehodgkiss stevehodgkiss Aug 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: begin/end/while is an alternative.

begin
  # ..
end while next_token.nil?

Copy link
Contributor Author

@patrobinson patrobinson Aug 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks one of my tests... and not the pagination test I just added.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, looks like it should be !next_token.nil?

next_token: next_token,
})

images.push(resp.image_details)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this will create an array of arrays containing image_details responses https://docs.aws.amazon.com/sdkforruby/api/Aws/ECR/Client.html#describe_images-instance_method

Maybe images += resp.image_details?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I flatten it later but that's a better way

Patrick Robinson added 3 commits August 22, 2018 21:46
@patrobinson patrobinson force-pushed the latest-container-by-repo branch from 2cda4b3 to 510dd94 Compare August 23, 2018 04:01
@patrobinson
Copy link
Contributor Author

@stevehodgkiss @mipearson @viraptor sorry I changed the interface and code a bit based on my renewed understanding of my use case. We now allow specifying a tag parameter and returning the sha of that tag, to avoid having to version tags and store the latest tag version somewhere.

Copy link
Contributor

@viraptor viraptor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for changing this to digests!

Copy link
Contributor

@mipearson mipearson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpicky changes requested only

images.sort! { |image_x, image_y| image_y.image_pushed_at <=> image_x.image_pushed_at }
latest_image = images.first
# aws_account_id.dkr.ecr.region.amazonaws.com/repository@sha256:digest
return "#{latest_image.registry_id}.dkr.ecr.#{@region}.amazonaws.com/#{parameters['repository_name']}@#{latest_image.image_digest}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary return

end

def resolve(parameters)
@region = parameters['region'] || @stack_definition.region
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe space this function out a bit to make it easier to understand logical groupings, eg:

      def resolve(parameters)
        if parameters['repository_name'].nil?
          raise ArgumentError, "repository_name parameter is required but was not supplied"
        end

        @region = parameters['region'] || @stack_definition.region
        ecr_client = Aws::ECR::Client.new(region: @region)

        images = fetch_images(parameters['repository_name'], parameters['registry_id'], ecr_client)
        return nil if images.empty?

        if !parameters['tag'].nil?
          images.select! { |image| image.image_tags.any? { |tag| tag == parameters['tag'] } }
        end
        images.sort! { |image_x, image_y| image_y.image_pushed_at <=> image_x.image_pushed_at }
        latest_image = images.first

        # aws_account_id.dkr.ecr.region.amazonaws.com/repository@sha256:digest
        "#{latest_image.registry_id}.dkr.ecr.#{@region}.amazonaws.com/#{parameters['repository_name']}@#{latest_image.image_digest}"
      end

end

def resolve(parameters)
if parameters['repository_name'].nil?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is registry_id also a required parameter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, as per the documentation it's only needed if you're fetching a container from a different account

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NM, I see the default now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, no I don't, the docs say it should default but I don't see the defaulting happening.
And no test for what happens when it's not...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API handles this for us. If we don't specify a registry ID, it defaults to the current account

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, no need for tests then either :-)

@patrobinson patrobinson merged commit c8cd371 into master Aug 24, 2018
@patrobinson patrobinson deleted the latest-container-by-repo branch August 24, 2018 00:18
patrobinson pushed a commit that referenced this pull request Aug 24, 2018
Supports latest_container parameter resolver #250
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants