Support multi stage builds #141

priyawadhwa · 2018-04-24T00:28:16Z

Fixes #4 and partially #9

dlorenc

Makes sense overall, a few high level comments. I'll take a closer look once we rebase this on top of the go-containers switch.

dlorenc · 2018-04-25T17:12:31Z

pkg/commands/copy.go

@@ -39,6 +40,11 @@ func (c *CopyCommand) ExecuteCommand(config *manifest.Schema2Config) error {
 	logrus.Infof("cmd: copy %s", srcs)
 	logrus.Infof("dest: %s", dest)

+	// Resolve from
+	if c.cmd.From != "" {


What if multiple steps have the same from line? Should we add any randomness here?

Sorry, what do you mean by randomness?

Right now, if "from" is specified as a previous stage, we change the build context to be the path to the directory where files from that stage are saved. That directory should remain unchanged even if multiple copy commands refer back to it

I was worried about a case like this:

FROM base1 RUN something FROM base1 RUN somethingelse FROM base2 COPY --from=0

But I guess we can't have conflicts here, because the users have to refer to them by index (or alias)

dlorenc · 2018-04-25T20:25:35Z

pkg/dockerfile/dockerfile.go

+	nameToIndex := make(map[string]string)
+	for i, stage := range stages {
+		index := strconv.Itoa(i)
+		nameToIndex[stage.Name] = index


it looks like the second one here overwrites the first one. Do we need both?

so if the stage has a name associated the second won't override the first, so I added both in just to make it easier to resolve '--from' from names to indexes later on

Ah ok. Maybe it would be more clear to have two separate maps?

I guess it would get confusing if someone tried to name a stage with an index...

Oh I see, that makes sense -- I changed it a bit to be less confusing with one map

dlorenc · 2018-04-25T20:30:35Z

pkg/dockerfile/dockerfile.go

+		if stageIndex <= index {
+			continue
+		}
+		ms, err := image.NewSourceImage(stage.BaseName)


This will have to get rewritten a bit after my go-containerregistry PR goes in.

priyawadhwa · 2018-04-27T17:03:02Z

This is RFAL!

dlorenc · 2018-04-30T23:26:13Z

cmd/executor/cmd/root.go

+	srcContext     string
+	snapshotMode   string
+	bucket         string
+	logLevel       string


looks like you dropped the dockerinsecuretlsverify in a merge conflict.

yah, it didn't look like it was doing anything now that we switched to go-containerregistry

(probably should have done that in a different commit)

dlorenc · 2018-04-30T23:28:30Z

pkg/commands/copy.go

@@ -39,6 +40,11 @@ func (c *CopyCommand) ExecuteCommand(config *manifest.Schema2Config) error {
 	logrus.Infof("cmd: copy %s", srcs)
 	logrus.Infof("dest: %s", dest)

+	// Resolve from
+	if c.cmd.From != "" {


I was worried about a case like this:

FROM base1 RUN something FROM base1 RUN somethingelse FROM base2 COPY --from=0

But I guess we can't have conflicts here, because the users have to refer to them by index (or alias)

dlorenc · 2018-04-30T23:29:03Z

pkg/dockerfile/dockerfile.go

+	nameToIndex := make(map[string]string)
+	for i, stage := range stages {
+		index := strconv.Itoa(i)
+		nameToIndex[stage.Name] = index


Ah ok. Maybe it would be more clear to have two separate maps?

dlorenc · 2018-04-30T23:29:22Z

pkg/dockerfile/dockerfile.go

+	nameToIndex := make(map[string]string)
+	for i, stage := range stages {
+		index := strconv.Itoa(i)
+		nameToIndex[stage.Name] = index


I guess it would get confusing if someone tried to name a stage with an index...

dlorenc · 2018-04-30T23:30:05Z

pkg/dockerfile/dockerfile.go

+			switch c := cmd.(type) {
+			case *instructions.EnvCommand:
+				envCommand := commands.NewEnvCommand(c)
+				if err := envCommand.ExecuteCommand(&imageConfig.Config); err != nil {


Why are we executing ENV commands in here? It seems like this shouldn't have any side effects.

so we need to keep track of which ENVs were declared before a copy command for a case like this:

FROM base1 RUN something FROM base2 ENV file file COPY --from=0 $file ./

So here I'm executing any EnvCommands on a temporary config for that stage, and that updated config is used to evaluate dependencies in any subsequent COPY commands

…multistage

priyawadhwa · 2018-05-08T17:48:41Z

PTAL

Priya Wadhwa added 2 commits April 23, 2018 17:25

Support multi stage builds

7dbc7a0

fixed bug in copy

cf713fe

priyawadhwa force-pushed the multistage branch from 9e641b4 to cf713fe Compare April 24, 2018 01:13

dlorenc reviewed Apr 25, 2018

View reviewed changes

support multi stage builds

904575d

priyawadhwa force-pushed the multistage branch from b1971e2 to a90ce14 Compare April 26, 2018 23:22

Fixed merge conflicts, support multistage builds

48688df

priyawadhwa force-pushed the multistage branch from a90ce14 to 48688df Compare April 26, 2018 23:39

fixed merge conflict

6c01712

dlorenc reviewed Apr 30, 2018

View reviewed changes

Priya Wadhwa added 4 commits April 30, 2018 22:37

Fixed ResolveStages

a1acbe8

Merge branch 'master' of github.com:GoogleContainerTools/kaniko into …

67a3727

…multistage

Update to new structure tests

36933a2

Merge branch 'master' of github.com:GoogleContainerTools/kaniko into …

5699134

…multistage

priyawadhwa added the kokoro:run label May 8, 2018

kokoro-team removed the kokoro:run label May 8, 2018

Merged master, fixed merge conflict

347ce66

priyawadhwa force-pushed the multistage branch from 5e772c4 to 347ce66 Compare May 9, 2018 22:08

gaocegege mentioned this pull request May 10, 2018

Automated build kubeflow/katib#77

Closed

Priya Wadhwa added 2 commits May 11, 2018 15:53

Merge master, refactor to work with ARG

282f8ab

Updated tests

459ddff

sharifelgamal approved these changes May 12, 2018

View reviewed changes

priyawadhwa merged commit cde277c into GoogleContainerTools:master May 14, 2018

priyawadhwa deleted the multistage branch May 14, 2018 22:41

jglick mentioned this pull request Feb 5, 2019

refactor: using multi-stage build to retrieve certificates jenkins-x-apps/jx-app-jacoco#67

Merged

priyawadhwa added the area/multi-stage builds issues related to kaniko multi-stage builds label Feb 9, 2019

aaron-prindle mentioned this pull request Jun 22, 2023

Can't install to container, before doing a build in GitLab. (Missing package manager?) #1440

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support multi stage builds #141

Support multi stage builds #141

priyawadhwa commented Apr 24, 2018

dlorenc left a comment

dlorenc Apr 25, 2018

priyawadhwa Apr 25, 2018

dlorenc Apr 30, 2018

dlorenc Apr 25, 2018

priyawadhwa Apr 25, 2018

dlorenc Apr 30, 2018

dlorenc Apr 30, 2018

priyawadhwa May 1, 2018

dlorenc Apr 25, 2018

priyawadhwa commented Apr 27, 2018

dlorenc Apr 30, 2018

priyawadhwa May 1, 2018

priyawadhwa May 1, 2018

dlorenc Apr 30, 2018

dlorenc Apr 30, 2018

dlorenc Apr 30, 2018

dlorenc Apr 30, 2018

priyawadhwa May 1, 2018

priyawadhwa commented May 8, 2018

Support multi stage builds #141

Support multi stage builds #141

Conversation

priyawadhwa commented Apr 24, 2018

dlorenc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

priyawadhwa commented Apr 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

priyawadhwa commented May 8, 2018