Skip to content

export(local): split opt#3161

Merged
tonistiigi merged 1 commit intomoby:masterfrom
crazy-max:local-exporter-wrap
Jun 30, 2023
Merged

export(local): split opt#3161
tonistiigi merged 1 commit intomoby:masterfrom
crazy-max:local-exporter-wrap

Conversation

@crazy-max
Copy link
Copy Markdown
Member

@crazy-max crazy-max commented Oct 11, 2022 •
edited
Loading

adds split local exporter option which can be used to split result in subfolders when multiple references are exported (default true).

Signed-off-by: CrazyMax crazy-max@users.noreply.github.com

@crazy-max crazy-max force-pushed the local-exporter-wrap branch 2 times, most recently from c59a7a1 to 8e1272d Compare October 14, 2022 10:02
tonistiigi
tonistiigi reviewed Oct 17, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@tonistiigi tonistiigi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if wrap is the best term for this. cc @jedevc for native speakers.

Should the build error if a file with the same name exists on multiple platforms?

@jedevc
Copy link
Copy Markdown
Member

jedevc commented Oct 18, 2022

wrap doesn't seem right, agreed.

I think merge would work (which we already use in the docs), though I'm not sure if we maybe want to reserve that name for doing something different?

Alternatively, we could have an option split (defaulting to true), and then allow setting split=false to get all the results together,

@crazy-max
Copy link
Copy Markdown
Member Author

crazy-max commented Oct 18, 2022

Alternatively, we could have an option split (defaulting to true), and then allow setting split=false to get all the results together,

split sounds good.

@crazy-max crazy-max force-pushed the local-exporter-wrap branch from 8e1272d to 1e93ded Compare October 28, 2022 06:07
@crazy-max crazy-max changed the title export(local): wrap opt export(local): split opt Oct 28, 2022
@crazy-max crazy-max force-pushed the local-exporter-wrap branch from 1e93ded to 050b438 Compare October 28, 2022 06:08
@tonistiigi tonistiigi added this to the v0.12.0 milestone May 4, 2023
@crazy-max crazy-max force-pushed the local-exporter-wrap branch from 050b438 to bde1ec4 Compare May 23, 2023 09:04
@crazy-max crazy-max requested review from jedevc and tonistiigi May 23, 2023 09:22
@crazy-max crazy-max marked this pull request as ready for review May 23, 2023 09:23
jedevc
jedevc reviewed May 30, 2023
View reviewed changes
Comment thread exporter/local/export.go
Copy link
Copy Markdown
Member

@jedevc jedevc May 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also add this functionality to the tar exporter?

Ideally we should keep the options for these as close as possible. I think we should be able to put this opt into CreateFSOPts now that #3289 is merged.

Copy link
Copy Markdown
Member Author

@crazy-max crazy-max May 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now that #3289 is merged.

Ah there's quite a lot of changes indeed, let me address that.

Copy link
Copy Markdown
Member Author

@crazy-max crazy-max May 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also add this functionality to the tar exporter?

Don't think wrapping output for tar exporter would make sense 🤔

Copy link
Copy Markdown
Member

@tonistiigi tonistiigi Jun 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why doesn't this behavior make sense for tar ?

This comment was marked as outdated.

Sign in to view

This comment was marked as resolved.

Sign in to view

Copy link
Copy Markdown
Member Author

@crazy-max crazy-max Jun 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do tar in follow-up if ok with you. I have some issues with MergeFS when platform-split is disabled. Seems we need Stat along FS. Or a fsutil.MergeDirFS method like the SubDirFS one could be better.

Comment thread exporter/local/export.go Outdated
Comment thread exporter/local/export.go Outdated
Comment thread frontend/dockerfile/dockerfile_test.go Outdated
@crazy-max crazy-max force-pushed the local-exporter-wrap branch 3 times, most recently from 0a4b7f4 to 1662821 Compare May 30, 2023 11:08
tonistiigi
tonistiigi reviewed Jun 28, 2023
View reviewed changes
Comment thread README.md Outdated
Comment thread exporter/local/export.go
Copy link
Copy Markdown
Member

@tonistiigi tonistiigi Jun 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why doesn't this behavior make sense for tar ?

Comment thread exporter/local/export.go Outdated
Comment thread exporter/local/export.go
Comment thread exporter/local/export.go Outdated
Comment thread exporter/local/fs.go Outdated
@crazy-max crazy-max force-pushed the local-exporter-wrap branch 2 times, most recently from 7b8b08a to 37187c2 Compare June 29, 2023 17:32
@crazy-max
export(local): split opt
e6818cf 
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
@crazy-max crazy-max force-pushed the local-exporter-wrap branch from 37187c2 to e6818cf Compare June 29, 2023 17:47
@tonistiigi tonistiigi closed this Jun 29, 2023
@tonistiigi tonistiigi reopened this Jun 29, 2023
tonistiigi
tonistiigi approved these changes Jun 30, 2023
View reviewed changes
Comment thread exporter/local/export.go
visitedMu.Lock()
defer visitedMu.Unlock()
if vp, ok := visitedPath[p]; ok {
return errors.Errorf("cannot overwrite %s from %s with %s when split option is disabled", p, vp, k)
Copy link
Copy Markdown
Member

@tonistiigi tonistiigi Jun 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error message could be more understandable to user. Eg. "file x exists in both platform1 and platform2 and would be lost by merging the platforms".

@tonistiigi tonistiigi merged commit 5b9a9ce into moby:master Jun 30, 2023
@crazy-max crazy-max deleted the local-exporter-wrap branch June 30, 2023 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jedevc jedevc jedevc left review comments

@tonistiigi tonistiigi tonistiigi approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

v0.12.0

Development

Successfully merging this pull request may close these issues.

3 participants

@crazy-max @jedevc @tonistiigi