Allow include/exclude for file provisioner #9671
Comments
steviecoaster
changed the title
|
Aside-meta-note: closing issues as "wontfix" with a version target is fundamentally confusing and sends mixed messages. By all means, target for whatever future version, but closing it is a pretty clear "not gonna happen". But yeah, please provide for this. I'd like to be able to upload the contents of a git repo without having to wait half an hour for it to slowly upload the |
|
Hello there, thanks for opening; I think this should be solved with the hcl2 fileset function ! |
|
I think you're right that the rationale for having closed the original issue was weak. This would be a nice UI improvement. Probably the right way to address this in the provisioner itself rather than the hcl2 fileset function would be to change the UploadDir call here https://github.com/hashicorp/packer/blob/master/provisioner/file/provisioner.go#L180 to a directory-walking function that checks against an "exlcude" list that the user can pass in as an option to the provisioner. Since it does look like HCL provides a built-in solution for this, I probably won't direct the Packer team to work on it, but I'd be happy to review a community PR that does this. |
|
Maybe I'm missing something, but I don't see any way to exclude specific files or directories using HCL2's |
|
Hello there, yes sorry, my previous message was rather short. The HCL And you only want the $ echo 'fileset("**", "*.txt")' | packer console pkr-consul
[
"../files/hello.txt",
"../files/world.txt",
"../linux/scripts/README.txt",
]or only $ echo 'fileset("linux/scripts", "*.sh")' | packer console pkr-consul
[
"script-1-install.sh",
"script-2-setup.sh",
]I'd advise on sorting the fileset after this because fileset will not sort anything ( so depending on how your filesystem works this might differ).
If that does not help you to solve your problem; please describe your issue a bit more so that we can help you better 🙂 |
|
I'm wondering if anyone has tried to tackle this yet. I'm administering a Rails app that has a bunch of stuff I need to exclude and in my case I can't delete the excess after the fact. I'm not sure if I will have time to tackle this issue anytime soon, but I am motivated to see it solved. |
|
I don't think anyone has, but I'd be interested to know if the fileset function solves this for you. |
|
Just for the record, I didn't respond because I ended up not using packer at all. |
|
A docker-based solution |
It sadly does not. I've removed my previous message where I thought I solved the problem with this snippet But I quickly discovered that doing it this way results in just a pile of files in target directory with no directory structure being kept. (I guess) You can potentially create a number of provisioners for every subdirectory, but even if it works I would hardly call it a solution. |
|
From the looks of my tree below, you may imagine what I've done: Some way to control how to ignore files would be great. |
|
I know this is ancient, but in case anyone else finds their way here looking for answers, I ended up using a gnarly construct like this: # packer hcl file
...
locals {
# where the files are on your local drive
srcprefix = "..."
# where to put them in the packer instance
dstprefix = "..."
# list of directories and filenames to ignore
ignore = convert(["node_modules", "build", "dist"], set(string))
# build map of file=>dir with the files we want removed from the list.
# to explain the `if`: `setintersection` is used to check if any of the path segments in the file name
# match directories or filenames we want to ignore. if the length of the result `is > 0`, discard
# the file. conversion to set(string) is needed as packer doesn't want to do the conversion itself.
filetodir = {
for file in setunion(
fileset(".", "${local.srcprefix}/foo/**"),
fileset(".", "${local.srcprefix}/bar/**"),
): file => dirname(file) if length(setintersection(convert(split("/", file), set(string)), local.ignore)) == 0
}
# build map of destination dir => [ source files ]
uploads = {
for dir in distinct(values(local.filetodir)):
replace(dir, local.srcprefix, local.dstprefix) => [for f,d in local.filetodir: f if d == dir ]
}
}
build {
...
# precreate all needed directories
provisioner shell {
inline = [for dir in keys(local.uploads): "mkdir -p ${dir}"]
}
# generate a dynamic provisioner for each directory with each directory's source files in a list
dynamic provisioner {
for_each = local.uploads
labels = ["file"]
content {
sources = provisioner.value
destination = "${provisioner.key}/"
}
}
...
}Hope this helps someone |
and others
It seems #1811 was closed as won't fix for....no other reason than it would be confusing.
This is a misstep as having that functionality would severely improve the usability of the file provisioner, without having to have multiple steps or convoluted shell/powershell code to clean up after it.
Please consider adding this feature.
The text was updated successfully, but these errors were encountered: