Accomodate caching via vagrant-cachier plugin

[gondo]

it would be very nice to have some kind of caching in place, so i dont have to download chef each time i do vagrant up

[patcon]

We're discussing this problem here: fgrehm/vagrant-cachier#13 (comment)

The issue can soon be solved by vagrant-cachier, as soon as we get a PR into vagrant core that gives file_cache_path a more sensible default: hashicorp/vagrant#1897

As referenced in the vagrant-cachier issue above, two things would be needed from vagrant-omnibus for this caching:

1. the ability to set a download_path to be passed as an argument into install.sh, perhaps directly from the new file_cache_path value in the vagrant config object:
   https://github.com/schisamo/vagrant-omnibus/blob/master/lib/vagrant-omnibus/action/install_chef.rb#L67-L70
2. Make sure that vagrant-omnibus hook actions play nicely with vagrant-cachier's timing:
   https://github.com/fgrehm/vagrant-cachier/blob/master/lib/vagrant-cachier/plugin.rb#L31-L36
   https://github.com/schisamo/vagrant-omnibus/blob/master/lib/vagrant-omnibus/plugin.rb#L33-L35

This isn't currently possible with the current install.sh script, but perhaps @schisamo or @jtimberman could help me find out how to contribute to it? I'm unsure where it's version-controlled :)

Also, perhaps the issue title could be updated to: Accomodate caching via vagrant-cachier plugin ?

[fgrehm]

hashicorp/vagrant#1897 has been merged #justsaying ;)

[patcon]

In other words, we're just about ready to merge fgrehm/vagrant-cachier#14, which means we can easily cache Chef::Config['file_cache_path'] if that's where this plugin drops the rpm's and deb's :)

[tknerr]

@patcon: as of vagrant 1.2.7 and vagrant-cachier 0.3.0 it's still not being cached, is it? Or should it be?

[tknerr]

Ah, now I see. It won't be cached because the install.sh the vagrant-omnibus plugin delegates to uses an arbitrary tmp directory:

...
tmp_dir=$(mktemp -d -t tmp.XXXXXXXX || echo "/tmp")

if exists wget;
then
downloader="wget"
wget -O "$tmp_dir/$filename" "$url" 2>/tmp/stderr
elif exists curl;
then
downloader="curl"
curl -L "$url" > "$tmp_dir/$filename"
else
echo "Cannot find wget or curl - cannot install Chef!"
exit 5
fi
...

[fgrehm]

@tknerr @patcon @gondo I've been thinking about this and I believe we can just use Vagrant::Util::Downloader to grab the installation script from the host and somehow patch the downloaded file to drop things on the right folder.

what do u guys think?

[tknerr]

@fgrehm yes that would be worth a try. I understand though why 'install.sh' is not using '/var/chef/cache' before it installs chef which creates that directory. Not really a chicken/egg problem but we have to ensure that the omnibus installer does not bail out if that directory already exists.

[patcon]

@fgrehm that's actually a really great idea! Perhaps we could patch vagrant-omnibus instead?

EDIT: Oops. Thought we were in the vagrant-cachier queues. Of course this would be a vagrant-omnibus fix :) /EDIT

The install.sh script has a line like this:

tmp_dir=$(mktemp -d -t tmp.XXXXXXXX || echo "/tmp")

which is used to construct lines like this later in the script:

curl -L "$url" > "$tmp_dir/$filename"

so we could replace the script command inline with something like:

https://github.com/schisamo/vagrant-omnibus/blob/master/lib/vagrant-omnibus/action/install_chef.rb#L65-L77

        def install(version, destination_dir)
          command = <<-INSTALL_OMNIBUS
            if command -v wget &>/dev/null; then
              wget -qO- #{INSTALL_SH} | sed "s/^tmp_dir=.*$/tmp_dir=#{destination_dir}/" | sudo bash -s -- -v #{version}
            elif command -v curl &>/dev/null; then
              curl -L #{INSTALL_SH} | sed "s/^tmp_dir=.*$/tmp_dir=#{destination_dir}/" | sudo bash -s -- -v #{version}
            else
              echo "Neither wget nor curl found. Please install one." >&2
              exit 1
            fi
          INSTALL_OMNIBUS
        end

So the sed should search a replace the destination tmp_dir with something of our choosing.

Haven't tested, so my regex might be off. I'm a vacay for another week, so likely won't get to this for a few weeks.

[fgrehm]

@patcon yeah, or we can use Vagrant::Util::Downloader on the host to get INSTALL_SH, do the find-and-replace using plain ruby on top of the downloaded file :)

[tknerr]

@patcon @fgrehm just noticed that you can set ENV['OMNIBUS_INSTALL_URL'] in https://github.com/schisamo/vagrant-omnibus/blob/v1.1.0/lib/vagrant-omnibus/action/install_chef.rb#L25

This could be used as a temporary workaround, but haven't verified it yet...

[tknerr]

@patcon @fgrehm gnaaah didn't work :-(

It's the same issue as in tmatilai/vagrant-proxyconf#13, i.e. vagrant-omnibus kicks in before vagrant-cachier configures the cache buckets:

...
[sample-app] Installing Chef 11.6.0 Omnibus package...
[sample-app] Configuring cache buckets...
[sample-app] Running provisioner: chef_solo...
...

You can find the sample above using a slightly modified install.sh is on the cache-omnibus branch here.

@fgrehm mind hooking in vagrant-cachier before vagrant-omnibus similar to tmatilai/vagrant-proxyconf@bcbdc39?

[lamont-granquist]

i'm working on a -f flag to install.sh to download or install from cached file

[tknerr]

This would be fixed via #68

[schisamo]

Now that #73 has been merged this should be fixed.