perf: speedup hermit exec

[nickajacks1]

Consists of two changes.

Previously, when `hermit exec` was run on a package with a `requires`
field, it would parse every known package to gather a list of candidates
to fulfill the requirement. In the vast majority of cases, the user will
already have the dependent package installed, so optimize by first
evaluating only the installed packages. This change has no appreciable
effect on the negative path where we *do* end up parsing all known
packages. The result is a speedup by about a factor of 5 to 10 on my machine.

Add concurrency to Loader.All by parsing each hcl file in a separate
goroutine. This results in an execution time that is about 3x faster on
my machine. This change is most visible when Hermit must parse all
packages during `hermit exec`.

Ran with -race to ensure I got the concurrency right.

Example: running maven without the change, with the change, and without hermit (as a control)
Without the change:

$ time mvn --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /home/njackson/.cache/hermit/pkg/maven-3.6.3
Java version: 11.0.10, vendor: AdoptOpenJDK, runtime: /home/njackson/.cache/hermit/pkg/openjdk@11
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.15.167.4-microsoft-standard-wsl2", arch: "amd64", family: "unix"

real    0m0.959s
user    0m1.114s
sys     0m0.096s

With the change:

$ time mvn --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /home/njackson/.cache/hermit/pkg/maven-3.6.3
Java version: 11.0.10, vendor: AdoptOpenJDK, runtime: /home/njackson/.cache/hermit/pkg/openjdk@11
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.15.167.4-microsoft-standard-wsl2", arch: "amd64", family: "unix"

real    0m0.177s
user    0m0.216s
sys     0m0.040s

Without hermit:

$ time mvn --version
Apache Maven 3.6.3
Maven home: /usr/share/maven
Java version: 11.0.25, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-openjdk-amd64
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.15.167.4-microsoft-standard-wsl2", arch: "amd64", family: "unix"

real 0m0.103s
user 0m0.130s
sys 0m0.023s

Fixes: #396

[alecthomas]

I like the idea, but I think it would be a good idea to throttle this a bit. There are ~500 packages currently, so this could generate a lot of I/O and garbage in a very short period of time. Maybe use an errgroup with SetLimit(), or a "semaphore" channel to limit to NumCPU() * 4 or something?

[nickajacks1]

Fixed. My understanding is that in order to actually throttle anything, we'd need to set the limit to below NumCPU. I arbitrarily chose max(3, runtime.NumCPU() / 4) since this seemed to reduce the max CPU% utilization a lot on both my beefy and meh machines while still having about the same speedup.

[alecthomas]

Excellent!

[nickajacks1]

For posterity: the majority of the remaining time in hermit exec is spent parsing HCL files. The overhead in hermit exec is proportional to the number of packages installed and the size of their HCL files and is on the order of 10's of milliseconds per invocation on an idle machine. In theory, the files stored on disk could be converted to something faster to parse, but that may be a can of worms...Barring that, the next thing I could think of is to parse fewer packages.

@alecthomas It looks like the net effect of parsing all packages is that we get all environment variables from all packages when we exec the process. Are you aware of any cases where that's particularly useful as opposed to only parsing the requested package? I can see it being potentially necessary for requires packages, but I'm not too sure.