Skip to content

Conversation

@StephanTLavavej
Copy link
Member

This updates the vcpkg submodule from a Jan 27, 2020 commit to their 2020.06 release. We should stay reasonably current.

To test this, I cleaned out my repo and the vcpkg submodule, bootstrapped vcpkg, built boost-math for x86 and x64, and then built and tested the repo for x86 and x64.

@StephanTLavavej StephanTLavavej added the build Related to the build system label Jul 14, 2020
@StephanTLavavej StephanTLavavej requested a review from a team as a code owner July 14, 2020 02:06
Copy link
Contributor

@CaseyCarter CaseyCarter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For other reviewers: yes, we do some caching of dependencies managed by vcpkg. And yes, we were smart enough to include the submodule revision in the cache key so there's no need to manually invalidate the cache:

- task: Cache@2
displayName: vcpkg/installed Caching
timeoutInMinutes: 10
inputs:
key: '"${{ parameters.targetPlatform }}" | $(Build.SourcesDirectory)/.git/modules/vcpkg/HEAD | "2020-03-01.01"'
path: '$(vcpkgLocation)/installed'
cacheHitVar: CACHE_RESTORED

@BillyONeal
Copy link
Member

Note that the vcpkg binary caching feature just landed so you might want to use a commit that has microsoft/vcpkg@52a9d9a

@StephanTLavavej
Copy link
Member Author

StephanTLavavej commented Jul 14, 2020

  • What does binary caching do?
  • How will it benefit the STL specifically?
  • Is it important enough to use vcpkg HEAD instead of picking it up in the next official release? (I recall hearing that vcpkg attempts to always be production quality, like the STL, and that its releases are relatively unremarkable, but I don't have significant experience with it.)

Edit to clarify: I'm perfectly happy to update the commit and retest, I'd just like to understand things a little better.

@cbezault
Copy link
Contributor

cbezault commented Jul 14, 2020

The releases are pretty much meaningless in vcpkg. They were only added for users to have better visibility into when changes landed in the repo and for externals who REALLY REALLY wanted release tags.

Binary caching is just the act of saving a library built via vcpkg so that it can be restored rapidly if it ever needs to be installed again.

vcpkg binary caching would mean that we could remove the caching steps of the CI with no latency increase for CI runs. The caching step would exist within vcpkg. This would require setting up a shared location which all the VMs could access.

edit:
Another thought I just had is that we're not actually rebuilding the libraries whenever we bump the toolset version when we create a new VMSS. If we're not going to enable vcpkg binary caching (which considers the compiler used in determining a cache hit) we should update the caching step key whenever there's a toolset update.

@CaseyCarter
Copy link
Contributor

Another thought I just had is that we're not actually rebuilding the libraries whenever we bump the toolset version when we create a new VMSS. If we're not going to enable vcpkg binary caching (which considers the compiler used in determining a cache hit) we should update the caching step key whenever there's a toolset update.

We could pretty easily replace "2020-03-01.01" with $(agentPool) in the key line in the excerpt of run-build.yml I quoted above.

@BillyONeal
Copy link
Member

vcpkg binary caching would mean that we could remove the caching steps of the CI with no latency increase for CI runs. The caching step would exist within vcpkg. This would require setting up a shared location which all the VMs could access.

I think we could eliminate the entire caching-in-ADO step. That means first builds on a given node will be slower but all subsequent ones will be faster because they're restoring from the local-to-the-node cache rather than the incredibly slow ADO cache mechanism.

@cbezault
Copy link
Contributor

Then we wouldn't even need vcpkg caching. We would just have vcpkg always install to the persistent disk of the node.

@BillyONeal
Copy link
Member

That would work; it might even make sense to make updating the vcpkg sha we want part of updating the VMs.

@StephanTLavavej
Copy link
Member Author

#1045 tracks the improvements discussed here - thanks everyone. In the future, we'll just update vcpkg to the current commit. Since I had already validated this one, I think we can just merge it as-is.

@StephanTLavavej StephanTLavavej merged commit 8fcc25f into microsoft:master Jul 14, 2020
@StephanTLavavej StephanTLavavej deleted the update_vcpkg branch July 14, 2020 22:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Related to the build system

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants