depends cache is not invalidated when the compiler changes

[maflcko]

The depends cache has no mechanism of being invalidated when the cpu architecture, the operating system, or even just the compiler changes from down under it.

I think long term we should be moving to proper compiler caches like ccache (#17103) or guix

[maflcko]

For example, just one of the bugs I ran into due to this: #17233 (comment)

[maflcko]

@dongcarl @theuni Any thoughts or objections?

[elichai]

How much time penalty we get by removing these two lines? https://github.com/bitcoin/bitcoin/blob/master/.travis.yml#L35:L36

We could also mount the ccache dir to the dockers and install+run ccache in the dockers.
that way we only cache the ccache dir

[laanwj]

How much time penalty we get by removing these two lines? https://github.com/bitcoin/bitcoin/blob/master/.travis.yml#L35:L36

What would that do?
If it causes a depends rebuild on every travis run, especially with the runs with Qt included, you're looking at hour(s).

[elichai]

How much time penalty we get by removing these two lines? https://github.com/bitcoin/bitcoin/blob/master/.travis.yml#L35:L36

What would that do?
If it causes a depends rebuild on every travis run, especially with the runs with Qt included, you're looking at hour(s).

even with ccache?

[laanwj]

I don't think ccache really helps that much with building qt. Mind that the qt build first builds a build system (qmake) for the host, then uses that to build the library for the target.
Boost has a similar weird build setup.
Even getting it to use ccache consistently will be a challenge, and after that you're still left with enough overhead.

[maflcko]

FYI. On the DrahtBot guix build, I am caching depends under a key that is equal to the hash of the whole guix tree. The same can probably not be done for depends and only to a lesser extent to gitian.

[dongcarl]

Following up on this after the merge of #20629

For Guix builds

5200929 ensures that the GUIX_ENVIRONMENT env var is inclueded in the id string used for cache invalidation. This means that any change in the definition of our Guix packages will appropriately invalidate the depends cache

For non-Guix builds

4c7d418 adds a lot of toolchain information to the id string in order to trigger cache invalidation should any of it changes. Not as perfect as the situation with Guix, but I believe it's a good enough approximation for most use-cases.

[maflcko]

kewl, thx. Closing for now