Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cache key do not include linux distro name and version #368

Closed
2 of 5 tasks
kolyshkin opened this issue Apr 10, 2023 · 7 comments · Fixed by #383
Closed
2 of 5 tasks

cache key do not include linux distro name and version #368

kolyshkin opened this issue Apr 10, 2023 · 7 comments · Fixed by #383
Assignees
Labels
bug Something isn't working

Comments

@kolyshkin
Copy link

kolyshkin commented Apr 10, 2023

Description:

setup-go@v4 added /home/runner/.cache/go-build caching, which does not take the OS version into account, as a result, cache saved from Ubuntu 22.04 can be used on Ubuntu 20.04, resulting in cryptic linker errors. Apparently, this issue is only happening with Go 1.19.x (I guess they've worked around that in Go 1.20, but am too lazy to take a look).

Found it by experimenting with adding Ubuntu 22.04 to github.com/opencontainers/runc CI matrix. Meaning, the tests are now run on BOTH Ubuntu 20.04 and 22.04, using both Go 1.19.x and 1.20.x. IOW, here's the test matrix, simplified:

jobs:
  test:
    strategy:
      matrix:
        os: [ubuntu-20.04, ubuntu-22.04]
        go-version: [1.19.x, 1.20.x]
    runs-on: ${{ matrix.os }}

Once I managed to get it running, I started to see errors like these on Ubuntu 20.04:

/tmp/go-build2163232479/b244/fs.test: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /tmp/go-build2163232479/b244/fs.test)
402
/tmp/go-build2163232479/b244/fs.test: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /tmp/go-build2163232479/b244/fs.test)
403
FAIL	github.com/opencontainers/runc/libcontainer/cgroups/fs	0.001s
# /home/runner/work/runc/runc/tests/integration/../../contrib/cmd/recvtty/recvtty: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/runner/work/runc/runc/tests/integration/../../contrib/cmd/recvtty/recvtty)
646
# /home/runner/work/runc/runc/tests/integration/../../contrib/cmd/recvtty/recvtty: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /home/runner/work/runc/runc/tests/integration/../../contrib/cmd/recvtty/recvtty)

By looking into setup-go logs for Ubuntu-20.04 + Go 1.19.x, I see:

Successfully set up Go version 1.19.x
/opt/hostedtoolcache/go/1.19.7/x64/bin/go env GOMODCACHE
/opt/hostedtoolcache/go/1.19.7/x64/bin/go env GOCACHE
/home/runner/go/pkg/mod
/home/runner/.cache/go-build
Received 42348815 of 42348815 (100.0%), 122.0 MBs/sec
Cache Size: ~40 MB (42348815 B)
/usr/bin/tar -z -xf /home/runner/work/_temp/19eb924c-5cbd-4755-aed2-942e62a6e0e7/cache.tgz -P -C /home/runner/work/runc/runc
Cache restored successfully
Cache restored from key: setup-go-Linux-go-1.19.7-592beaece5e8c0bb8caa2c1fc405d1151913594bf8854051414226cfa9818852
go version go1.19.7 linux/amd64

Now, for Ubuntu 22.04 + Go 1.19.x I see:

Successfully set up Go version 1.19.x
/opt/hostedtoolcache/go/1.19.7/x64/bin/go env GOMODCACHE
/opt/hostedtoolcache/go/1.19.7/x64/bin/go env GOCACHE
/home/runner/go/pkg/mod
/home/runner/.cache/go-build
Received 42348815 of 42348815 (100.0%), 44.4 MBs/sec
Cache Size: ~40 MB (42348815 B)
/usr/bin/tar -z -xf /home/runner/work/_temp/3aa24702-2318-4693-8622-514b8f2482cc/cache.tgz -P -C /home/runner/work/runc/runc
Cache restored successfully
Cache restored from key: setup-go-Linux-go-1.19.7-592beaece5e8c0bb8caa2c1fc405d1151913594bf8854051414226cfa9818852
go version go1.19.7 linux/amd64

💡 The issue here, as you can see, that the same cache is being used for Ubuntu 20.04 and 22.04.

Action version:
v4

Platform:

  • Ubuntu
  • macOS
  • Windows

Runner type:

  • Hosted
  • Self-hosted

Tools version:
1.19.x, 1.20.x

Repro steps:
See e.g. this test run: https://github.com/opencontainers/runc/actions/runs/4654779815/jobs/8236809130 (from opencontainers/runc#3820).

Expected behavior:
No cryptic linkers errors, cache being unique per OS distro.

Actual behavior:
See above.

@dmitry-shibanov
Copy link
Contributor

Hello @kolyshkin. Thank you for your report. We'll take a look on issue.

@kolyshkin kolyshkin changed the title cache key do not include ubuntu version cache key do not include linux distro name and version Apr 10, 2023
@kolyshkin
Copy link
Author

This action uses the following cache key: ${{ runner.os }}-go${{ go-version }}-${{ hashFiles('<go.sum-path>') }} (as documented in https://github.com/actions/setup-go/blob/main/docs/adrs/0000-caching-dependencies.md) as per actions/cache recommendation.

Surely, mixing caches from different Ubuntu versions is not a good thing.

The fix is to add a distro name and version (such as ubuntu-22.04) but I am not sure if there's a simple way (i.e. a variable) to get this info.

A workaround (in my specific case) would be to add something like cache-key-prefix: {{ matrix.os}}, if cache-key-prefix (see #366) would be available. I don't like this much as it requires user configuration, and that's why I call it a workaround not a fix.

PS found that this problem is not specific to this repo, and someone has already developed a workaround: https://github.com/marketplace/actions/github-actions-runner-os-system-information (note the "Example").

@kolyshkin
Copy link
Author

The fix is to add a distro name and version (such as ubuntu-22.04) but I am not sure if there's a simple way (i.e. a variable) to get this info.

Filed actions/runner#2527

@kolyshkin
Copy link
Author

Hello @kolyshkin. Thank you for your report. We'll take a look on issue.

Can I inquire about the potential timeline for the fix? Also, I tried to help by making the fix easier (actions/runner#2527) but it didn't work.

samuelkarp added a commit to samuelkarp/containerd that referenced this issue May 8, 2023
actions/setup-go#368 and
opencontainers/runc#3820 (comment)
discuss issues with the cache key for actions/setup-go@v4.  Rather than
reverting the upgrade to v4 (per discussion in
containerd#8372), disable caching
explicitly.

Signed-off-by: Samuel Karp <[email protected]>
@songu21
Copy link

songu21 commented May 23, 2023

d

@dsame dsame self-assigned this Jun 6, 2023
dsame added a commit to akv-platform/setup-go that referenced this issue Jun 6, 2023
@dsame dsame mentioned this issue Jun 6, 2023
2 tasks
dsame added a commit to akv-platform/setup-go that referenced this issue Jun 6, 2023
akhilerm added a commit to akhilerm/cri-tools that referenced this issue Jun 8, 2023
update all the github actions using node12 to version that use node16

- update actions/checkout@v2 -> v3
- update actions/cache@v2 -> v3
- update actions/setup-go@v2 -> v4

cache has been set to disabled for setup-go
due to actions/setup-go#368

Signed-off-by: Akhil Mohan <[email protected]>
dsame added a commit to akv-platform/setup-go that referenced this issue Jul 10, 2023
@dsame dsame linked a pull request Jul 12, 2023 that will close this issue
2 tasks
@dsame
Copy link
Contributor

dsame commented Jul 12, 2023

The issue is closed because of #383 has been merged

@kolyshkin
Copy link
Author

Fixed in https://github.com/actions/setup-go/releases/tag/v4.1.0 which was released today. Thanks!

kolyshkin added a commit to kolyshkin/runc that referenced this issue Aug 8, 2023
Since actions/setup-go#368 is now fixed
(in https://github.com/actions/setup-go/releases/tag/v4.1.0), there
is no need to disable caching when using different distros.

Signed-off-by: Kir Kolyshkin <[email protected]>
kolyshkin added a commit to kolyshkin/runc that referenced this issue Aug 8, 2023
Since actions/setup-go#368 is now fixed
(in https://github.com/actions/setup-go/releases/tag/v4.1.0), there
is no need to disable caching when using different distros.

Signed-off-by: Kir Kolyshkin <[email protected]>
jsturtevant pushed a commit to jsturtevant/containerd that referenced this issue Sep 21, 2023
actions/setup-go#368 and
opencontainers/runc#3820 (comment)
discuss issues with the cache key for actions/setup-go@v4.  Rather than
reverting the upgrade to v4 (per discussion in
containerd#8372), disable caching
explicitly.

Signed-off-by: Samuel Karp <[email protected]>
juliusl pushed a commit to juliusl/containerd that referenced this issue Jan 26, 2024
actions/setup-go#368 and
opencontainers/runc#3820 (comment)
discuss issues with the cache key for actions/setup-go@v4.  Rather than
reverting the upgrade to v4 (per discussion in
containerd#8372), disable caching
explicitly.

Signed-off-by: Samuel Karp <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants