ci : store ccache on HF buckets (test with cuda-ubuntu for now)

[CISC]

Overview

Use Hugging Face buckets to save and restore ccache, testing on cuda-ubuntu for now.

Additional information

Keeps the latest ccache stored in a folder (so we can organize them per repo etc) on HF buckets instead of on GHA cache.

Only supports *nix ccache for now, no sccache or Windows support.

Test runs:
https://github.com/ggml-org/tmp/actions/runs/32833839548/job/97758228073
https://github.com/ggml-org/tmp/actions/runs/32833839548/job/97758227953
https://github.com/ggml-org/tmp/actions/runs/32840956765/job/97780129104

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Nê
• Language disclosure: Zaza

[CISC]

Seems these containers are fairly minimal. :)

[ggerganov]

Seems these containers are fairly minimal. :)

In what sense?

[CISC]

Seems these containers are fairly minimal. :)

In what sense?

hip was missing venv, cuda was missing python3, and all were missing jq.

[ggerganov]

Should we merge?

[CISC]

Should we merge?

Should be fine. :)

[netrunnereve]

I'm late but I think there should be some sort of check here (maybe for the HF token?) that lets the run continue if we're running on a fork that doesn't have these buckets set up.

Also I'm guessing that these caches will start showing up in https://huggingface.co/ggml-org/buckets eventually?

[CISC]

I'm late but I think there should be some sort of check here (maybe for the HF token?) that lets the run continue if we're running on a fork that doesn't have these buckets set up.

Yeah, I intended to, but forgot. A simple check that hf_bucket is not blank should suffice.

Also I'm guessing that these caches will start showing up in https://huggingface.co/ggml-org/buckets eventually?

They are already, but the bucket is private.

[CISC]

I'm late but I think there should be some sort of check here (maybe for the HF token?) that lets the run continue if we're running on a fork that doesn't have these buckets set up.

Yeah, I intended to, but forgot. A simple check that hf_bucket is not blank should suffice.

Actually seems to be working, silently fails with Invalid HF URI, will add a check anyway though.

[CISC]

Ouch, vars are not available to PRs?
https://github.com/ggml-org/llama.cpp/actions/runs/32939294855/job/98086775226#step:6:125

Edit: Come to think of it secrets aren't either, which means it can't download from a private bucket. :(

[CISC]

@ggerganov Ok, this one might be a dud, we can remove the variable and make the bucket public, but I'm not sure if it's safe to expose the ccache files?

Edit: It should work as cache for a HF hosted runner though.

[ggerganov]

@ggerganov Ok, this one might be a dud, we can remove the variable and make the bucket public, but I'm not sure if it's safe to expose the ccache files?

I don't think this would be any different from the ccache files hosted on Github. They are also public in the sense that any workflow that we run can read and modify them.

[CISC]

@ggerganov Ok, this one might be a dud, we can remove the variable and make the bucket public, but I'm not sure if it's safe to expose the ccache files?

I don't think this would be any different from the ccache files hosted on Github. They are also public in the sense that any workflow that we run can read and modify them.

True, but since they won't run before we approve them, there's at least some barrier there. Not actually checked if there's anything that can be exfiltrated from them?

[ggerganov]

True, but since they won't run before we approve them, there's at least some barrier there.

I think it's the same for the HF bucket ccache? Does it not work like this:

• We make the bucket public
• Writing to the bucket still requires a token with permissions
• Only approved workflows have access to the token because it is a secret in the llama.cpp repo

[CISC]

True, but since they won't run before we approve them, there's at least some barrier there.

I think it's the same for the HF bucket ccache? Does it not work like this:

• We make the bucket public
• Writing to the bucket still requires a token with permissions
• Only approved workflows have access to the token because it is a secret in the llama.cpp repo

Sure, my worry was just that it may contain a log with a secret or something, but took a look now and it doesn't seem so.