test: cache datasets check to avoid failing ci

[sam-hey]

Enhance the dataset availability test by caching the result when the dataset is available. The test should be skipped if it has already been performed by another CI on the same day.

Example:
https://github.com/embeddings-benchmark/mteb/actions/runs/14171665536/job/39696653523

Code Quality

• Code Formatted: Format the code using make lint to maintain consistent style.

Testing

• New Tests Added: Write tests to cover new functionality. Validate with make test-with-coverage.
• Tests Passed: Run tests locally using make test or make test-with-coverage to ensure no existing functionality is broken.

[Samoed]

I don't think we want this, because we won't be able to check if some datasets were removed (like currently https://huggingface.co/datasets/navjordj/SNL_summarization)

[sam-hey]

It will check daily for any missing datasets, reducing the number of false positives caused by failed with 429 Client Error: Too Many Requests for URL. Additionally, newly added datasets will always be checked.

[sam-hey]

If you take a look here, almost all requests fail and need to be rerun:

[KennethEnevoldsen]

Hmm, it seems like this PR has gotten stale - will close it