Add ESC50 and zero-shot classification

[RahulSChand]

For #2069

Code Quality

• [ ✅ ] Code Formatted: Format the code using make lint to maintain consistent style.

Documentation

• Updated Documentation: Add or update documentation to reflect the changes introduced in this PR.

Testing

• New Tests Added: Write tests to cover new functionality. Validate with make test-with-coverage.
• Tests Passed: Run tests locally using make test or make test-with-coverage to ensure no existing functionality is broken.

Adding datasets checklist

Reason for dataset addition: ...

• [ ✅] I have run the following models on the task (adding the results to the pr). These can be run using the mteb -m {model_name} -t {task_name} command.
  • sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
  • intfloat/multilingual-e5-small
• [ ✅] I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
• If the dataset is too big (e.g. >2048 examples), considering using self.stratified_subsampling() under dataset_transform()
• [✅ ] I have filled out the metadata object in the dataset file (find documentation on it here).
• Run tests locally to make sure nothing is broken using make test.
• [ ✅] Run the formatter to format the code using make lint.

Adding a model checklist

• [ ✅] I have filled out the ModelMeta object to the extent possible
• [✅ ] I have ensured that my model can be loaded using
  • [ ✅] mteb.get_model(model_name, revision) and
  • [ ]✅ mteb.get_model_meta(model_name, revision)
• [✅ ] I have tested the implementation works on a representative set of tasks.

[RahulSChand]

@Samoed Hey, I finally got around to fixing this PR. I was waiting for this other PR #2082 to merge since many file changes were similar. I have updated the PR with

1. removed all unnecessary files
2. Removed esc50 from multi-label task, its only in zero-shot for now
3. Added F1 score
4. Made sure make test passes
5. Ran the code end to end using the below script

Currently on the fused clap model is there, when this PR gets merged, I will create another small PR with just the unfused model added as well.

import mteb
model_name = "laion/clap-htsat-fused"
model = mteb.get_model(model_name=model_name)

tasks = mteb.get_tasks(tasks=["ESC50_Zeroshot"])

evaluation = mteb.MTEB(tasks=tasks)
results = evaluation.run(model)

print(results[0].scores)

[Samoed]

Great job! Here are a few small suggestions

[RahulSChand]

Great job! Here are a few small suggestions

Is this now in a state where its good to be approved and merged?

[Samoed]

I can approve, but maybe @KennethEnevoldsen or @isaac-chung want something to add