Add AESDD dataset

[mina-parham]

related to #2359

Code Quality

• Code Formatted: Format the code using make lint to maintain consistent style.

Documentation

• Updated Documentation: Add or update documentation to reflect the changes introduced in this PR.

Testing

• New Tests Added: Write tests to cover new functionality. Validate with make test-with-coverage.
• Tests Passed: Run tests locally using make test or make test-with-coverage to ensure no existing functionality is broken.

Adding datasets checklist

Reason for dataset addition: ...

• I have run the following models on the task (adding the results to the pr). These can be run using the mteb -m {model_name} -t {task_name} command.
  • sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
  • intfloat/multilingual-e5-small
• I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
• If the dataset is too big (e.g. >2048 examples), considering using self.stratified_subsampling() under dataset_transform()
• I have filled out the metadata object in the dataset file (find documentation on it here).
• Run tests locally to make sure nothing is broken using make test.
• Run the formatter to format the code using make lint.

[isaac-chung]

Dataset doesn't seem to exist. Also, could you please run this and report the scores?

mteb run -m facebook/wav2vec2-base -t AESDD
mteb run -m microsoft/wavlm-base -t AESDD

[isaac-chung]

I was not able to find this dataset. I was only able to find this one.

[KennethEnevoldsen]

@mina-parham it seems like this PR has gotten stale - do you have the time to finalize it?

[mina-parham]

@mina-parham it seems like this PR has gotten stale - do you have the time to finalize it?

sure I'll try to finalize this

[isaac-chung]

@mina-parham this dataset does not seem to work. mteb run -t AESDD -m facebook/wav2vec2-base gives

  File "/opt/homebrew/anaconda3/envs/mteb/lib/python3.11/site-packages/datasets/utils/file_utils.py", line 643, in get_from_cache
    raise FileNotFoundError(f"Couldn't find file at {url}")
FileNotFoundError: Couldn't find file at https://drive.google.com/uc?export=download&id=1-pelMaCrfwoUCmwxUtlacRUBwbFnXlXA&confirm=t

[mina-parham]

@mina-parham this dataset does not seem to work. mteb run -t AESDD -m facebook/wav2vec2-base gives

  File "/opt/homebrew/anaconda3/envs/mteb/lib/python3.11/site-packages/datasets/utils/file_utils.py", line 643, in get_from_cache
    raise FileNotFoundError(f"Couldn't find file at {url}")
FileNotFoundError: Couldn't find file at https://drive.google.com/uc?export=download&id=1-pelMaCrfwoUCmwxUtlacRUBwbFnXlXA&confirm=t

The error I’m getting on my side is:

ValueError: The repository for EdwardLin2023/AESDD contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/EdwardLin2023/AESDD.
Please pass the argument trust_remote_code=True to allow custom code to be run.

So even after adding trust_remote_code=True, I’m still getting the same error.
Do you know how to fix this?

[isaac-chung]

The error I’m getting on my side is:
ValueError: The repository for EdwardLin2023/AESDD contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/EdwardLin2023/AESDD.
Please pass the argument trust_remote_code=True to allow custom code to be run.
So even after adding trust_remote_code=True, I’m still getting the same error.
Do you know how to fix this?

Have you tried make install to install the repo in edit mode? I still get this error with your latest changes.