add AbsTaskSpectralClustering

[OnAnd0n]

Code Quality

• Code Formatted: Format the code using make lint to maintain consistent style.

Documentation

• Updated Documentation: Add or update documentation to reflect the changes introduced in this PR.

Testing

• New Tests Added: Write tests to cover new functionality. Validate with make test-with-coverage.
• Tests Passed: Run tests locally using make test or make test-with-coverage to ensure no existing functionality is broken.

Adding datasets checklist

Reason for dataset addition: ...

• I have run the following models on the task (adding the results to the pr). These can be run using the mteb -m {model_name} -t {task_name} command.
  • sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
  • intfloat/multilingual-e5-small
• I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
• If the dataset is too big (e.g. >2048 examples), considering using self.stratified_subsampling() under dataset_transform()
• I have filled out the metadata object in the dataset file (find documentation on it here).
• Run tests locally to make sure nothing is broken using make test.
• Run the formatter to format the code using make lint.

[OnAnd0n]

@Samoed @KennethEnevoldsen

I have implemented Spectral Clustering to reflect cosine similarity, as we previously discussed.
question regarding clustering evaluation in MTEB
Initially, you suggested the following format:

class AbsTaskSpectralClustering(AbsTaskClusteringFast)
     clustering_model = sklearn.cluster.spectralclustering     
     ...

# and

class AbsTaskCosineClustering(AbsTaskClusteringFast)
     clustering_model = ...  
     ...

However, I noticed that the create_task_list() function collects task categories using a two-level iteration.
If a three-level structure(AbsTaskSpectralClustering -> AbsTaskClusteringFast -> AbsTask) is introduced,
tasks might not be properly collected into Task_list.

def create_task_list() -> list[type[AbsTask]]:
    tasks_categories_cls = list(AbsTask.__subclasses__())
    tasks = [
        cls
        for cat_cls in tasks_categories_cls
        for cls in cat_cls.__subclasses__()
        if cat_cls.__name__.startswith("AbsTask")
    ]
    return tasks

Could you review whether this approach is reasonable and applicable?
Thank you for your time.

[Samoed]

This should be moved to evaluators

[Samoed]

This will be removed in 2.0

Suggested change

limit: int | None = None,

[Samoed]

This will be removed in 2.0

Suggested change

	if limit is not None:
	sentences = sentences[:limit]
	labels = labels[:limit]

[Samoed]

I think you clustering is almost 1to1 to original clustering. Maybe it would be better to move evaluator to properties of task and your tasks will use

        for cluster_set in tqdm.tqdm(dataset, desc="Clustering"):
            evaluator = self.evaluator(

class Task(AbsClustering):
    evaluator = SpectralClusteringEvaluator

[OnAnd0n]

@Samoed
Thanks for your advice!

I will revise it again at 'evaluation' level (if it is deemed meaningful).
Additionally, I will apply try/except as well.

[KennethEnevoldsen]

We should build this on the fast clustering task, not AbsTaskClustering (it is much slower and gives less consistent estimates)

[Samoed]

Need to convert with try... ecept I think this should be moved inside __call__

[KennethEnevoldsen]

@OnAnd0n will close this PR as it seems to have gotten stale - do feel free to reopen if you have the time to adress the comments