Restructure Retrieval benchmarks: rename, clean slugs, add Miscellaneous, and relocate tasks

[q275343119]

This PR makes the following adjustments to the Retrieval benchmark structure:

1. Rename “RTEB (Retrieval)” to “Retrieval”.
2. Remove the “RTEB” slug from sub-benchmarks (e.g., change “RTEB Finance” → “Finance”).
3. Add a new Miscellaneous category under Retrieval.
4. Move the following benchmarks from MTEB/Miscellaneous to Retrieval/Miscellaneous:
   • BEIR
   • NanoBEIR
   • BRIGHT
   • BRIGHT (long)
   • Codenformation Retrieval
   • Instruction Following
   • Long-context Retrieval
   • Reasoning Retrieval

Preview space: https://huggingface.co/spaces/SmileXing/leaderboard

[Samoed]

I think this can be a bit confusing, because we have other language specific datasets

[q275343119]

Oh, I see — two different English buttons are displaying the same content.
After looking into it, I found that the issue comes from this part of the code:

def _create_button(
    i: int,
    benchmark: Benchmark,
    state: gr.State,
    label_to_value: dict[str, str],
    **kwargs,
):
    val = benchmark.name
    label = (
        benchmark.display_name if benchmark.display_name is not None else benchmark.name
    )
    label_to_value[label] = benchmark.name
    button = gr.Button(
        label,
        variant="secondary" if i != 0 else "primary",
        icon=benchmark.icon,
        key=f"{i}_button_{val}",
        elem_classes="text-white",
        **kwargs,
    )

Since label_to_value is a dict, assigning with the same key will overwrite the previous value whenever two buttons share the same label.

So...I could not use the same display_name in the Benchmark 😥

[Samoed]

I think we should add test for this

[fzliu]

I think this can be a bit confusing, because we have other language specific datasets

Good point; I can see arguments for either direction. @KennethEnevoldsen what do you think?

[KennethEnevoldsen]

Hmm, so visually, I don't see a big issue with having two "English" benchmarks, as we have the structure (especially with the change below)

Whether this is possible in the code, I'm not sure, though. I will have to run a test on that Monday.

[KennethEnevoldsen]

(will get back to this Monday to test the buttons)

[KennethEnevoldsen]

We might need to rename "select benchmark" to "General Purpose"

[KennethEnevoldsen]

Hmm, so visually, I don't see a big issue with having two "English" benchmarks, as we have the structure (especially with the change below)

Whether this is possible in the code, I'm not sure, though. I will have to run a test on that Monday.

[q275343119]

I added blank after display_name to prevent duplicate labels from appearing.

[KennethEnevoldsen]

Since I can't update this branch, I made a separate PR here #3222 @q275343119, can you review it to see if it works for you?

[q275343119]

Since I can't update this branch, I made a separate PR here #3222 @q275343119, can you review it to see if it works for you?

It works for me.
Adding the suffix is a nice idea— no problem on my side.

[KennethEnevoldsen]

great I will close this then in favor of the other PR