Multilingual NLI Tasks

[hynky1999]

Goal

Add 3 NLI tasks supporting 26 unique languages.

• xnli https://aclanthology.org/D18-1269/
• xnli2.0 https://arxiv.org/abs/2301.06527
• indic_xnli https://arxiv.org/abs/2204.08776
• cmnli + ocnli https://arxiv.org/abs/2004.05986
• rcb https://arxiv.org/abs/2401.04531

While the xnli2.0 is superior I decided to keep xnli, as some people might want to use it.
Since it uses template it support all 3 types of formulation out of the box :)

How to test:

lighteval accelerate --output_dir=./tmp --custom_tasks="lighteval.tasks.multilingual.tasks" --tasks="custom|{task}|0|0" --model_args=pretrained=gpt2 --override_batch_size=1 --max_samples=100 --save_details

where task in indicnxnli_tel_cf xnli_en_cf xnli2.0_en_cf ocnli_zho_cf cmnli_zho_cf rcb_rus_cf

Comments

• We talked about removing the suite all together. Since right now we use suite I decided to use custom suite. I can switch to multilingual or whatever

[clefourrier]

Lgtm, feel free to add a bit more doc (for example, the above arxiv links to the corresponding classes). Suite should also be lighteval for now since we're adding them to the core

[clefourrier]

Could be nice to add just a bit of intro doc at the top of the file to explain what these tasks are overall about (= what is NLI, which datasets are used, etc)