Skip to content

add ritrieve_zh_v1 cmteb evaluation results#162

Merged
Samoed merged 1 commit intoembeddings-benchmark:mainfrom
richinfo-ai:add_cmteb_results
Mar 25, 2025
Merged

add ritrieve_zh_v1 cmteb evaluation results#162
Samoed merged 1 commit intoembeddings-benchmark:mainfrom
richinfo-ai:add_cmteb_results

Conversation

@richinfo-ai
Copy link
Contributor

@richinfo-ai richinfo-ai commented Mar 25, 2025

add ritrieve_zh_v1 cmteb evaluation results

Checklist

  • Run tests locally to make sure nothing is broken using make test.
  • Run the results files checker make pre-push.

Adding a model checklist

add ritrieve_zh_v1 cmteb evaluation results
@Samoed Samoed enabled auto-merge (squash) March 25, 2025 13:31
@richinfo-ai
Copy link
Contributor Author

As shown in https://github.com/embeddings-benchmark/mteb/blob/main/docs/reproducible_workflow.md, You can reproduce cmteb results by the following codes:

import mteb

if __name__ == "__main__":
    output_folder = "./mteb_zh_results"
    model = mteb.get_model("richinfoai/ritrieve_zh_v1", revision="f8d5a707656c55705027678e311f9202c8ced12c")
    evaluation = mteb.MTEB(tasks=list(mteb.get_benchmark("MTEB(cmn, v1)")))
    evaluation.run(
        model,
        output_folder=output_folder,
        verbosity=3,
        overwrite_results=False,
        encode_kwargs={"normalize_embeddings": True}
    )

@Samoed Samoed merged commit faecdd2 into embeddings-benchmark:main Mar 25, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants