New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replicate DEMETR results for BLEU #8

Open

3 tasks done

klh5 opened this issue Nov 27, 2024 · 1 comment · May be fixed by #10

Open

3 tasks done

Replicate DEMETR results for BLEU #8

klh5 opened this issue Nov 27, 2024 · 1 comment · May be fixed by #10

Assignees

Labels

Collaborator

klh5 commented Nov 27, 2024 •

edited

Loading

As a first step and sanity check, we should attempt to replicate the BLEU result from the DEMETR paper.

Download and investigate dataset (would be good to get some basic stats)
Apply BLEU to (Ref, MT) and (Ref, Pert) pairs
Obtain accuracy metrics

klh5 added the prototyping label

klh5 self-assigned this

klh5 linked a pull request

that will close this issue

8 replicate demetr results for bleu #10

Open

klh5 linked a pull request

that will close this issue

8 replicate demetr results for bleu #10

Open

Collaborator Author

klh5 commented Dec 6, 2024

Add COMET metrics

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment