Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Metrics] ROUGE #51

Closed
justusschock opened this issue Mar 30, 2020 · 7 comments · Fixed by #399
Closed

[Metrics] ROUGE #51

justusschock opened this issue Mar 30, 2020 · 7 comments · Fixed by #399
Assignees
@karthikrangasai
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed New metric topic: Text

Comments

@justusschock
Copy link
Member

🚀 Feature

Implement ROUGE
@Borda Borda changed the title [Metrics Package] ROUGE Metrics: ROUGE Mar 30, 2020
@edenlightning edenlightning changed the title Metrics: ROUGE [Metrics] ROUGE Jun 18, 2020
@Borda Borda transferred this issue from Lightning-AI/pytorch-lightning Mar 12, 2021
@Borda Borda added enhancement New feature or request help wanted Extra attention is needed labels Mar 17, 2021
@carmocca
Copy link
Contributor

@SeanNaren you could take this at some point, as you already have a working implementation relying on https://pypi.org/project/rouge-score/

@stale stale bot added the wontfix label Jun 1, 2021
@stale stale bot closed this as completed Jun 10, 2021
@carmocca
Copy link
Contributor

https://github.com/PyTorchLightning/lightning-transformers/blob/1c5c91f5b4962f9162bcbd41fee7a7ac5eae00a9/lightning_transformers/task/nlp/summarization/metric.py

@Borda Borda reopened this Jul 7, 2021
@stale stale bot removed the wontfix label Jul 7, 2021
@SkafteNicki
Copy link
Member

Copy implementation from here:
https://github.com/PyTorchLightning/lightning-flash/blob/master/flash/text/seq2seq/summarization/metric.py

@gagan3012 gagan3012 mentioned this issue Jul 13, 2021
Closed
4 tasks
@randolph-john
Copy link

A word of caution, the PyTorchLightning implementation of Rouge does not give the same results as the original PERL script. See here: google-research/google-research#239. We also encountered this issue internally at Facebook.

@Lightning-AI Lightning-AI deleted a comment from stale bot Jul 14, 2021
@Lightning-AI Lightning-AI deleted a comment from stale bot Jul 14, 2021
@Borda
Copy link
Member

Borda commented Jul 16, 2021

@ananyahjha93 any update here? 🐰

@SkafteNicki
Copy link
Member

@Borda if I am not wrong, @karthikrangasai would like to work on this issue when his Bleu PR has landed

@karthikrangasai
Copy link
Contributor

@SkafteNicki yeah sure. I would like to work on this issue.

@karthikrangasai karthikrangasai mentioned this issue Jul 23, 2021
Merged
4 tasks
@karwojan karwojan mentioned this issue May 23, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@karthikrangasai karthikrangasai

Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed New metric topic: Text
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add Rouge score karthikrangasai/metrics
7 participants
@Borda @ananyahjha93 @carmocca @justusschock @SkafteNicki @randolph-john @karthikrangasai