Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for float16 precision support when running translate-* tasks #936

Open
gregtatum opened this issue Nov 20, 2024 · 0 comments
Open
Labels
on-prem Running the pipeline on-premises machines

Comments

@gregtatum
Copy link
Member

From #934.

One downside of moving [precision and batch sizing] to the [pipeline/translate/decoder.yml] config is that some GPUs on Berlin cluster that we plan to include soon don't support half precision decoding. And if I remember correctly it fails silently and produces bad results on decoding, so it's quite dangerous. So, this setting should be likely propagated through marian args or there should be a detection mechanism in the script.

In the translate script we could check for precision support. Maybe there is some kind of heuristic that could be written to determine the batch size as well based on GPU capacity.

@gregtatum gregtatum added the on-prem Running the pipeline on-premises machines label Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
on-prem Running the pipeline on-premises machines
Projects
None yet
Development

No branches or pull requests

1 participant