[WIP][Torch] Seeds / Determinism #361
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #361 +/- ##
==========================================
- Coverage 89.45% 89.42% -0.03%
==========================================
Files 57 57
Lines 3918 3926 +8
==========================================
+ Hits 3505 3511 +6
- Misses 413 415 +2 ☔ View full report in Codecov by Sentry. |
VivekPanyam
force-pushed
the
torch_deterministic
branch
from
d099945
to
559877d
Compare
|
@selitvin Should we generalize these options more before landing or do you think it's fine to have framework-specific options in |
| @@ -233,6 +233,9 @@ TorchNeuropodBackend::TorchNeuropodBackend(const std::string &neuropod_path, con | |||
|
|
|||
| void TorchNeuropodBackend::load_model_internal() | |||
| { | |||
| at::globalContext().setDeterministicCuDNN(options_.torch_cudnn_deterministic); | |||
There was a problem hiding this comment.
Should we avoid making these calls all-together is a user opted out to reduce a chance of modifying the default behavior?
VivekPanyam
force-pushed
the
torch_deterministic
branch
from
559877d
to
8a6ad22
Compare
| // Whether or not to run in deterministic mode. See https://pytorch.org/docs/stable/notes/randomness.html#cudnn | ||
| // Note: this currently only applies to TorchScript models and affects all torchscript models in the process. | ||
| // Should only be used with OPE to avoid this issue. | ||
| bool torch_cudnn_deterministic = false; |
There was a problem hiding this comment.
My personal perferences:
Even we know that this is for Torch only for now, I would call it: backends_cudnn_deterministic and backends_cudnn_benchmark. And then comment specifies that currently this is Torch only. So, we are going to add more options probably in future and it is better to keep them generic and name accordingly (if only 1 backend supports it - fine, but it helps to see gaps/difference between backends and even can be a start point to add options to other framework.
This PR implements Torch specific seed and determinism options.
Closes #325