-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue with SemiValue batching and parallelization #490
Comments
@kosmitive could you have a look at this if you have time? |
Yes, it might be related to the parallelization as due to parallel processing, the order in which numbers arrive might be prone to a racing condition. I recall this occurs only for semivalues as we break down the calculation to single marginals. Or do you think it might be a different problem? For the desribed problem, we could introduce a order resolver on the main thread, but at the cost of blowing up RAM on average of about |
@AnesBenmerzoug Here is the reference which was made in the tests pyDVL/tests/value/test_semivalues.py Lines 228 to 229 in 7c003be
|
@kosmitive I changed that test to use a deterministic scoring method coming from a toy game, so the order of batches shouldn't have an effect on the final result. |
Potentially resolved by #558 |
While working on PR #341, I realized that there is a bug in the batching feature of semivalues when using
n_jobs
> 1.The results are almost the same but not exactly the same.
The text was updated successfully, but these errors were encountered: