Fix failing test with Dask 2025.2: ours issue not theirs#2663
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2663 +/- ##
==========================================
- Coverage 95.08% 95.07% -0.01%
==========================================
Files 255 255
Lines 14884 14884
==========================================
- Hits 14152 14151 -1
- Misses 732 733 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
I don't think this is bug in Dask, but rather a problem with our test. We have the following setup: mask = np.ma.masked_array(np.arange(2), mask=[1, 0]) # [-- 1]
array = da.arange(2) # [0, 1]Applying this mask to the array gives expected = da.ma.masked_array(np.ones(2), np.arange(2)) # [1.0, --]which is clearly different to the actual result. So, I think we should adapt our test, not pin Dask. I have no idea why this wasn't a test fail before. Most likely Dask fixed a bug in v2025.2.0... Test code (gives always the same result, regardless of Dask version; also gives same result for pure-numpy code): import dask.array as da
import numpy as np
mask = np.ma.masked_array(np.arange(2), mask=[1, 0])
array = da.arange(2)
x = da.ma.masked_where(mask, array)
print("mask", mask)
print("array", array.compute())
print("result", x.compute())
print("result data", x.compute().data)
print("result mask", x.compute().mask) |
|
Test ran successful with Dask 2025.2.0 here and with 2025.1.0 locally 🎉 |
|
very many thanks for looking into this, Manu! The test is failing with dask==2025.2 but not with dask==2025.1 so we have to be a wee bit careful about it. Let me test your MRE myself, and thanks for including it! |
|
Yes, but with this PR it runs in both Dask versions. As I mentioned above, it rather seems that there was a bug in <=2025.1.0 which let the test pass even though the expected result was clearly wrong (looks like a typo to me). This PR here fixes the expected result of our test. |
valeriupredoi
left a comment
There was a problem hiding this comment.
many thanks, Manu! Indeed - sanity checks all good:
dask==2025.1
mask [-- 1]
array [0 1]
result [-- --]
result data [0 1]
result mask [ True True]
dask==2025.2
mask [-- 1]
array [0 1]
result [-- --]
result data [0 1]
result mask [ True True]
dask 2024.8.0
mask [-- 1]
array [0 1]
result [-- --]
result data [0 1]
result mask [ True True]
|
fixed PR title, and description, @schlunma cheers muchly for your detective work here, pls merge at will 🍺 |
Co-authored-by: Manuel Schlund <manuel.schlund@dlr.de>
Co-authored-by: Manuel Schlund <manuel.schlund@dlr.de>
Description
@sloosvel and I found a failing test with Dask==2025.2 in conda-forge/esmvalcore-feedstock#83 while we were trying to release RC1. @schlunma looked into it here, and concluded the test (ours) needs be fixed, so he's applied the fix - many thanks, Manu!
Before you get started
Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.
To help with the number pull requests: