Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/porting/translate/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@ You can create an overwrite file to manually set the threshold in you data direc

![image1.png](../../images/translate/image1.png)

### Testing with Custom Data

The default netcdf names can be overwritten by setting `self.override_input_netcdf_name` and/or `self.override_output_netcdf_name`
for the input and output netcdf, respectively. These new names should be the entire file name (including, but not requiring,
-In or -Out suffixes), excluding the `.nc` file tag. These files **must** be located in the same folder as the default files.

### Overwriting Arguments to your compute function

The compute_func will be called automatically in the test. If your names in the netcdf are matching the `kwargs` of your function directly, no further action required:
Expand Down
51 changes: 49 additions & 2 deletions ndsl/stencils/testing/test_translate.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,28 @@ def test_sequential_savepoint(
return
if not case.exists:
pytest.skip(f"Data at rank {case.grid.rank} does not exist.")
input_data = dataset_to_dict(case.ds_in)

if hasattr(case.testobj, "override_input_netcdf_name"):
import xarray as xr

from ndsl.logging import ndsl_log

out_data = (
xr.open_dataset(
os.path.join(
case.data_dir, f"{case.testobj.override_input_netcdf_name}.nc"
)
)
.isel(rank=case.grid.rank)
.isel(savepoint=case.i_call)
)
input_data = dataset_to_dict(out_data)
ndsl_log.warning(
f"You are loading {case.testobj.override_input_netcdf_name} as a custom input file! Here be dragons."
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the warning for here? Are there specific issues users should be aware of?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to remind people that they are using non-standard options and to proceed with caution.

)
else:
input_data = dataset_to_dict(case.ds_in)

input_names = (
case.testobj.serialnames(case.testobj.in_vars["data_vars"])
+ case.testobj.in_vars["parameters"]
Expand All @@ -196,7 +217,28 @@ def test_sequential_savepoint(
output = case.testobj.compute(input_data)
failing_names: List[str] = []
passing_names: List[str] = []
all_ref_data = dataset_to_dict(case.ds_out)
if hasattr(case.testobj, "override_output_netcdf_name"):
import xarray as xr

from ndsl.logging import ndsl_log

out_data = (
xr.open_dataset(
os.path.join(
case.data_dir, f"{case.testobj.override_output_netcdf_name}.nc"
)
)
.isel(rank=case.grid.rank)
.isel(savepoint=case.i_call)
)

output_data = dataset_to_dict(out_data)
ndsl_log.warning(
f"You are loading {case.testobj.override_output_netcdf_name} as a custom output file! Here be dragons."
)
else:
output_data = dataset_to_dict(case.ds_out)
all_ref_data = output_data
ref_data_out = {}
results = {}

Expand Down Expand Up @@ -460,6 +502,11 @@ def _save_datatree(
varname = names[index]
# Read in dimensions and attributes
if hasattr(testobj, "outputs") and testobj.outputs != {}:
if not isinstance(testobj.outputs, dict):
raise ValueError(
f"Expecting `outputs` on translate test to be a dict, got {type(testobj.outputs)}."
" Are you overriding `self.outputs`?"
)
dims = [
dim_name + f"_{index}" for dim_name in testobj.outputs[varname]["dims"]
]
Expand Down