-
Notifications
You must be signed in to change notification settings - Fork 572
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Different loss values for seemingly same forecast #32
Comments
One confusing thing about the |
Hey Timo, |
I'm not sure why this was chosen as the default, but sometimes you want to know how your error changes as you predict the same time from different points in the past. There should probably be an option for choosing left vs right alignment. @alvarosg may have more context here. |
The step-04.nc should be a subset of step-40.nc, I think with the same initial time. That should be pretty easy to verify by loading both and looking at the data. Then just make sure you're extracting the data correctly for your use case. Feel free to send a PR with a left/right alignment option. |
Ok. Yep. I see how it picks out the inputs and targets now. I'll make a few changes and try to submit a pull request this week. Thanks! |
I performed "1 Eval Step" forecast with Graphcast small using dataset_source-era5_date-2022-01-01_res-1.0_levels-13_steps-01.nc, steps-12.nc, and steps-40.nc.
The loss is different for each case even though we are only performing a 6 hr forecast in each case. Why might this be? As I understand it, the prediction should be the same in all these cases and the target should also be the same (ERA5 Reanalysis 6hr into the future).
Loss Values:
0.9296875 for 6hr forecast 1-step data 01 Jan 2022
0.69140625 for 6hr forecast 12-step data 01 Jan 2022
0.66015625 for 6hr forecast 40-step data 01 Jan 2022
The text was updated successfully, but these errors were encountered: