Skip to content

Set up the reject list in observation chronicle for GTS synoptic snow depth data#79

Merged
CoryMartin-NOAA merged 7 commits into
developfrom
feature/snow_add_rejectlist
Mar 5, 2025
Merged

Set up the reject list in observation chronicle for GTS synoptic snow depth data#79
CoryMartin-NOAA merged 7 commits into
developfrom
feature/snow_add_rejectlist

Conversation

@jiaruidong2017
Copy link
Copy Markdown
Contributor

This PR takes the branch feature/snow-example-chronicle from @CoryMartin-NOAA and adds a initial list of the rejected station IDs in observation chronicle for GTS synoptic snow depth data by using NOAA-EMC/jcb#17
This PR also adds/removes additional stations chronologically in the observation chronicle.

This PR needs JCB updated to the commit hash b114fe43 or later.

Copy link
Copy Markdown

@ClaraDraper-NOAA ClaraDraper-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. For each yaml, can you please add a comment that we can use to track the station list that has been excluded. At the very least, include today's date.

@jiaruidong2017
Copy link
Copy Markdown
Contributor Author

Looks good to me. For each yaml, can you please add a comment that we can use to track the station list that has been excluded. At the very least, include today's date.

Good point. Could we just add the comments showing the latest updated dates (currently 2022-12-31)? If we update to 2024-12-31 later, we can make further changes. Thanks.

@ClaraDraper-NOAA
Copy link
Copy Markdown

Looks good to me. For each yaml, can you please add a comment that we can use to track the station list that has been excluded. At the very least, include today's date.

Good point. Could we just add the comments showing the latest updated dates (currently 2022-12-31)? If we update to 2024-12-31 later, we can make further changes. Thanks.

Yeah - I think the date is enough. Thanks.

@CoryMartin-NOAA
Copy link
Copy Markdown
Contributor

I'm confused on the point about the comments, the point of the chronicle is to track these, what do we gain from adding a date comment to the template YAMLs?

@TravisElless-NOAA
Copy link
Copy Markdown
Contributor

I understood the comment request to be like a github history in a comment. (i.e., station 71120 was committed 2-26-25). Then as we work back in time and add additional stations there would be a comment that this station action was created on xx commit date. While going forward the chronicle will capture the future actions on the same date, but looking back for the reanalysis the decision date about the action is a little less clear

Comment thread observation_chronicle/snow/adpsfc_snow.yaml Outdated
@jiaruidong2017
Copy link
Copy Markdown
Contributor Author

I'm confused on the point about the comments, the point of the chronicle is to track these, what do we gain from adding a date comment to the template YAMLs?

Sorry for causing the confusion. Because we created the reject list just from the data over 2021 and 2022, it should be valid for the current reject list used in this period, and we should be cautious if we apply them beyond these time period. When we verify the reject list with other years, we can extend the time or remove the time tags.

@ClaraDraper-NOAA
Copy link
Copy Markdown

I'm confused on the point about the comments, the point of the chronicle is to track these, what do we gain from adding a date comment to the template YAMLs?

Sorry for causing the confusion. Because we created the reject list just from the data over 2021 and 2022, it should be valid for the current reject list used in this period, and we should be cautious if we apply them beyond these time period. When we verify the reject list with other years, we can extend the time or remove the time tags.

I talked with @CoryMartin-NOAA about this today. He made the point that chronicle will be designed to use git version control for tracking versions, so we don't need to add comments like dates like I asked for - so please ignore that. I think the real issue I was worried about was that there's no useful info on the decisions behind each entry. The better way to fix that would be to put more useful info into the "justification" entry when rejecting stations - either describing the reason these stations are being rejected ("unit problem", "unrealistic values", "stuck alues", etc) OR (say if there are multiple reasons) describing the process used to generate this list ("initial pass for unrealistic stations", "annual review of O-F stats", etc). Does that make sense?

@jiaruidong2017
Copy link
Copy Markdown
Contributor Author

I'm confused on the point about the comments, the point of the chronicle is to track these, what do we gain from adding a date comment to the template YAMLs?

Sorry for causing the confusion. Because we created the reject list just from the data over 2021 and 2022, it should be valid for the current reject list used in this period, and we should be cautious if we apply them beyond these time period. When we verify the reject list with other years, we can extend the time or remove the time tags.

I talked with @CoryMartin-NOAA about this today. He made the point that chronicle will be designed to use git version control for tracking versions, so we don't need to add comments like dates like I asked for - so please ignore that. I think the real issue I was worried about was that there's no useful info on the decisions behind each entry. The better way to fix that would be to put more useful info into the "justification" entry when rejecting stations - either describing the reason these stations are being rejected ("unit problem", "unrealistic values", "stuck alues", etc) OR (say if there are multiple reasons) describing the process used to generate this list ("initial pass for unrealistic stations", "annual review of O-F stats", etc). Does that make sense?

Thanks @ClaraDraper-NOAA for the updates, and I just remove the comments for now. I totally agree with you that it is important to track the reasons for the stations being rejected. Let's talk this in our tomorrow's meeting.

@CoryMartin-NOAA
Copy link
Copy Markdown
Contributor

I'm going to merge this as-is, and we can update with more descriptive information later as we see fit

@CoryMartin-NOAA CoryMartin-NOAA merged commit 5426da4 into develop Mar 5, 2025
@CoryMartin-NOAA CoryMartin-NOAA deleted the feature/snow_add_rejectlist branch March 5, 2025 16:14
DavidNew-NOAA pushed a commit that referenced this pull request Mar 5, 2025
… depth data (#79)

This PR takes the branch `feature/snow-example-chronicle` from
@CoryMartin-NOAA and adds a initial list of the rejected station IDs in
observation chronicle for GTS synoptic snow depth data by using
NOAA-EMC/jcb#17
This PR also adds/removes additional stations chronologically in the
observation chronicle.

This PR needs JCB updated to the commit hash `b114fe43` or later.
danholdaway added a commit that referenced this pull request Mar 26, 2025
* develop:
  Add sample conventional surface pressure template for generating summary stats (#85)
  Revert "Add aerosol bias correction to yaml templates" (#84)
  JEDI-based ensemble recentering and analysis calculation and native grid increments (#72)
  Add aerosol bias correction to yaml templates (#83)
  Set up the reject list in observation chronicle for GTS synoptic snow depth data (#79)
  Add separate salt and temp configs for argo (#78)
  Add marine j2 yamls for tropical moorings and surface drifters (#81)
  Update for missing snow files (#76)
  Quick Filename Fix for Stats YAMLs (#80)
  Remove the ens B from the marine hybrid B (#73)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants