Skip to content

Follow-up modifications for GFS post process #2467

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
WenMeng-NOAA wants to merge 9 commits into from
Closed

Follow-up modifications for GFS post process #2467

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
4e4c108
Follow-up fixes for gfs post process.
WenMeng-NOAA Apr 8, 2024
49760d2
Remove commented lines
WenMeng-NOAA Apr 8, 2024
9d23951
Update flux file naming convention
WenMeng-NOAA Apr 8, 2024
4c7f570
More updates in flux file name in hpssarch_gen.sh.
WenMeng-NOAA Apr 8, 2024
8f0367e
Merge remote-tracking branch 'upstream/develop' into post_gfs
WenMeng-NOAA Apr 8, 2024
afd97d1
Update with Rahul's comments
WenMeng-NOAA Apr 9, 2024
e92e404
further modifications
WenMeng-NOAA Apr 9, 2024
9d6b6b8
Update naming conventions per Walter's comments
WenMeng-NOAA Apr 10, 2024
c0c9541
Merge branch 'develop' into post_gfs
aerorahul Apr 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion parm/product/gfs.anl.paramlist.a.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,6 @@ HLCY:3000-0 m above ground
ICAHT:max wind
ICAHT:tropopause
ICETK:surface
ICETMP:surface
ICMR:50 mb
ICMR:1000 mb
ICMR:100 mb
Expand Down
2 changes: 0 additions & 2 deletions parm/product/gfs.f000.paramlist.a.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,6 @@ ICAHT:max wind
ICAHT:tropopause
ICEC:surface
ICETK:surface
ICETMP:surface
ICMR:50 mb
ICMR:1000 mb
ICMR:100 mb
Expand Down Expand Up @@ -697,4 +696,3 @@ WEASD:surface
WILT:surface
SNOD:surface
VIS:surface
ICEG:10 m above mean sea level
3 changes: 0 additions & 3 deletions parm/product/gfs.fFFF.paramlist.a.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ ACPCP:surface
ALBDO:surface
APCP:surface
APTMP:2 m above ground
var discipline=0 master_table=2 parmcat=0 parm=21:2 m above ground
AOTK:entire atmosphere
CAPE:180-0 mb above ground
CAPE:255-0 mb above ground
Expand Down Expand Up @@ -177,7 +176,6 @@ ICAHT:max wind
ICAHT:tropopause
ICEC:surface
ICETK:surface
ICETMP:surface
ICSEV:100 mb
ICSEV:150 mb
ICSEV:200 mb
Expand Down Expand Up @@ -758,4 +756,3 @@ WEASD:surface
WILT:surface
SNOD:surface
VIS:surface
ICEG:10 m above mean sea level
4 changes: 2 additions & 2 deletions scripts/exglobal_atmos_products.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ fi
# Section creating pressure grib2 interpolated products

# Files needed by ${INTERP_ATMOS_MASTERSH}
MASTER_FILE="${COM_ATMOS_MASTER}/${PREFIX}master.grb2${fhr3}"
MASTER_FILE="${COM_ATMOS_MASTER}/${PREFIX}master${fhr3}.grb2"

@aerorahul aerorahul Apr 9, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WenMeng-NOAA
The workflow looks for the master file to kick off the products generation job.
Please update this line to:

-                                   'history_file_tmpl': f'{self.cdump}.t@Hz.master.grb2#dep#'},
+                                   'history_file_tmpl': f'{self.cdump}.t@Hz.master#dep#.grb2'},

The analysis products job trigger also needs to be updated. Please update this line to:

-        data = f'{atm_master_path}/{self.cdump}.t@Hz.master.grb2anl'
+        data = f'{atm_master_path}/{self.cdump}.t@Hz.masteranl.grb2'

Finally, the offline UPP for analysis needs to write out the updated filename to COM. Please update these lines with:

-      - ["{{ DATA }}/GFSPRS.GrbF00", "{{ COM_ATMOS_MASTER }}/{{ RUN }}.t{{ current_cycle | strftime('%H') }}z.master.grb2anl"]
-      - ["{{ DATA }}/GFSPRS.GrbF00.idx", "{{ COM_ATMOS_MASTER }}/{{ RUN }}.t{{ current_cycle | strftime('%H') }}z.master.grb2ianl"]
+      - ["{{ DATA }}/GFSPRS.GrbF00", "{{ COM_ATMOS_MASTER }}/{{ RUN }}.t{{ current_cycle | strftime('%H') }}z.masteranl.grb2"]
+      - ["{{ DATA }}/GFSPRS.GrbF00.idx", "{{ COM_ATMOS_MASTER }}/{{ RUN }}.t{{ current_cycle | strftime('%H') }}z.masteranl.grb2.idx"]

@aerorahul aerorahul Apr 9, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note more updates will be needed in the hpssarch_gen.sh for these changes in analysis filenames.

@WenMeng-NOAA WenMeng-NOAA Apr 9, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will work on that.

@aerorahul aerorahul Apr 9, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WenMeng-NOAA
I just looked at the EE2 implementation standards and the correct extension for grib2 files is grib2. We should replace grb2 with grib2.

@WenMeng-NOAA WenMeng-NOAA Apr 9, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul Just confirm with you that updates will be in all master/flux/goes/pgrb2/pgrb2b.

@WenMeng-NOAA WenMeng-NOAA Apr 9, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul Updated.


# Get inventory from ${MASTER_FILE} that matches patterns from ${paramlista}
# Extract this inventory from ${MASTER_FILE} into a smaller tmpfile or tmpfileb based on paramlista or paramlistb
Expand Down Expand Up @@ -172,7 +172,7 @@ done # for (( nset=1 ; nset <= downset ; nset++ ))
#---------------------------------------------------------------

# Create the index file for the sflux master, if it exists.
FLUX_FILE="${COM_ATMOS_MASTER}/${PREFIX}sfluxgrb${fhr3}.grib2"
FLUX_FILE="${COM_ATMOS_MASTER}/${PREFIX}sflux${fhr3}.grb2"
if [[ -s "${FLUX_FILE}" ]]; then
${WGRIB2} -s "${FLUX_FILE}" > "${FLUX_FILE}.idx"
fi
Expand Down
4 changes: 2 additions & 2 deletions ush/forecast_postdet.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -479,8 +479,8 @@ EOF
${NLN} "${COM_ATMOS_HISTORY}/${RUN}.t${cyc}z.sfcf${FH3}.nc" "sfcf${FH3}.nc"
${NLN} "${COM_ATMOS_HISTORY}/${RUN}.t${cyc}z.atm.logf${FH3}.txt" "log.atm.f${FH3}"
if [[ ${WRITE_DOPOST} = ".true." ]]; then
${NLN} "${COM_ATMOS_MASTER}/${RUN}.t${cyc}z.master.grb2f${FH3}" "GFSPRS.GrbF${FH2}"
${NLN} "${COM_ATMOS_MASTER}/${RUN}.t${cyc}z.sfluxgrbf${FH3}.grib2" "GFSFLX.GrbF${FH2}"
${NLN} "${COM_ATMOS_MASTER}/${RUN}.t${cyc}z.masterf${FH3}.grb2" "GFSPRS.GrbF${FH2}"
${NLN} "${COM_ATMOS_MASTER}/${RUN}.t${cyc}z.sfluxf${FH3}.grb2" "GFSFLX.GrbF${FH2}"

@aerorahul aerorahul Apr 9, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
${NLN} "${COM_ATMOS_MASTER}/${RUN}.t${cyc}z.masterf${FH3}.grb2" "GFSPRS.GrbF${FH2}"
${NLN} "${COM_ATMOS_MASTER}/${RUN}.t${cyc}z.sfluxf${FH3}.grb2" "GFSFLX.GrbF${FH2}"
${NLN} "${COM_ATMOS_MASTER}/${RUN}.t${cyc}z.masterf${FH3}.grib2" "GFSPRS.GrbF${FH2}"
${NLN} "${COM_ATMOS_MASTER}/${RUN}.t${cyc}z.sfluxf${FH3}.grib2" "GFSFLX.GrbF${FH2}"

fi
done
else # TODO: Is this even valid anymore?
Expand Down
10 changes: 5 additions & 5 deletions ush/hpssarch_gen.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,8 +168,8 @@ if [[ ${type} = "gfs" ]]; then
fhr=$(printf %03i "${fh}")
if [[ ${ARCH_GAUSSIAN} = "YES" ]]; then
{
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxgrbf${fhr}.grib2"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxgrbf${fhr}.grib2.idx"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2.idx"

@aerorahul aerorahul Apr 9, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2.idx"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grib2"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grib2.idx"

} >> "${DATA}/gfs_flux.txt"

{
Expand Down Expand Up @@ -395,8 +395,8 @@ if [[ ${type} == "gdas" ]]; then
fh=0
while [[ ${fh} -le 9 ]]; do
fhr=$(printf %03i "${fh}")
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxgrbf${fhr}.grib2"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxgrbf${fhr}.grib2.idx"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2.idx"

@aerorahul aerorahul Apr 9, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2.idx"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grib2"
echo "${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grib2.idx"

echo "${COM_ATMOS_GRIB_0p25/${ROTDIR}\//}/${head}pgrb2.0p25.f${fhr}"
echo "${COM_ATMOS_GRIB_0p25/${ROTDIR}\//}/${head}pgrb2.0p25.f${fhr}.idx"
echo "${COM_ATMOS_GRIB_1p00/${ROTDIR}\//}/${head}pgrb2.1p00.f${fhr}"
Expand All @@ -408,7 +408,7 @@ if [[ ${type} == "gdas" ]]; then
done
flist="001 002 004 005 007 008"
for fhr in ${flist}; do
file="${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxgrbf${fhr}.grib2"
file="${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2"

@aerorahul aerorahul Apr 9, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
file="${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grb2"
file="${COM_ATMOS_MASTER/${ROTDIR}\//}/${head}sfluxf${fhr}.grib2"

@WenMeng-NOAA WenMeng-NOAA Apr 9, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul Updated.

if [[ -s "${file}" ]]; then
echo "${file}"
echo "${file}.idx"
Expand Down