-
Notifications
You must be signed in to change notification settings - Fork 5
Changes for starting to work on WCOSS #65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 12 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
3eaf393
start to make things work on wcoss2
4cda1d9
j02 not n20
7fa2a79
push changes
7f59207
reduce modules
f6ec8ab
commit a real-time xml template
695c507
Try to speed up on WCOSS
CoryMartin-NOAA 7efdfd3
See if copilot can code without guardrails
CoryMartin-NOAA 3f3cbc7
Merge branch 'main' into feature/aod-wcoss
02800ed
More copilot changes
CoryMartin-NOAA f8da91a
change config
CoryMartin-NOAA 221ab8f
Fix coding norms
CoryMartin-NOAA ca68430
try to fix rads
CoryMartin-NOAA 060933b
address iliana's comments
CoryMartin-NOAA File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| obsforge: | ||
| PSLOT: obsforge | ||
| HOMEobsforge: /lfs/h2/emc/da/noscrub/cory.r.martin/obsforge/obsforge/ | ||
| SDATE: 202504250000 | ||
| EDATE: 202506250000 | ||
| COMROOT: /lfs/h2/emc/ptmp/cory.r.martin/obsforge | ||
| DCOMROOT: /lfs/h1/ops/prod/dcom/ | ||
| DATAROOT: /lfs/h2/emc/stmp/cory.r.martin/RUNDIRS | ||
| SCHEDULER: pbspro | ||
| ACCOUNT: GFS-DEV | ||
| QUEUE: dev | ||
| PARTITION: dev | ||
| KEEPDATA: NO | ||
|
CoryMartin-NOAA marked this conversation as resolved.
Outdated
|
||
| assim_freq: 6 | ||
|
|
||
| aoddump: | ||
| provider: VIIRSAOD | ||
| platforms: ['npp', 'j01', 'n21'] | ||
| thinning_threshold: 0 | ||
| channel: 4 | ||
| preqc: 2 | ||
| WALLTIME_AOD_DUMP: '00:10:00' | ||
| TASK_GEOM_AOD_DUMP: '1:ppn=1:tpp=1' | ||
| MEMORY_AOD_DUMP: 96GB | ||
|
|
||
| marinedump: | ||
| providers: | ||
| ghrsst: | ||
| list: | ||
| - sst_viirs_n21_l3u | ||
| - sst_viirs_n20_l3u | ||
| - sst_viirs_npp_l3u | ||
| - sst_avhrrf_ma_l3u | ||
| - sst_avhrrf_mb_l3u | ||
| - sst_avhrrf_mc_l3u | ||
| - sst_ahi_h08_l3c | ||
| - sst_abi_g17_l3c | ||
| - sst_abi_g16_l3c | ||
| qc config: | ||
| min: -2.0 | ||
| max: 45.0 | ||
| stride: 15 | ||
| min number of obs: 10 | ||
| rads: | ||
| list: | ||
| - rads_adt_3a | ||
| - rads_adt_3b | ||
| - rads_adt_6a | ||
| - rads_adt_c2 | ||
| - rads_adt_j2 | ||
| - rads_adt_j3 | ||
| - rads_adt_sa | ||
| - rads_adt_sw | ||
| qc config: | ||
| min: -2.0 | ||
| max: 3.0 | ||
| error ratio: 1.0 | ||
| nesdis_amsr2: | ||
| list: | ||
| - icec_amsr2_north | ||
| - icec_amsr2_south | ||
| qc config: | ||
| min: 0.0 | ||
| max: 1.0 | ||
| nesdis_mirs: | ||
| list: | ||
| - icec_amsu_ma1_l2 | ||
| - icec_atms_n20_l2 | ||
| - icec_atms_n21_l2 | ||
| - icec_atms_npp_l2 | ||
| # - icec_gmi_gpm_l2 | ||
| qc config: | ||
| min: 0.0 | ||
| max: 1.0 | ||
| smap: | ||
| list: | ||
| - sss_smap_l2 | ||
| qc config: | ||
| min: 0.1 | ||
| max: 40.0 | ||
| smos: | ||
| list: | ||
| - sss_smos_l2 | ||
| qc config: | ||
| min: 0.1 | ||
| max: 40.0 | ||
|
|
||
| WALLTIME_MARINE_DUMP: '00:10:00' | ||
| TASK_GEOM_MARINE_DUMP: '1:ppn=20:tpp=2' | ||
| MEMORY_MARINE_DUMP: 32GB | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| config.hercules.yaml | ||
| config.wcoss2.yaml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| <?xml version="1.0"?> | ||
| <!DOCTYPE workflow | ||
| [ | ||
| <!-- | ||
| PROGRAM | ||
| Main workflow manager for ObsForge | ||
|
|
||
| --> | ||
| <!ENTITY PSLOT "{{ PSLOT }}"> | ||
| <!ENTITY ROTDIR "{{ COMROOT }}/{{ PSLOT }}"> | ||
| <!ENTITY JOBS_DIR "{{ HOMEobsforge }}/jobs/rocoto"> | ||
| <!ENTITY MAXTRIES "2"> | ||
| ]> | ||
|
|
||
| <workflow realtime="F" scheduler="{{ SCHEDULER }}" cyclethrottle="1" taskthrottle="25"> | ||
|
|
||
| <log verbosity="10"><cyclestr>{{ COMROOT }}/{{ PSLOT }}/logs/@Y@m@d@H.log</cyclestr></log> | ||
|
|
||
| <!-- Define the cycles --> | ||
| <cycledef group="gdas">{{ SDATE }} {{ EDATE }} 06:00:00</cycledef> | ||
| <cycledef group="gfs">{{ SDATE }} {{ EDATE }} 06:00:00</cycledef> | ||
|
|
||
| <!-- | ||
| ============================================ | ||
| Task: gfs_aod_dump | ||
| ============================================ | ||
| --> | ||
| <task name="gfs_aod_dump" cycledefs="gfs" maxtries="&MAXTRIES;"> | ||
|
|
||
| <command>{{ HOMEobsforge }}/jobs/rocoto/aoddump.sh</command> | ||
|
|
||
| <jobname><cyclestr>obsforge_gfs_aod_dump_@H</cyclestr></jobname> | ||
| <account>{{ ACCOUNT }}</account> | ||
| <queue>{{ QUEUE }}</queue> | ||
| <walltime>{{ WALLTIME_AOD_DUMP }}</walltime> | ||
| <nodes>{{ TASK_GEOM_AOD_DUMP }}</nodes> | ||
| <memory>{{ MEMORY_AOD_DUMP }}</memory> | ||
| <native>-l place=vscatter:shared</native> | ||
|
|
||
| <join><cyclestr>{{ COMROOT }}/{{ PSLOT }}/logs/@Y@m@d@H/gfs_aod_dump_prep.log</cyclestr></join> | ||
|
|
||
| <envar><name>RUN_ENVIR</name><value>emc</value></envar> | ||
| <envar><name>HOMEobsforge</name><value>{{ HOMEobsforge }}</value></envar> | ||
| <envar><name>NET</name><value>gfs</value></envar> | ||
| <envar><name>RUN</name><value>gfs</value></envar> | ||
| <envar><name>CDATE</name><value><cyclestr>@Y@m@d@H</cyclestr></value></envar> | ||
| <envar><name>PDY</name><value><cyclestr>@Y@m@d</cyclestr></value></envar> | ||
| <envar><name>cyc</name><value><cyclestr>@H</cyclestr></value></envar> | ||
| <envar><name>KEEPDATA</name><value>{{ KEEPDATA }}</value></envar> | ||
| <envar><name>COMROOT</name><value>{{ COMROOT }}</value></envar> | ||
| <envar><name>DCOMROOT</name><value>{{ DCOMROOT }}</value></envar> | ||
| <envar><name>DATAROOT</name><value>{{ DATAROOT }}/{{ PSLOT }}/gfs.<cyclestr>@Y@m@d@H</cyclestr></value></envar> | ||
|
|
||
| <!--<dependency> | ||
| <datadep><cyclestr>/scratch2/NCEPDEV/stmp1/Cory.R.Martin/obsforge/gfs.@Y@m@d/@H/atmos/gfs.t@Hz.updated.status.tm00.bufr_d</cyclestr></datadep> | ||
| </dependency> | ||
| --> | ||
|
|
||
| </task> | ||
|
|
||
| <!-- | ||
| ============================================ | ||
| Task: gdas_aod_dump | ||
| ============================================ | ||
| --> | ||
| <task name="gdas_aod_dump" cycledefs="gdas" maxtries="&MAXTRIES;"> | ||
|
|
||
| <command>{{ HOMEobsforge }}/jobs/rocoto/aoddump.sh</command> | ||
|
|
||
| <jobname><cyclestr>obsforge_gdas_aod_dump_@H</cyclestr></jobname> | ||
| <account>{{ ACCOUNT }}</account> | ||
| <queue>{{ QUEUE }}</queue> | ||
| <walltime>{{ WALLTIME_AOD_DUMP }}</walltime> | ||
| <nodes>{{ TASK_GEOM_AOD_DUMP }}</nodes> | ||
| <memory>{{ MEMORY_AOD_DUMP }}</memory> | ||
| <native>-l place=vscatter:shared</native> | ||
|
|
||
| <join><cyclestr>{{ COMROOT }}/{{ PSLOT }}/logs/@Y@m@d@H/gdas_aod_dump_prep.log</cyclestr></join> | ||
|
|
||
| <envar><name>RUN_ENVIR</name><value>emc</value></envar> | ||
| <envar><name>HOMEobsforge</name><value>{{ HOMEobsforge }}</value></envar> | ||
| <envar><name>NET</name><value>gfs</value></envar> | ||
| <envar><name>RUN</name><value>gdas</value></envar> | ||
| <envar><name>CDATE</name><value><cyclestr>@Y@m@d@H</cyclestr></value></envar> | ||
| <envar><name>PDY</name><value><cyclestr>@Y@m@d</cyclestr></value></envar> | ||
| <envar><name>cyc</name><value><cyclestr>@H</cyclestr></value></envar> | ||
| <envar><name>KEEPDATA</name><value>{{ KEEPDATA }}</value></envar> | ||
| <envar><name>COMROOT</name><value>{{ COMROOT }}</value></envar> | ||
| <envar><name>DCOMROOT</name><value>{{ DCOMROOT }}</value></envar> | ||
| <envar><name>DATAROOT</name><value>{{ DATAROOT }}/{{ PSLOT }}/gdas.<cyclestr>@Y@m@d@H</cyclestr></value></envar> | ||
|
|
||
| <!--<dependency> | ||
| <datadep><cyclestr>/scratch2/NCEPDEV/stmp1/Cory.R.Martin/obsforge/gdas.@Y@m@d/@H/atmos/gdas.t@Hz.updated.status.tm00.bufr_d</cyclestr></datadep> | ||
| </dependency> | ||
| --> | ||
|
|
||
| </task> | ||
|
|
||
| <!-- | ||
| ============================================ | ||
| Task: gfs_marine_dump | ||
| ============================================ | ||
| --> | ||
| <task name="gfs_marine_dump" cycledefs="gfs" maxtries="&MAXTRIES;"> | ||
|
|
||
| <command>{{ HOMEobsforge }}/jobs/rocoto/marinedump.sh</command> | ||
|
|
||
| <jobname><cyclestr>obsforge_gfs_marine_dump_@H</cyclestr></jobname> | ||
| <account>{{ ACCOUNT }}</account> | ||
| <queue>{{ QUEUE }}</queue> | ||
| <walltime>{{ WALLTIME_MARINE_DUMP }}</walltime> | ||
| <nodes>{{ TASK_GEOM_MARINE_DUMP }}</nodes> | ||
| <memory>{{ MEMORY_MARINE_DUMP }}</memory> | ||
| <native>-l place=vscatter:shared</native> | ||
|
|
||
| <join><cyclestr>{{ COMROOT }}/{{ PSLOT }}/logs/@Y@m@d@H/gfs_marine_dump_prep.log</cyclestr></join> | ||
|
|
||
| <envar><name>RUN_ENVIR</name><value>emc</value></envar> | ||
| <envar><name>HOMEobsforge</name><value>{{ HOMEobsforge }}</value></envar> | ||
| <envar><name>NET</name><value>gfs</value></envar> | ||
| <envar><name>RUN</name><value>gfs</value></envar> | ||
| <envar><name>CDATE</name><value><cyclestr>@Y@m@d@H</cyclestr></value></envar> | ||
| <envar><name>PDY</name><value><cyclestr>@Y@m@d</cyclestr></value></envar> | ||
| <envar><name>cyc</name><value><cyclestr>@H</cyclestr></value></envar> | ||
| <envar><name>KEEPDATA</name><value>{{ KEEPDATA }}</value></envar> | ||
| <envar><name>COMROOT</name><value>{{ COMROOT }}</value></envar> | ||
| <envar><name>DCOMROOT</name><value>{{ DCOMROOT }}</value></envar> | ||
| <envar><name>DATAROOT</name><value>{{ DATAROOT }}/{{ PSLOT }}/gfs.<cyclestr>@Y@m@d@H</cyclestr></value></envar> | ||
|
|
||
| <!--<dependency> | ||
| <datadep><cyclestr>/scratch2/NCEPDEV/stmp1/Cory.R.Martin/obsforge/gfs.@Y@m@d/@H/atmos/gfs.t@Hz.updated.status.tm00.bufr_d</cyclestr></datadep> | ||
| </dependency> | ||
| --> | ||
|
|
||
| </task> | ||
|
|
||
| </workflow> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.