Add fv3-jedi lam_RRFS backgrounds

[gthompsnJCSDA]

Description

The files in this PR coincide with the addition of FV3-limited area model RRFS backgrounds that go together with the same feature branch being prepared for fv3-jedi. This should permit a sample 6-hour forecast background provided by Ming Hu (NOAA-GSL) from FV3 single cube face with 961x961 grid points at ~12km spacing.

Issue(s) addressed

A specific issue was not opened for this PR

Acceptance Criteria (Definition of Done)

A successful ctest

Dependencies

A corresponding PR is found in fv3-jedi repo (soon).

Impact

No impacts to other repos.

[danholdaway]

This PR increases the size of testinput_tier_1 from 800Mb to 6Gb. The regional cmaq files are 100Mb. Can we aim for something more along those lines?

[danholdaway]

Almost all the other tests run with C12 (800km resolution).

[gthompsnJCSDA]

This PR increases the size of testinput_tier_1 from 800Mb to 6Gb. The regional cmaq files are 100Mb. Can we aim for something more along those lines?

I really have no clue how to "reduce" resolution of FV3 output model data. Maybe this is a good reason to really do something with testing of tier_2 data. I'm just trying to make a space to hold the files needed for real-world RRFS so I can get GSL to do joint development work with me.

Personally I really see this as necessary for ease of use with our partners. It's certainly logical to set into tier2, not 1. This was also why I set number of PEs to 24 instead of 6 (per Fabio's comment).

[danholdaway]

Sharing data for experiment reasons is typically done through R2D2 not through fv3-jedi-data. If we were to add 12km files for GEOS, GFS, FV3-LAM, CMAQ etc this repo would be hundreds of gigabytes and everyone would have to clone it every time, which is obviously not what we want. Machines like Hera already take 30mins sometimes to clone all the test data. Here is an example of re-gridding the data using fv3-jedi https://github.com/JCSDA-internal/fv3-jedi/blob/develop/test/testinput/convertstate_gfs.yaml. You can use this to regrid everything as a one off to C12 and then add the low resolution data to this repo.

[danholdaway]

Let me know if you need more re an explainer on re-gridding the data.

[gthompsnJCSDA]

sticking a draft label on this one until I figure out how to decimate the data to be smaller file sizes.

[danholdaway]

Thanks @gthompsnJCSDA

[gthompsnJCSDA]

Let me know if you need more re an explainer on re-gridding the data.

Dan, can we keep the full resolution backgrounds as tier2 test and I make the low-res versions for tier1?

[gthompsnJCSDA]

@danholdaway Can you advise me further on convertstate application? The stdout of slurm job has the following prints before failure...

 For k_split (remapping)=           1
n_split is set to 01 for resolution-dt=0961x0961x6-  300.000
Using n_sponge : 001
Using non_ortho :       T
 Starting PEs :           24
 Starting Threads :            2

FATAL from PE     0: mpp_domains_define.inc: At least one pe in pelist is not used by any tile in the mosaic

And when I see that nsplit line with 961x961x6, it makes me think it is trying to run 6-faces of the cube rather than just one. Does the yaml for this have a regional=true setting like running hofx_nomodel?

[gthompsnJCSDA]

Disregard the prior comment. I saw a way to do the same namelist file as an input to the geometry section, so it appears that I may be moving forward again.

[gthompsnJCSDA]

@danholdaway I was able to generate reduced resolution C24 grids, but now to complete my experiments of hofx_nomodel, I need the "grid_spec.nc" file and its more important C24_grid.tileZ.haloY.nc file - which I have no idea how to create. Can you advise this last step please? I see there's a gfs C12 grid, but I don't see a specific named file for C24 resolution and it probably won't apply anyway because it's not a single face of the cube.

[danholdaway]

@gthompsnJCSDA I'm surprised that you were able to re-grid them without this file. The file in question would be read during the geometry creation phase, something required also in the re-gridding task. @CoryMartin-NOAA @TingLei-daprediction given your experience with the regional setup of fv3-jedi would you be able to assist or put Greg in touch with someone who could?

[CoryMartin-NOAA]

@gthompsnJCSDA I would suggest using the existing grid_spec.nc file, etc. that are available for the LAM-CMAQ tests:
https://github.com/JCSDA-internal/fv3-jedi-data/tree/develop/testinput_tier_1/inputs/lam_cmaq . It is a 45x24 L64 approximate CONUS domain.

[TingLei-daprediction]

Yes, I agree with @CoryMartin-NOAA <https://github.com/CoryMartin-NOAA>, the easiest way is to use the data in the existing fv3-lam tests . Another way is to start from the community weather app https://github.com/ufs-community/ufs-mrweather-app to set up the grid needed and create all those files following its' setup. ( that is what I did for my ,say, 13km grids). Ting

…

On Tue, Nov 2, 2021 at 9:39 AM Cory Martin ***@***.***> wrote: @gthompsnJCSDA <https://github.com/gthompsnJCSDA> I would suggest using the existing grid_spec.nc file, etc. that are available for the LAM-CMAQ tests: https://github.com/JCSDA-internal/fv3-jedi-data/tree/develop/testinput_tier_1/inputs/lam_cmaq . It is a 45x24 L64 approximate CONUS domain. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADSYWF4VPG77AARZ537E7CTUJ7ZYLANCNFSM5G724JLA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[gthompsnJCSDA]

Thanks @TingLei-daprediction and @CoryMartin-NOAA. I made a standard RRFS-CONUS-25km grid using UFS (short-range) and used the convertstate_gfs.x to reduce resolution and domain space and now I am waiting on the corresponding PR CI-testing to see if my modifications pass all the tests. @danholdaway I think this could be re-reviewed once those tests pass.

[danholdaway]

Thanks @gthompsnJCSDA. I'm happy with the size increase you're proposing.