refactor mediator history to permit new functionality

[mvertens]

Description of changes

refactor mediator history to permit new functionality

Specific notes

This PR is a large refactor of CMEPS history function capability. There are 3 categories where this capability is now extended or added: instantaneous, average and auxiliary output.

• Each component post phase (i.e. med_phases_post_xxx_mod.F90) now calls med_phases_history_write_comp and passes it its component id.

• In turn med_phases_write_comp calls instantaneous, average and auxiliary output for that component.

• med_phases_history_write_inst_comp

  • writes the import and export state for that component based on the new input attribute settings
    history_n_xxx_inst and history_option_xxx_inst - where xxx = [atm,ice,glc,lnd,ocn,rof,wav and med]
  • output files will have the form <casename>.cpl.hi.atm.yyyy-mm-dd-sssss.nc

• med_phases_history_write_avg_comp

  • writes the import and export state for that component based on the new input attribute settings
    history_n_xxx_avg and history_option_xxx_avg - where xxx = [atm,ice,glc,lnd,ocn,rof,wav]
  • output files will have the form <casename>.cpl.ha.atm.yyyy-mm-dd-sssss.nc

• med_phases_history_write_aux_comp

  • writes auxiliary history files that are normally high frequency mediator output files that contain more than one time sample on a file and that are used to spin up a prognostic component (e.g. the ocn or lnd carbon cycle)
  • the following auxiliary files are now supported and to enable them simply set the histaux_*_enabled flage to .true.

  histaux_atm2med_file1_auxname = atm.1h.inst
  histaux_atm2med_file1_doavg = .false.
  histaux_atm2med_file1_flds = Faxa_swndr:Faxa_swvdr:Faxa_swndf:Faxa_swvdf
  histaux_atm2med_file1_history_n = 1
  histaux_atm2med_file1_history_option = nhours
  
  histaux_atm2med_file2_enabled = .false.
  histaux_atm2med_file1_ntperfile = 24
  histaux_atm2med_file2_auxname = atm.1h.avrg
  histaux_atm2med_file2_doavg = .true.
  histaux_atm2med_file2_flds = Sa_u:Sa_v
  histaux_atm2med_file2_history_n = 1
  histaux_atm2med_file2_history_option = nhours
  histaux_atm2med_file2_ntperfile = 24
  
  histaux_atm2med_file3_enabled = .false.
  histaux_atm2med_file3_auxname = atm.3hprec.avrg
  histaux_atm2med_file3_doavg = .true.
  histaux_atm2med_file3_flds = Faxa_rainc:Faxa_rainl:Faxa_snowc:Faxa_snowl
  histaux_atm2med_file3_history_n = 3
  histaux_atm2med_file3_history_option = nhours
  histaux_atm2med_file3_ntperfile = 8
  
  histaux_atm2med_file4_enabled = .false.
  histaux_atm2med_file4_auxname = atm.3h.avrg
  histaux_atm2med_file4_doavg = .true.
  histaux_atm2med_file4_flds = Sa_z:Sa_topo:Sa_u:Sa_v:Sa_tbot:Sa_ptem:Sa_shum:Sa_dens:Sa_pbot:Sa_pslv:Faxa_lwdn:Faxa_rainc:Faxa_rainl:Faxa_snowc:Faxa_snowl:Faxa_swndr:Faxa_swvdr:Faxa_swndf:Faxa_swvdf:Sa_co2diag:Sa_co2prog
  histaux_atm2med_file4_history_n = 3
  histaux_atm2med_file4_history_option = nhours
  histaux_atm2med_file4_ntperfile = 8
  
  histaux_atm2med_file5_auxname = atm.24h.avrg
  histaux_atm2med_file5_doavg = .true.
  histaux_atm2med_file5_enabled = .false.
  histaux_atm2med_file5_flds = Faxa_bcph:Faxa_ocph:Faxa_dstwet:Faxa_dstdry:Sa_co2prog:Sa_co2diag
  histaux_atm2med_file5_history_n = 1
  histaux_atm2med_file5_history_option = ndays
  histaux_atm2med_file5_ntperfile = 1
  
  histaux_lnd2med_file1_auxname = lnd.ncpl.inst
  histaux_lnd2med_file1_doavg = .false.
  histaux_lnd2med_file1_enabled = .false.
  histaux_lnd2med_file1_flds = all
  histaux_lnd2med_file1_history_n = 1
  histaux_lnd2med_file1_history_option = nsteps
  histaux_lnd2med_file1_ntperfile = 1
  

  • for file 1 history output is of the form .cpl.hx.atm.1h.inst.yyyy-mm-dd-sssss.nc - and the <atm.1h.inst>
    is specified by <histaux_atm2med_file1_auxname>

Contributors other than yourself, if any: None

CMEPS Issues Fixed: None

Are changes expected to change answers?

• bit for bit
• different at roundoff level
• more substantial

Any User Interface Changes (namelist or namelist defaults changes)? (see above summary)

• Yes
• No

Testing performed if application target is CESM:(either UFS-S2S or CESM testing is required):

• CESM testlist_drv.xml
  • machines and compilers: cheyenne/intel compared to testlist_drv.xml baselines generated with cesm2_3_beta05
  • details (e.g. failed tests):
• (other) please described in detail
  • machines and compilers
  • details (e.g. failed tests):

Testing performed if application target is UFS-coupled:

• (recommended) UFS-coupled testing
  • description:
  • details (e.g. failed tests):

Testing performed if application target is UFS-HAFS:

• (recommended) UFS-HAFS testing
  • description:
  • details (e.g. failed tests):

Hashes used for testing:

• CESM:
  • repository to check out: https://github.com/ESCOMP/CESM.git
  • branch:
  • hash:
• UFS-coupled, then umbrella repostiory to check out and associated hash:
  • repository to check out:
  • branch:
  • hash:
• UFS-HAFS, then umbrella repostiory to check out and associated hash:
  • repository to check out:
  • branch:
  • hash:

[uturuncoglu]

@mvertens it would be also nice to have short description about the PR and list of new features that will be bringing. I think also there are some changes in ESMF attributes (nuopc.runconfig or nems.configure) and it would be nice to have them in the description too.

[rsdunlapiv]

@uturuncoglu - @mvertens asked that you please test this PR in HAFS. You only need to test the instantaneous history options (history_n_xxx_inst and history_option_xxx_inst), not the average or aux options.

[uturuncoglu]

@rsdunlapiv I could not be sure it is ready or not since she was working on it. Anyway, I will test it and update you soon.

[uturuncoglu]

@rsdunlapiv @mvertens i tested it under HAFS with both instantaneous and average history output and all looks fine.

[uturuncoglu]

@DeniseWorthen JFYI, you don't need to add any additional phases into the run sequence. I add following to nems.configure for atmosphere and ocean under MED_attributes section.

  history_n_atm_inst = 6
  history_option_atm_inst = nhours
  history_ymd_atm_inst = -999

  history_n_ocn_inst = 6
  history_option_ocn_inst = nhours
  history_ymd_ocn_inst = -999

  history_n_atm_avg = 6
  history_option_atm_avg = nhours
  history_ymd_atm_avg = -999

  history_n_ocn_avg = 6
  history_option_ocn_avg = nhours
  history_ymd_ocn_avg = -999

[DeniseWorthen]

@uturuncoglu Can I ask how you tested the average function to determine it was working correctly? I tried the following:

1. create a 3 day run for cpld_control writing all history files with inst/nsteps.
2. create a 3 day run for cpld_control writing all history files with avg/ndays

I used nco to create a timeseries of the atmImp_Sa_tbot for case 1. Averaging the case 1 timeseries over the timesteps (ie, 96 values) corresponding to one of the days doesn't give the same value as case 2.

[uturuncoglu]

@DeniseWorthen I did not check the correctness. I just run it and be sure it is functional or not. I am not sure how daily averages are calculated under CMEPS. It would be nice to test to configure the average with nsteps = 1, this might give the exact results as instantaneous. Right? If there is something wrong with it that could show.

[DeniseWorthen]

I did do a test that confirmed that inst/day and inst/hour are consistent with inst/nsteps. I also tested that averaging 4 values from inst/nsteps is consistent with the avg/hour file. So I was surprised that averaging over an entire day did not work.

[uturuncoglu]

@DeniseWorthen Maybe it is just about handling 00Z.

[DeniseWorthen]

Something I noticed with the averaging function is that the filename refers to the end of the Advance. This makes perfect sense for the inst option, since the internal timestamp matches the filename. I wonder whether the way it is set now it is optimal for the avg option though.

When you use avg, the timestamp in the file appears as the mid-point of the averaging period, which is fine. But the file name uses the end of the averaging period. So, for example, with avg/days, the filename would be 2016-10-05-00000 but the internal timestamp would be 04-OCT-2016 12:00.

In any case, I've verified that the avg/day works as expected so I will approve the PR.

[mvertens]

@JEdwards - I think I have addressed your comments. Please let me know if there are still issues. Could we please go ahead and merge? I'm happy to do that if needed.