Provide bug fixes and add ALTKOB json file: marine monthly in situ BUFR

[ShastriPaturi]

This PR does two things:

1. Fixes bugs in the marine monthly in situ BUFR python converters
2. Ads json file for ALTKOB surface obs

fixes #877

[ShastriPaturi]

@emilyhcliu, the 6-hrly dumps for 20210801 are on orion at:
/work2/noaa/da/spaturi/forRuss/gdas.20210801

Let me know, if you can access it. It does have altkob bufr data as well.

[emilyhcliu]

For booking purpose, here is the list of marine-related data types added to bufr2ioda processing:

• altkob_surface
• bathythermal_profiles
• subpfl_argo_profiles
• subpfl_glider_profiles
• tesac_mamals
• tesac_profiles
• tackob_surface
• xbtctd_profiles

[emilyhcliu]

Marin data processing were tested with the latest feature branch ( bugfix/monthly_marine_bufr2ioda) for prepatmiodaobs step

The ROTDIR for the test on Orion: /work2/noaa/da/eliu/gdas-validation/comroot/gdas_eval_sevcsr_JEDI/gdas.20210801/00/obs

Summary of test results:
1. The following two types ran bufr2ioda successfully; IODA files created successfully

• bathythermal_profiles
• tesac_profiles

2. Trackob_surface failed in python (see below)

• tackob_surface

Traceback (most recent call last):
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_trackob_surface.py", line 277, in <module>
    bufr_to_ioda(config, logger)
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_trackob_surface.py", line 106, in bufr_to_ioda
    dateTime = r.get_datetime('year', 'month', 'day', 'hour', 'minute', group_by='depth')
IndexError: vector::_M_range_check: __n (which is 15) >= this->size() (which is 15)

3. The following three types received the following error message:

• tesac_mamals
• subpfl_argo_profiles
• subpfl_glider_profiles
• altkob_surface
• xbtctd_profiles

forrtl: severe (9): permission to access file denied, unit 12, file /work/noaa/rstprod/dump/gdas.20210801/00/atmos/gdas.t00z.subpfl.tm00.bufr_d
forrtl: severe (9): permission to access file denied, unit 12, file /work/noaa/rstprod/dump/gdas.20210801/00/atmos/gdas.t00z.altkob.tm00.bufr_d

I updated the obs directory in ROTDIR with the bufr_d files from /work2/noaa/da/spaturi/forRuss/gdas.20210801/00/atmos, where all required marine data (total 8) exist. However, the bufr2ioda processing still looking for the global dump (/work/noaa/rstprod/dump/gdas.20210801/00) for input.

The obs ROTDIR for the gdasprepatmiodaob is here:
/work2/noaa/da/eliu/gdas-validation/comroot/gdas_eval_sevcsr_JEDI/gdas.20210801/00/obs

Investigating....
OK. I thnk I know what's going one.
The json files need to be modified. I am testing now. Will report here shortly.

[ShastriPaturi]

@emilyhcliu, the bufr2ioda_trackob_surface.py was missing the depth variable.
It should be fine now.

[emilyhcliu]

To do the test correctly, I have created a temporary dump for 2021080100
The temporary dump is located at /work/noaa/stmp/eliu/dump
This dump contains 2021080100 data from the official dump and Shastri's marine data dump

The DMPDIR in the config.base is assigned to the temporary dump: /work/noaa/stmp/eliu/dump

There are total 8 bufr2ioda processings for marine data:
altkob_surface
bathythermal_profiles
subpfl_argo_profiles
subpfl_glider_profiles
tesac_mamals
tesac_profiles
tackob_surface
xbtctd_profiles

6 Out of the 8 marine data processings, 6 of them completed successfully:

-rw-r----- 1 eliu da    872371 Jan 26 16:03 gdas.t00z.argo_profiles.subpfl.nc
-rw-r----- 1 eliu da     98862 Jan 26 16:03 gdas.t00z.bathy_profiles.bathy.nc
-rw-r----- 1 eliu da    459087 Jan 26 16:03 gdas.t00z.glider_profiles.subpfl.nc
-rw-r----- 1 eliu da    460361 Jan 26 16:03 gdas.t00z.tesac_profiles.tesac.nc
-rw-r----- 1 eliu da     20127 Jan 26 16:03 gdas.t00z.trackob_profiles.trkob.nc
-rw-r----- 1 eliu da     24060 Jan 26 16:03 gdas.t00z.xbtctd_profiles.xbtctd.nc

The following 2 data types failed:

• altkobs_surface.
• tesac_mammals_profiles

Here are the error messages related to the failures:
altkobs_surface

Traceback (most recent call last):
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_altkob_surface.py", line 275, in <module>
    bufr_to_ioda(config, logger)

  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_altkob_surface.py", line 106, in bufr_to_ioda
    dateTime = r.get_datetime('year', 'month', 'day', 'hour', 'minute', group_by='depth')
IndexError: vector::_M_range_check: __n (which is 14) >= this->size() (which is 14)

tesac_mammals_profiles

Traceback (most recent call last):
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_tesac_mammals_profiles.py", line 315, in <module>
    bufr_to_ioda(config, logger)
  File "/work2/noaa/da/eliu/gdas-validation/global-workflow/sorc/gdas.cd/ush/ioda/bufr2ioda/bufr2ioda_tesac_mammals_profiles.py", line 172, in bufr_to_ioda
    logger.debug(f" temp          min, max, length, dtype = {temp.min()}, {temp.max()}, {len(temp)}, {temp.dtype}")
  File "/work2/noaa/da/python/opt/core/miniconda3/4.6.14/envs/gdasapp/lib/python3.7/site-packages/numpy/ma/core.py", line 5701, in min
    axis=axis, out=out, **kwargs).view(type(self))
  File "/work2/noaa/da/python/opt/core/miniconda3/4.6.14/envs/gdasapp/lib/python3.7/site-packages/numpy/core/_methods.py", line 44, in _amin
    return umr_minimum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation minimum which has no identity

[ShastriPaturi]

@emilyhcliu, I removed the depth variable with was the cause of the problem in [bufr2ioda_altkob_surface.py].
However, I was able to successfully generate the ioda output with bufr2ioda_tesac_mammals_profiles.py for 2021080100 with no changes to the bufr2ioda_tesac_mammals_profiles.py and the json file generated by run_bufr2ioda.py.

If it still is a problem, may I suggest you comment out or not to test bufr2ioda_tesac_mammals_profiles.py?
Also, thanks a lot for identifying bugs in the python converters.

[RussTreadon-NOAA]

@ShastriPaturi : Will this PR, #879, or #914 fix issue #907. It would be good to turn on JEDI DA ATM CI testing in g-w (issue #2294). Doing so requires that marine bufr2ioda converters to not fail when expected input bufr files are not present.

[ShastriPaturi]

@ShastriPaturi : Will this PR, #879, or #914 fix issue #907. It would be good to turn on JEDI DA ATM CI testing in g-w (issue #2294). Doing so requires that marine bufr2ioda converters to not fail when expected input bufr files are not present.

@RussTreadon-NOAA yes.

[RussTreadon-NOAA]

Great! This is good news @ShastriPaturi. When do you expect #879 and #914 to be merged into GDASApp develop? The timeline for these PRs impacts the timeline for g-w issue #2294.

[ShastriPaturi]

Great! This is good news @ShastriPaturi. When do you expect #879 and #914 to be merged into GDASApp develop? The timeline for these PRs impacts the timeline for g-w issue #2294.

@RussTreadon-NOAA I expect this to be done this week

[RussTreadon-NOAA]

@ShastriPaturi , are you OK with merging PR #937 into develop before this PR?

[guillaumevernieres]

@ShastriPaturi , are you OK with merging PR #937 into develop before this PR?

@RussTreadon-NOAA , I'll speak for @ShastriPaturi and say yes, merge #937 before this PR.

[ShastriPaturi]

@ShastriPaturi , are you OK with merging PR #937 into develop before this PR?

@RussTreadon-NOAA , I'll speak for @ShastriPaturi and say yes, merge #937 before this PR.

Yes @RussTreadon-NOAA