How to correctly retrieve IMERG products

[edsml-ds423]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Hi,

First of all, I want to just say great job on the gpm-api. I was previously getting IMERG V07B data via FTPS in Python but moving forward, I might migrate and use your API.

Before doing so I wanted to just sense check the downloaded data. I could be missing something but I did a test on the timeslice 27th September 20:00-20:30. To do so, I downoloaded the file via your api and I downloaded the file directly from https://arthurhouhttps.pps.eosdis.nasa.gov/gpmdata/2022/09/27/imerg/. I then checked the data.

I found a mismatch between the values and if I plot a cropped region containing Hurricane Ian (peak intensity at the chosen timeslice), you can visually see the difference.

The two files that I am comparing are:

• gpm-api: 3B-HHR-E.MS.MRG.3IMERG.20220927-S200000-E202959.1200.V07B

• direct download: 3B-HHR.MS.MRG.3IMERG.20220927-S200000-E202959.1200.V07B.HDF5

Can I ask what server are you getting the version 7 files from?

Thanks,
Dan

Expected Behavior

I was expecting the API and directly downloaded data to match exactly.

Steps To Reproduce

1. Download data using the GPM-API tutorial notebook using the following dates:

start_time = datetime.datetime.strptime("2022-09-27 00:00:00", "%Y-%m-%d %H:%M:%S")
end_time = datetime.datetime.strptime("2022-09-28 00:00:00", "%Y-%m-%d %H:%M:%S")

2. Download the data directly from https://arthurhouhttps.pps.eosdis.nasa.gov/gpmdata/2022/09/27/imerg/
3. Compare the results either using np.allclose() or plot.

Environment

- OS:
- python:

Anything else?

No response

[ghiggi]

Hi @edsml-ds42. Sorry for the slight delay in answering, I just got back from holidays.

I think you are looking at different products:

• 3B-HHR.MS.MRG.3IMERG.20220927-S200000-E202959.1200.V07B.HDF5 correspond to the IMERG Final Run (IMERG-FR)
• 3B-HHR-E.MS.MRG.3IMERG.20220927-S200000-E202959.1200.V07B correspond to the IMERG Early Run (IMERG-ER)

To get the file you directly downloaded, you need to specify product="IMERG-FR", product_type = "RS".
To download IMERG Early Run, you need to specify product="IMERG-ER", product_type = "NRT".
Note that you can download data from either storage="PPS" or storage="GES_DISC".

Currently, I can already tell you that if you try downloading IMERG from GES_DISC you will likely download duplicates files (one for version V07A and V07B). This might cause problems in gpm.open_dataset. This problem occurs because they are updating IMERG products to version V07B these days .... then I guess they will delete version V07A and the problem will solve.

[edsml-ds423]

Hi @ghiggi,

No worries at all!

Thanks for pointing that out, all makes sense. I want the Early Run and I am currently using:
product="IMERG-ER", product_type = "NRT", storage="PPS" so should be good.

[ghiggi]

Glad I could help @edsml-ds423 ;) Have a nice day !