Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new LIS-based Western US Datasets #66

Open
4 of 10 tasks
TimLahmers opened this issue Nov 14, 2022 · 1 comment
Open
4 of 10 tasks

Add new LIS-based Western US Datasets #66

TimLahmers opened this issue Nov 14, 2022 · 1 comment

Comments

@TimLahmers
Copy link

TimLahmers commented Nov 14, 2022

  • Identify the point of contact and ensure someone is providing them updates: @TimLahmers

  • Data provider has read guidelines on data preparation

  • Identify data location:
    NASA-LIS Variables:
    s3://veda-data-store-staging/EIS/COG/LIS_WEST_US_DA/PRE GPP_ANOM/
    s3://veda-data-store-staging/EIS/COG/LIS_WEST_US_DA/PRE GPP_PCTL/
    s3://veda-data-store-staging/EIS/COG/LIS_WEST_US_DA/PRE PRECIP/
    s3://veda-data-store-staging/EIS/COG/LIS_WEST_US_DA/PRE RZSM/
    LIS-Derived FWI Quantities:
    s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE BUI/
    s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE DC/
    s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE DMC/
    s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE FFMC/
    s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE FWI/
    s3://veda-data-store-staging/EIS/COG/LIS_FORC_FWI_WEST_US/PRE ISI/

  • Number of items: 10

  • Verify that files are valid COGs (e.g. with rio cogeo validate)

  • Gather STAC collection metadata

    • id:
    • title: LIS-Derived Fire Variables
    • description: NASA-LIS (with multivariate snow, soil moisture, TWS, and LAI assimilation) Surface Anomalies, including Monthly Precipitation Anomalies, Daily Root Zone Soil Moisture Percentiles, Daily GPP Percentiles, and Daily GPP Anomalies and components of Fire Weather Index (FWI) from LIS forcing variables. FWI quantities include Fine Fuel Moisture Code (FFMC), Duff Moisture Code (DMC), Drought Code (DC), Buildup Index (BUI), Initial Spread Index (ISI), and Fire Weather Index (FWI). FWI and LIS surface variables are for the Western US with 0.01 degree (~1-km) resolution.
    • license:
    • temporal interval: 24-Hours
    • whether it is periodic on the dashboard (periodic = regular time series of layers without gaps): FWI 01-2002 - 10-2021; LIS 07-2002 - 10-2021
    • the dashboard time density:
  • Review and follow https://github.com/NASA-IMPACT/cloud-optimized-data-pipelines/blob/main/OPERATING.md

  • Open PR for publishing those datasets to the Staging API:

  • Notify QA / move ticket to QA state

  • Once approved, merge and close.

Resources on metadata

If not already familiar with these conventions for generating STAC collection and item metadata:
- Collections: NASA-IMPACT/veda-backend#29 and STAC version 1.0 specification for collections
- Items: NASA-IMPACT/veda-backend#28 and STAC version 1.0 specification for items
- NOTE: The delta-backend instructions are specific to datasets for the climate dashboard, however not all datasets are going to be a part of the visual layers for the dashboard so you can ignore the instructions that are specific to "dashboard" extension, "item_assets" in the collection and "cog_default" asset type in the item.

@connectwithprakash
Copy link

connectwithprakash commented Jan 19, 2023

The files under NASA-LIS Variables do not follow the naming convention for date.
Also, the directory name for all of the data locations do not contain PRE keyword.

@gadomski gadomski transferred this issue from NASA-IMPACT/veda-data-pipelines Sep 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants