Reintroduce paths

[amilan17]

Details

The problem is that OSCAR/Surface is still using the paths to organize the filtering of codes and in some cases they are unable to map the tags to the paths and cannot introduce the new codes to the application. I think we can address this by adding a column for tags specifically and continue encoding of paths in the CSV tables. This will allow us to continue developing the tag approach, but it also continues the paradigm of having the same code twice for different paths.

Requestor

Anna Milan (WMO), @amilan17
@fstuerzl @luciacap

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2023.04.06-TT-WIGOSMD notes:
(Joerg) the paths are a legacy approach and are implemented in OSCAR/Surface; Tags provide a way to multiple applications for users to filter in a more meaning way. Moving from one-to-one relationship to one-to-many; e.g.. /oceans//temperature /atmosphere//temperature. OSCAR NextGen will be deployed in 2025 (probably) and will be (hopefully) able to accommodate the tag filtering.

• Team agrees to modify the CSV tables to have three columns: paths, domains, tags. Noting that we may need to create duplicate codes for different domains.

[amilan17]

@fstuerzl @luciacap 

Does the path need to include the parameter name? For example, can it just be 

\Atmosphere\Temperature

instead of 

\Atmosphere\Temperature\Air temperature (at specified distance from reference surface)

[amilan17]

I added columns to 1-01* and 5-02* tables and rebased other active branches. Content still needs to be adjudicated.

[amilan17]

@fstuerzl Are the paths used in OSCAR for ObservingMethods?

[fstuerzl]

@amilan17 yes, the paths are also used for methods in the same way as for the variables (and programs). In the internal reference table for methods, all entries have a parent method, so that a hierarchical structure can be build up:

[amilan17]

questions/notes for discussion at meeting:
General

1. Does the path need to include the variable name?
2. Primary domain and subpath is used in path column, additional domains or other tags can be added to the other columns as relevant.
3. Should review consistency of all paths during before validation.

[amilan17]

1-01-03

• need to review, paths and tags have diverged

notation	2021-path	2023-tags	name
13	\Ocean\Topography\Bathymetry	topography	Bathymetry
18	\Ocean\Basic Physical Properties\Pressure	physical properties	Pressure
62	\Ocean\Currents\Ocean Current - Direction	currents	Ocean current direction
63	\Ocean\Currents\Ocean Current - Speed	currents	Ocean current speed
64	\Ocean\Topography\Ocean dynamic topography (ODT)	physical properties	Ocean dynamic topography (ODT)
65	\Ocean\Currents\Ocean surface currents (vector)	physical properties	Surface currents
67	\Ocean\Miscellaneous\Carbon species\Dissolved inorganic carbon (DIC)	biogeochemistry, inorganic carbon	Dissolved inorganic carbon (DIC)
68	\Ocean\Miscellaneous\Carbon species\Dissolved organic carbon	biogeochemistry, dissolved organic carbon	Dissolved organic carbon
69	\Ocean\Miscellaneous\Nitrogen species\Dissolved organic nitrogen	biogeochemistry, nitrogen species	Dissolved organic nitrogen
70	\Ocean\Miscellaneous\Diffuse attenuation coefficient (DAC), turbidity	physical properties	Diffuse attenuation coefficient (DAC), turbidity
71	\Ocean\Biology\Chlorophyll concentration	biogeochemistry, inorganic carbon	Chlorophyll a concentration
72	\Ocean\Miscellaneous\Ocean suspended sediments concentration	biogeochemistry, particulate matter	Ocean suspended sediments concentration
73	\Ocean\Miscellaneous\Oil spill cover	biogeochemistry	Oil spill cover
74	\Ocean\Biology\Primary production	biology	Primary production
75	\Ocean\Miscellaneous\Sea surface Mass Flux	physical properties	Sea surface mass flux
76	\Ocean\Miscellaneous\Sea surface heat flux	physical properties, ocean surface heat flux	Ocean Surface heat flux
77	\Ocean\Topography\Sea surface height anomaly (SSHA)	physical properties, sea surface height	Sea surface height anomaly (SSHA)
78	\Ocean\Miscellaneous\Total alkalinity (TA)	biogeochemistry, inorganic carbon	Total alkalinity (TA)
79	\Ocean\Miscellaneous\pH	biogeochemistry, inorganic carbon	pH
82	\Ocean\Biology\SiO3= (metasilicate)	biogeochemistry	SiO3= (metasilicate)
85	\Ocean\Other Gas\O2-Ar ratio	biogeochemistry	O2-Ar ratio
86	\Ocean\Other Gas\Ocean subsurface dissolved oxygen concentration	biogeochemistry, oxygen	Ocean subsurface dissolved oxygen concentration
87	\Ocean\Other Gas\Oxygen, dissolved	biogeochemistry, oxygen	Dissolved oxygen
89	\Ocean\Radiation\Water-leaving spectral radiance	physical properties, radiation	Water-leaving spectral radiance
90	\Ocean\Reactive Gas\C2H6S (dimethylsulfide, DMS)	biogeochemistry, volatile organic compound	C2H6S (dimethylsulfide, DMS)
91	\Ocean\Basic Physical Properties\Ocean salinity	physical properties	Subsurface salinity
92	\Ocean\Basic Physical Properties\Sea surface salinity	physical properties	Sea surface salinity
94	\Ocean\Basic Physical Properties\Ocean temperature	physical properties, subsurface temperature	Subsurface temperature
95	\Ocean\Basic Physical Properties\Sea surface temperature	physical properties	Sea surface temperature
96	\Ocean\Topography\Coastal sea level (tide)	physical properties, sea surface height	Coastal sea level (tide)
98	\Ocean\Waves\Dominant wave direction	physical properties, sea state	Dominant wave direction
99	\Ocean\Waves\Dominant wave period	physical properties, sea state	Dominant wave period
100	\Ocean\Waves\Significant wave height	physical properties, sea state	Significant wave height
101	\Ocean\Waves\Wave 1D energy frequency spectrum	physical properties, sea state	Wave 1D energy frequency spectrum
102	\Ocean\Waves\Wave directional energy frequency spectrum	physical properties, sea state	Wave directional energy frequency spectrum
103	\Ocean\Wind\Wind stress	physical properties, ocean surface stress	Wind stress
311	\Ocean\Biology\Nitrogen species\NO2- (nitrite)	biogeochemistry, nutrients	NO2- (nitrite)
312	\Ocean\Biology\Nitrogen species\NO3- (nitrate)	biogeochemistry, nutrients	NO3- (nitrate)
399	\Ocean\Biology\Phosporus species\PO4= (phosphate)	biogeochemistry, nutrients	PO4= (phosphate)
400	\Ocean\Biology\Phosporus species\Total organic phosphorus	biogeochemistry, phosporus species	Total organic phosphorus
401	\Ocean\Cryosphere\Sea ice\Sea-ice cover	cryosphere, sea ice	Sea-ice cover
402	\Ocean\Cryosphere\Sea ice\Sea-ice elevation	cryosphere, sea ice	Sea-ice elevation
403	\Ocean\Cryosphere\Sea ice\Sea-ice motion	cryosphere, sea ice	Sea-ice motion
404	\Ocean\Cryosphere\Sea ice\Sea-ice surface characteristics	cryosphere, sea ice	Sea-ice surface characteristics
405	\Ocean\Cryosphere\Sea ice\Sea-ice surface temperature	cryosphere, sea ice	Sea-ice surface temperature
406	\Ocean\Cryosphere\Sea ice\Sea-ice thickness	cryosphere, sea ice	Sea-ice thickness
407	\Ocean\Cryosphere\Sea ice\Sea-ice type	cryosphere, sea ice	Sea-ice type
408	\Ocean\Transmissivity\CTD\Transmissivity	physical properties, CTD	Transmissivity
549	\Ocean\Other Gas\CTD\pO2	biogeochemistry, CTD	pO2
731	\Ocean\Basic Physical Properties\Conductivity	physical properties, CTD	Conductivity
732	\Ocean\Basic Physical Properties\Density	physical properties, CTD	Density
733	\Ocean\Biology\Zooplankton (size and-or development stage)	biology	Zooplankton (size and/or development stage)
735	\Ocean\Greenhouse Gas\pCO2	biogeochemistry, inorganic carbon	pCO2
736	\Ocean\Greenhouse Gas\pCO2 air-sea flux	biogeochemistry, inorganic carbon	pCO2 air-sea flux
739	\Ocean\Miscellaneous\Fluorescence (DOM)	physical properties	Fluorescence (DOM)
740	\Ocean\Topography\Mean dynamic topography (MDT)	physical properties, sea surface height	Mean dynamic topography (MDT)
741	\Ocean\Topography\Mean sea surface (MSS)	physical properties, sea surface height	Mean sea surface (MSS)
742	\Ocean\Topography\Sea surface height (SSH, OST)	physical properties, sea surface height	Sea surface height (SSH, OST)
743	\Ocean\Waves\Wave height	physical properties, sea state	Wave height
744	\Ocean\Waves\Wave period	physical properties, sea state	Wave period
745	\Ocean\Waves\Directional waves	physical properties, sea state	Directional waves
10080	\Ocean\Tides	physical properties, sea surface height	Tides
12244		biogeochemistry, other gas	N2, Ar
12245		biology, plankton	Zooplankton biomass and diversity
12246		biogeochemistry	Ocean subsurface tracers

[amilan17]

1-01-05

Need to determine how to organize paths with new labels associated with hydrological variables.

Terrestrial
	Cryosphere
	Groundwater
	Lake
	Land surface
	River
	Soil
	Well
new	Hydrological
new	Aquifer
new	liquidPhase
new	surface water
new	watershed
new	reservoir
new	stream

The paths could be:

• Terrestrial
  • Cryosphere
  • Hydrological
    • Groundwater (does well go here?)
    • Surface water
      • River
      • Lake
  • Land surface
  • Soil (will need to determine the 6 terrestrial, 2 hydro variables)

and the following can remain as tags: watershed, reservoir, aquifer, stream, liquidPhase

[amilan17]

1-01-01

• "Gas" vs "Gas Phase"
• Need to determine paths for 12252-12262. Change "remote sensing" to something that is more independant of method?

[luciacap]

Hi Anna Apologies for the late reply. I checked also with Franziska and we both don’t see the need for the parameter name in the path. From: Anna Milan ***@***.***> Sent: Thursday, April 13, 2023 4:00 PM To: wmo-im/wmds ***@***.***> Cc: Cappelletti Lucia ***@***.***>; Mention ***@***.***> Subject: Re: [wmo-im/wmds] Reintroduce paths (Issue #472) @fstuerzl<https://github.com/fstuerzl> @luciacap<https://github.com/luciacap> Does the path need to include the parameter name? For example, can it just be \Atmosphere\Temperature instead of \Atmosphere\Temperature\Air temperature (at specified distance from reference surface) — Reply to this email directly, view it on GitHub<#472 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIPTXM225QGX2QTXYY75E6LXBABHPANCNFSM6AAAAAAWNB6ETQ>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[joergklausen]

+1 for listing everything except the variable name itself in the path. If handled in this way, we must assure that it is handled the same way throughout the repo. As paths are not included in the codes registry, I consider this a purely technical change that doesn't need to be listed in the FT amendment. Agreed?

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2023.04.21-TT-WIGOSMD note:
removal of the variable name from the path will not have an impact on the FT

[joergklausen]

1-01-01

• "Gas" vs "Gas Phase"
• Need to determine paths for 12252-12262. Change "remote sensing" to something that is more independant of method?

If I recall correctly, the previous ad-hoc team on ACV has concluded that 'aerosol' should be turned into 'particlce phase'; and to be consistent, 'gas' should turn into 'gas phase'.

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2023.05.04-TT-WIGOSMD notes:
@nuneslf recommends re-introducing paths without going to FT procedure.

[amilan17]

1. In table 1-01-01, is "N species" the same as "Nitrogen species"?

[joergklausen]

Yes, that is correct.

…

___________________ Dr. Jörg Klausen Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Operation Center 1 | P.O. Box 257 | CH-8058 Zurich-Airport Phone: +41 58 460 92 23 ***@***.******@***.***> | Mobile: +41 79 702 94 67 | https://orcid.org/0000-0003-2915-6028 All about weather and climate, please visit our website www.meteoswiss.ch<https://www.meteoswiss.admin.ch/> and MeteoSwiss App<http://www.meteoswiss.admin.ch/home/services-and-publications/beratung-und-service/meteoswiss-app.html> All about OSCAR/Surface at https://oscar.wmo.int/surface All about GAWSIS at https://gawsis.meteoswiss.ch<https://gawsis.meteoswiss.ch/> From: Anna Milan ***@***.***> Sent: Wednesday, 24 May, 2023 10:54 To: wmo-im/wmds ***@***.***> Cc: Klausen Jörg ***@***.***>; Comment ***@***.***> Subject: Re: [wmo-im/wmds] Reintroduce paths (Issue #472) 1. In table 1-01-01, is "N species" the same as "Nitrogen species"? — Reply to this email directly, view it on GitHub<#472 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIFWXY4BMMUN6NAU2CKEFYLXHXEEDANCNFSM6AAAAAAWNB6ETQ>. You are receiving this because you commented.Message ID: ***@***.***>

[amilan17]

@joergklausen @fstuerzl Can you please review the path changes in this diff?
master...8f72c05

Table 1-01-01 paths	Count of path
\Atmosphere	8
\Atmosphere\Bioaerosol\Fungi\Capnodiales	1
\Atmosphere\Bioaerosol\Fungi\Pleosporales	1
\Atmosphere\Bioaerosol\Pollen	1
\Atmosphere\Bioaerosol\Pollen\Adoxaceae	1
\Atmosphere\Bioaerosol\Pollen\Altingiaceae	1
\Atmosphere\Bioaerosol\Pollen\Amaranthaceae	2
\Atmosphere\Bioaerosol\Pollen\Aquifoliaceae	1
\Atmosphere\Bioaerosol\Pollen\Arecaceae	1
\Atmosphere\Bioaerosol\Pollen\Asteraceae	4
\Atmosphere\Bioaerosol\Pollen\Betulaceae	6
\Atmosphere\Bioaerosol\Pollen\Brassicaceae	2
\Atmosphere\Bioaerosol\Pollen\Buxaceae	1
\Atmosphere\Bioaerosol\Pollen\Cannabaceae	1
\Atmosphere\Bioaerosol\Pollen\Caryophyllaceae	1
\Atmosphere\Bioaerosol\Pollen\Cupressaceae	1
\Atmosphere\Bioaerosol\Pollen\Cyperaceae	1
\Atmosphere\Bioaerosol\Pollen\Elaeagnaceae	1
\Atmosphere\Bioaerosol\Pollen\Ericaceae	1
\Atmosphere\Bioaerosol\Pollen\Euphorbiaceae	1
\Atmosphere\Bioaerosol\Pollen\Fagaceae	3
\Atmosphere\Bioaerosol\Pollen\Ginkgoaceae	1
\Atmosphere\Bioaerosol\Pollen\Juglandaceae	1
\Atmosphere\Bioaerosol\Pollen\Juncaceae	1
\Atmosphere\Bioaerosol\Pollen\Malvaceae	1
\Atmosphere\Bioaerosol\Pollen\Moraceae	1
\Atmosphere\Bioaerosol\Pollen\Oleaceae	4
\Atmosphere\Bioaerosol\Pollen\Pinaceae	6
\Atmosphere\Bioaerosol\Pollen\Plantaginaceae	1
\Atmosphere\Bioaerosol\Pollen\Platanaceae	1
\Atmosphere\Bioaerosol\Pollen\Poaceae	3
\Atmosphere\Bioaerosol\Pollen\Polygonaceae	1
\Atmosphere\Bioaerosol\Pollen\Ranunculaceae	1
\Atmosphere\Bioaerosol\Pollen\Rosaceae	1
\Atmosphere\Bioaerosol\Pollen\Salicaceae	2
\Atmosphere\Bioaerosol\Pollen\Sapindaceae	2
\Atmosphere\Bioaerosol\Pollen\Taxaceae	1
\Atmosphere\Bioaerosol\Pollen\Ulmaceae	1
\Atmosphere\Bioaerosol\Pollen\Umbelliferae	1
\Atmosphere\Bioaerosol\Pollen\Urticaceae	2
\Atmosphere\Clouds	8
\Atmosphere\Clouds\Ice	4
\Atmosphere\Clouds\Liquid water	1
\Atmosphere\Clouds\Optical properties	4
\Atmosphere\Clouds\Position	3
\Atmosphere\Clouds\Type	4
\Atmosphere\Dynamics	3
\Atmosphere\Gas phase	5
\Atmosphere\Gas phase\Alcohol	2
\Atmosphere\Gas phase\Alkane	2
\Atmosphere\Gas phase\Greenhouse gas	8
\Atmosphere\Gas phase\Greenhouse gas	1
\Atmosphere\Gas phase\Greenhouse gas\CFCs	5
\Atmosphere\Gas phase\Greenhouse gas\Halocarbons	6
\Atmosphere\Gas phase\Greenhouse gas\Halocarbons\CI Species	10
\Atmosphere\Gas phase\Greenhouse gas\Halon	2
\Atmosphere\Gas phase\Greenhouse gas\HCFCs	5
\Atmosphere\Gas phase\Greenhouse gas\HFCs	11
\Atmosphere\Gas phase\Greenhouse gas\Inorganic nitrogen species	1
\Atmosphere\Gas phase\Greenhouse gas\Organic nitrogen species\Nitrile	10
\Atmosphere\Gas phase\Greenhouse gas\PFCs	3
\Atmosphere\Gas phase\Other gas	3
\Atmosphere\Gas phase\Reactive gas	17
\Atmosphere\Gas phase\Reactive gas\Nitrogen containing compounds	9
\Atmosphere\Gas phase\Reactive gas\Sulfur containing compounds	3
\Atmosphere\Gas phase\Reactive gas\VOC	72
\Atmosphere\Gas phase\Terpene	7
\Atmosphere\Humidity	7
\Atmosphere\Lightning	6
\Atmosphere\Lightning\Position	3
\Atmosphere\Particle phase	2
\Atmosphere\Particle phase\Composition	1
\Atmosphere\Particle phase\Composition\Inorganic anions	9
\Atmosphere\Particle phase\Composition\Inorganic carbonaceous	4
\Atmosphere\Particle phase\Composition\Inorganic cations	12
\Atmosphere\Particle phase\Composition\Inorganic nitrogen species	12
\Atmosphere\Particle phase\Composition\Major inorganic components	6
\Atmosphere\Particle phase\Composition\Organic anions	8
\Atmosphere\Particle phase\Composition\Organic carbonaceous	2
\Atmosphere\Particle phase\Composition\Trace elements	53
\Atmosphere\Particle phase\Optical properties	21
\Atmosphere\Particle phase\Physical properties	27
\Atmosphere\Particle phase\Physical properties - primary	1
\Atmosphere\POPs\PAH	18
\Atmosphere\POPs\POP	11
\Atmosphere\Precipitation	6
\Atmosphere\Pressure	3
\Atmosphere\Radiation	3
\Atmosphere\Radiation\IR	3
\Atmosphere\Radiation\Solar	4
\Atmosphere\Radiation\UV	5
\Atmosphere\Radionuclide	80
\Atmosphere\Remote sensing	4
\Atmosphere\Remote sensing\Optical properties	7
\Atmosphere\Temperature	4
\Atmosphere\Total Atmospheric Deposition	4
\Atmosphere\Total Atmospheric Deposition\Inorganic acid	1
\Atmosphere\Total Atmospheric Deposition\Inorganic anions	7
\Atmosphere\Total Atmospheric Deposition\Inorganic cations	4
\Atmosphere\Total Atmospheric Deposition\Inorganic nitrogen species	3
\Atmosphere\Total Atmospheric Deposition\Organic acid	3
\Atmosphere\Total Atmospheric Deposition\Trace elements	13
\Atmosphere\Visibility	2
\Atmosphere\Visibility\Obscurations	6
\Atmosphere\Wind	6
Atmosphere\Dynamics	1
Atmosphere\Gas phase\Greenhouse gas\CFCs	1
Atmosphere\Gas phase\Greenhouse gas\Halon	2
Atmosphere\Gas phase\Greenhouse gas\HCFCs	1
Atmosphere\Gas phase\Greenhouse gas\OVOC	18
Grand Total	675

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2023.06.01-TT-WIGOSMD/ notes:
Please review PR: #483
@joergklausen to review PR when ready;

[jbianchi81]

Apologies for joining the discussion late.

Overall the paths look OK but I've found one possible inconsistency:

165,\Terrestrial\Groundwater
12261,\Terrestrial\Hydrological\Groundwater\Soil

I suggest adding Hydrological after Terrestrial for all hydrological variables.
Also, in the following the order should be coherent:

12261,\Terrestrial\Hydrological\Groundwater\Soil
12262,\Terrestrial\Hydrological\Aquifer\Groundwater

I suggest maybe moving Groundwater before Aquifer or removing Groundwater
And in

12271,\Terrestrial\Hydrological\Liquid phase\Surface water

Hydrological\Liquid phase could be redundant, maybe we should eliminate Liquid phase

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2023.06.15-TT-WIGOSMD notes:
team discussed options for removing 'domain' from the paths; @joergklausen will review table 1-01-01 in PR

[amilan17]

https://github.com/wmo-im/tt-wigosmd/wiki/2023.06.30-TT-WIGOSMD notes:

Anna mentioned that this does not need to be ready for FT2023-2 amendment document and will ask Joerg to review when ready for feedback;

[amilan17]

Paths are complete and integrated into the master branch.