Data additions for v1

[jhamman]

We had a very productive call today discussing the data configurations we want to test while adding the v1 data layers to this map tool. This issue documents the outputs of that call:

v0_emissions (3d) -> already have
2001-2020

biomass (3d) -> already have
2014-2020

v1_emissions (4d) -> need to make
2015-2020

biomass + v0_emissions + v1_emissions (4d) -> need to make
mixed time axes

[katamartin]

I started to initially prototype with the 3D v0_emissions and biomass datasets and realized with @freeman-lab that we want to try implementing with combined 4D biomass +v0_emissions + v1_emissions dataset sooner rather than later. This is for a couple of reasons:

1. Mixing datasets means that we'll be fetching chunks as layers are turned on and off whereas this data will be prefetched in a combined dataset. We're operating with the assumption that a decent portion of users are going to look at >1 layer, and combining into one dataset should perceivably improve that toggling experience.
2. We have some real concerns to work through in @carbonplan/maps to get rendering multiple datasets working properly (see rendering multiple rasters maps#13 for more context).

Something else to consider: are we intent on showing all possible years across all datasets? This would really simplify the UI for distinct time domains and, especially in the 4D approach, decrease the amount of data we need to load. Curious for your thoughts here in particular @tcchiao @orianac! The UI concerns can definitely be solved given a little thought.

[orianac]

Sounds good @katamartin!

re: the time extent question, it'd be nice to be able to show the 2001-2014 portion for v0. Though restricting the emissions to just the intersecting time period of 2015-2020 would be fine, and make v0 look more like a comparison dataset as opposed to a core product. It would introduce a tiny bit of backward compatibility in terms of the blog post (we said it was available 2001-2020) but that might be unimportant in the grand scheme of things. Out of curiosity, would it be possible to have the 2001-2014 portion as an a separate layer that you could optionally turn on? Would that help with anything?

A less extreme example of the time extent inconsistency is that for the use case of comparing biomass and v1 emissions, people are likely going to be comparing emissions at time t to biomass at time t-1. That means it'd be helpful to have one additional timestep for biomass compared to v1 emissions (e.g. start in 2014 for biomass and 2015 for v1 emissions). We could truncate them, but it makes the datasets a little less comparable (just because we'd end up missing the 2014 benchmark year).

[freeman-lab]

Great discussion! A couple thoughts to add:

• 100% agree on wanting to see the delta in that initial year, so I think the question is do we truncate to 2014-2020 or do the full extent of 2001-2020, we should not truncate to 2015-2020

• I'm personally not bothered with respect to the blog post mentioning 2001 because we do still share the dataset (which goes all the way back) and have example notebooks that load it

• I could make the case that once v1 is in the webmap (biomass and emissions) that dataset should really be the focus of the UI, in which case having 2/3 of the extent of the time range dedicated to a dataset that's not the primary focus feels a little odd

[katamartin]

Out of curiosity, would it be possible to have the 2001-2014 portion as an a separate layer that you could optionally turn on?

I could totally see this as a separate toggle only enabled when looking at v0 data! I think this could help with making the data viewable in the tool but not drawing attention to it by default (which we definitely would by making the default, top-level time slider 2001-2020).

Separately, started a very rough draft PR #106 for pulling in the v1 data, which is mostly working! One small issue we noticed was we're specifying variable with chunks: [2, 6, 128, 128], but it's actually [3, 6, 128, 128], which means that we get no data trying to view the 3rd band. Is it possible to quickly update that in this version of the data (or in the next rev if that's soon to come!)?

[freeman-lab]

just to expand on the chunk point, in general we want just one chunk across the non-spatial dimensions, so the chunk size should always match the array size on those dimensions, i.e. if with more layers the shape is [5, 6, ...] then chunk size should be [5, 6, ...] and so on

we will surely relax that when we figure out how to stream chunked data on demand along non-spatial dimensions, but we're not quite there yet :)

[freeman-lab]

and if we're going to keep 2001-2014 for v0, i do like the idea of having the time slider only show the longer range when v0 is selected (and if you have slided to an earlier year and then select a different layer, it'll jump to 2014)

especially cause you'll only be able to select one at a time, that seems pretty clean to me!

[katamartin]

Documenting conversation from yesterday: the plan is to show data from 2014-2020.

[katamartin]

Closing this issue out for now while we continue to work with the v1 data.