Skip to content

Conversation

@pwolfram
Copy link
Contributor

@pwolfram pwolfram commented Oct 4, 2016

Following #65, cleans up unused mention of groupName.

@xylar
Copy link
Collaborator

xylar commented Oct 5, 2016

@pwolfram and @milenaveneziani, I'd prefer this not get merged until I get a chance to make sure I understand the usage of group names in MPAS tools. I'm not convinced we want to be removing groupName here just because it got removed elsewhere.

@pwolfram pwolfram closed this Oct 5, 2016
@pwolfram
Copy link
Contributor Author

pwolfram commented Oct 5, 2016

@xylar, this makes sense. We will need some type of way to inject groupName into features, however, because we currently have lost that capability. I'm going to close this for now because it sounds like we need more code using groupName, not less. We can always reopen it later if necessary.

@douglasjacobsen
Copy link
Member

@xylar, @milenaveneziani, and @pwolfram: Let me explain what groupName is supposed to be, why it was added, and how it's used.

First, the motivation...

When the horizontal stats AM was being created, it was pretty clear that we'd need some capability to allow us to perform regional statistics. The horizontal stats AM, specifically, performs horizontal averages within some region(s) and provides that information to you. When it was being created, it was also clear that we'd like a single instance of the horizontal stats AM to work over more than one region at a time, and assuming other AMs also were able to work over multiple regions, they should be able to work over independent sets of regions. A good example of this is that you might have the horizontal stats AM work over the Lab Sea and the southern ocean, while you might want the MOC AM to work over the Atlantic and the IndoPacific basins. However you wouldn't want them to work over the regions that the other is working on (this is more necessary when the number of regions is very large).

So, at the time the initial thought was the define a namelist attribute that was a list of all of the region names you wanted an AM to work over (something like config_horizontalStatsAM_region_list), however this seemed very impractical (to me at least) because this list could become very large if you did have an AM which you wanted computed over say 100 regions. At this point, you run into issues of (1) how to specify so many regions, and (2) how to ensure they don't run over the character limit of a string. So, I added the concept of "featureGroups" (I say feature here, but originally it was just region).

The purpose of groups now is that you can refer to several features at the same time with a single name. So, in the example I had above, let's say we named the Lab Sea and the Southern Ocean region group something like "horizStatsGroup" and the Atlantic/IndoPacific basins the "MOCGroup". Now these AMs could have options arguments that are something like config_horizontalStatsAM_region_name which could be any (1) Single region name or (2) Group name. Each AM could have their own group that they refer to, and all of these could be bundled into a single masks.nc file.

Now, that's how they are intended to be used. A bit about how to actually use them...

In this repository, when you (previously) run the merge_features.py script, it adds a "groupName" attribute to the top of the resulting features.geojson file. (If I had to guess, without actually looking, this was removed with the conversion to json.dump, but again that's just a guess). The groupName attribute should (in practice) take on a unique name that will allow you to refer to all of the regions contained within that file. After building up one group, you would backup that features file, and start creating another one by merging into a new file. You would repeat this process until all of your groups are created.

After you have one or more feature group files, you would move to using the mask creator tool. It's usage is as follows:
./MpasMaskCreator.x in_mesh.nc out_masks.nc -f feature_group1.geojson -f feature_group2.geojson ... -f feature_groupN.geojson

This tool takes an arbitrary number of -f flags which point to feature files representing groups of features you want in the output masks file. It will build the groups for you, and if their names are the default "enterGroupName" it will even give them unique names. Additionally, an extra group is always created, which is named all and contains all of the features which have been added to the file.

After that, usage within actual MPAS is really up to the individual analysis members to have a working implementation, but the idea for AMs which should operate over regions / transects / points is that they should:

  1. Take a region or a group name as a namelist option
  2. Assume it's a group name first, and search through the available groups.
  3. If a group is not found, search through the list of available regions.
  4. If a region is not found, there is an error and the model should abort.

Otherwise use whichever was found. If a region was found, build an internal group which only consists of that single region. If a group was found, get the relevant arrays from the regions pool and iterate over them.

Hopefully that helps. This might not be the most useful place for this information, so feel free to link it somewhere else if you want to.

@douglasjacobsen
Copy link
Member

Just a small note as well. After some bisecting, the merge that causes this problem is indeed a629bc2 from PR #52.

@milenaveneziani
Copy link
Collaborator

@douglasjacobsen: thank you so much for writing such a detailed explanation (and today of all days!). I guess I am only unclear about this step: I seem to understand that you need to feed the mask_creator with a geojson file that contains all the regions in a certain group (say maybe something like MOCgroup.geojson, containing all the *_MOC basins). Is that correct? If so, how do I create this merged, group-based feature? driver_scripts/setup_MOC_basins creates single *_MOC features, for example.
Thanks again.

@douglasjacobsen
Copy link
Member

@milenaveneziani So, if you wanted to create a group of features that represent all of the MOC basins, you might use the following workflow:

rm -f features.geojson
./driver_scripts/setup_MOC_basins.py
./merge_features.py -f IndoPacific_MOC.geojson
./merge_features.py -f Atlantic_MOC.geojson

At the end of this, you'll have a features.geojson file that is a single group, but contains both the Atlantic and the IndoPacific MOC regions.

This features.geojson file (assuming you don't want any more regions in this group) would then be one of the inputs to the mask creator with the -f flag.

@milenaveneziani
Copy link
Collaborator

ah! that's great. Thanks @douglasjacobsen!

@xylar
Copy link
Collaborator

xylar commented Oct 15, 2016

As I suggest in #73, ./driver_scripts/setup_MOC_basins.py and other similar driver scripts should include the merge commands that @douglasjacobsen suggests above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants