Ocean/moc streamfunction am

[DieMuhkuh]

Second try on merging this branch.

Please see the (rather short) discussion on the previous try:
#951

[DieMuhkuh]

@milenaveneziani I created the plot you requested.

I'd gladly attend the meeting on Monday and present some results.

[DieMuhkuh]

Gist for Jupyter notebook

As requested by @mark-petersen, I created a gist for the python script.
https://gist.github.com/DieMuhkuh/8f1e6774cb08771059815e372bae5979

[milenaveneziani]

Thanks @DieMuhkuh. That'd be great.

[DieMuhkuh]

1 Year averages

Global MOC

North Atlantic MOC

[douglasjacobsen]

For whoever deals with this PR in the future, you can't use the same buffer as the send and receive buffer in these calls. The compilers on mira fail to build / run with this, so another buffer needs to be created to store the global reductions.

[mark-petersen]

Thanks for looking at this. I'll be sure to use a different variable here.

[mark-petersen]

@DieMuhkuh please put the MOC regions geojson files on a gist and add a link here. Also, if you have examples of the netcdf files created with those geojson files (probably QU 120?) please copy those to the turquoise and provide a link.

[DieMuhkuh]

I just regained access to the turquoise network and retrieved the geojson file. The geojson file for the North Atlantic region is located here:
https://gist.github.com/DieMuhkuh/2bc2ee1289b2b3aa2791770f5c1c37c6
An example netCDF file for the EC30to60km grid can be found here:
/lustre/scratch1/turquoise/nilsfeige/EC30TO60restart.0010-01-01_00.00.00.nc

[milenaveneziani]

@DieMuhkuh: to further test this PR you should use the newer ocean basin regions (or you could start with the global domain). We should coordinate a bit on this.

[mark-petersen]

Doug wrote a 'linter' that checks for line length and white space, which can cause errors on other machines. Please run any *.F and *.xml files you have edited or added in the PR through the linter as follows:

wf-fe2.lanl.gov> pwd
/turquoise/usr/projects/climate/mpeterse/repos/MPAS/ocean_develop/src/core_ocean/analysis_members
wf-fe2.lanl.gov> module use /usr/projects/climate/SHARED_CLIMATE/modulefiles/all/
wf-fe2.lanl.gov> module load python/anaconda-2.7-climate
wf-fe2.lanl.gov> /turquoise/usr/projects/climate/mpeterse/repos/MPAS-Tools/MPAS-Tools/python_scripts/mpas_source_linter/mpas_source_linter.py  -f mpas_ocn_zonal_mean.F

You can use my mpas_source_linter.py or your own copy in MPAS-Tools. If there is lint, it will produce a *.e file with line numbers and errors. You should be able to correct them, delete *.e and rerun the linter. Imbedded xml files will have the following error, which is to be expected:

wf-fe2.lanl.gov> /turquoise/usr/projects/climate/mpeterse/repos/MPAS-Tools/MPAS-Tools/python_scripts/mpas_source_linter/mpas_source_linter.py -f Registry_zonal_mean.xml
wf-fe2.lanl.gov> cat Registry_zonal_mean.xml.e 
*** ERROR PARSING XML FILE *** 
Error is: junk after document element: line 35, column 1

 If this is a full file, the issue may be valid. If it's part of a file, you can probably ignore this error.

[mark-petersen]

@DieMuhkuh (Nils) please look at this PR to the geometric features repo:
issue: MPAS-Dev/geometric_features#69 (comment)
PR by @xylar: MPAS-Dev/geometric_features#75
You may need to fork that repo first.

Nils, could you test it with the RRS30to10 mesh and verify that the transect matches the boundary of the region?

[mark-petersen]

FYI to @xylar, @milenaveneziani and @pwolfram: Nils saw that MpasMaskCreator does not create transect groups, and has fixed it on a local branch of MPAS-Tools. We will make a PR soon. He needed that to test the transect group name in the MOC AM.

[xylar]

@DieMuhkuh and @mark-petersen, as I wrote in MPAS-Dev/geometric_features#69 (comment), I don't think the mask creator currently will behave as you want it to. That is, I don't think there's any guarantee that the southern boundary of a region as defined in geometric_features corresponds to the set of edges that are the southern boundary of the region as found by the mask creator.

In general, I think the way that the mask creator finds the path (cell centers, edges or vertices) that correspond to a transect in geometric_features is not what we would want, and likely needs to be fixed. At least when choosing cell centers to represent a transect, the mask creator seems to zig-zag fairly arbitrarily between points where the transect is actually defined, rather than taking the shortest path.

[mark-petersen]

@DieMuhkuh I merged this onto my merge branch for ACME alpha 9. Could you please get the branch ocean/test_merge_moc_streamfunction_AM from my fork and run your standard 240km test with MOC on for a few timesteps? Make sure everything works as expected. Try it with DEBUG=true to catch any memory leaks.

[DieMuhkuh]

@mark-petersen I ran a testrun with the branch you named and it worked out just fine. The testfolder and the AM results are in /lustre/scratch2/turquoise/nilsfeige/mpasRuns/ocean/global_ocean/QU_240km/default/forward. Please remember that there has to be a regionGroupName and transectGroupName defined in the namelist. Same goes for the transect transport AM. Using 'all' as the namelist entry for the transectGroupName will always fail (for both AMs).

[mark-petersen]

Merged into #1098: Ocean merge branch for ACME alpha 9, bit-for-bit
Please do not add commits to this PR without talking to me.