Preliminary offline tracer capability

[ashao]

The changes introduced here allow for tracers to be done offline using the MOM6 framework mainly through the subroutine step_tracers. Associated with this pull request is also an addition to MOM6-examples that adds a Baltic_ALE_z_offline_tracers test case with instructions on how to run tracers offline. The information below can also be found in MOM_offline_control.F90 and in the README.offline in the Baltic_ALE_z_offline_tracers test case

OVERVIEW

'Offline tracer modeling' uses physical fields (e.g. mass transports and layer thicknesses) saved
from a previous integration of the physical model to transport passive tracers. These fields are
accumulated or averaged over a period of time (in this test case, 1 day) and used to integrate
portions of the MOM6 code base that handle the 3d advection and diffusion of passive tracers.
This capability has currently targeted the Baltic_ALE_z test case, though some work has also been
done with the OM4 1/2 degree configuration. Work is ongoing to develop recommendations and best
practices for investigators seeking to use MOM6 for offline tracer modeling.

The subroutine step_tracers that coordinates this can be found in MOM.F90 and is only called
using the solo ocean driver. This is to avoid issues with coupling to other climate components
that may be relying on fluxes from the ocean to be coupled more often than the offline time step.
Other routines related to offline tracer modeling can be found in tracers/MOM_offline_control.F90

As can also be seen in the comments for the step_tracers subroutine, an offline time step
comprises the following steps
1) Using the layer thicknesses and tracer concentrations from the previous timestep,
half of the accumulated vertical mixing (eatr and ebtr) is applied in the call to
tracer_column_fns.
For tracers whose source/sink terms need dt, this value is set to 1/2 dt_offline
2) Half of the accumulated surface freshwater fluxes are applied
START ITERATION
3) Accumulated mass fluxes are used to do horizontal transport. The number of iterations
used in advect_tracer is limited to 2 (e.g x->y->x->y). The remaining mass fluxes are
stored for later use and resulting layer thicknesses fed into the next step
4) Tracers and the h-grid are regridded and remapped in a call to ALE. This allows for
layers which might 'vanish' because of horizontal mass transport to be 'reinflated'
and essentially allows for the vertical transport of tracers
5) Check that transport is done if the remaining mass fluxes equals 0 or if the max
number of iterations has been reached
END ITERATION
6) Repeat steps 1 and 2
7) Force a remapping to the stored layer thicknesses that correspond to the snapshot of
the online model at the end of an accumulation interval
8) Reset T/S and h to their stored snapshotted values to prevent model drift

---

EVALUATING

A framework for formally regression testing the offline capability still needs to be developed.
However, as a simple way of testing whether the offline model is nominally behaving as expected,
the total inventory of the advection test tracers (tr1, tr2, etc.) should be conserved between
time steps except for the last 4 decimal places.

[ashao]

I'm going to close this pull request for now, until I can figure out why I get answers changing between dev/master and my own branch.

[ashao]

Turns out that it was just because of compiler options on my workstation. This should now be good to go.

[adcroft]

Heads up - this does not do what you think it does!