Use the generic oops interpolator

[travissluka]

Description

• closes Have interpolation return missing values on land #542

Deletes the custom SOCA interpolation and uses the generic OOPS interpolation that @fmahebert put together, resulting in 768 fewer lines of SOCA code 🎉

In order to use the OOPS interpolation I had to fix the way halo regions were handled when being passed around atlas fieldsets

• Previously, the halo could contain points that were duplicated on the PE, either in the halo elsewhere or in the non-halo region. This wouldn't work with the OOPS interpolation because it can't handle duplicate points. I now construct an array geom%valid_halo_mask that is true for the non-halo gridpoints and for the non-duplicate halo gridpoints (see soca_geom_find_invalid_halo()). Anytime a field is packed for atlas fieldset usage this mask is used.
• It turns out we have invalid halo points being used and didn't even know it! (Not sure if this ever caused any real problems 🤷‍♂️ ). It seems that a small handful of halo points are not updated by the fms halo update calls, because they aren't used by mom6. I now set the geom lat/lon to INVALID_HALO (-999.0) instead of 0.0 before filling it with the correct lat/lon in the compute domain and doing the first halo update. I then test in soca_geom_find_invalid_halo() to see where there are still invalid lat/lon in the halo then set geom%valid_halo_mask to false for those gridpoints.

The one limitation, for now, is that all interpolation uses the same grid. Meaning U/V are assumed to be on the h grid. Options (in a subsequent PR) are

1. give up the desire to have interpolation work on the native U/V grid. We don't even have u/v observations! We'll just make sure u/v are destaggered before running hofx
2. I can maybe add back in some logic to have multiple oops interpolators constructed so that we can interpolate on the native u/v grid. I don't like this option, because it puts back in more soca-specific code.

We don't have to decide now. Just keep in mind if/when it is ever realistic that we'll have u/v observations.

Testing

• Answers are different, as one would expect from changing the interpolation. On the plus side, the minimum coolskin temp is no longer -69 for hofx3d, but now a more realistic -4. Also, the interpolated sea ice concentration no longer goes above 1.0

• The getvalues test has been mostly disabled (the masked fields were turned off). This is because of a limitation in the unit test. I need to talk to @fmahebert sometime to see how to fix this. (In short, I can't create proper soca analytic init values for the state and geovals because I never know for sure if a point near land will give a missing value or an interpolated value, it depends randomly on how the triangulation is performed for the oops interpolation) I'll deal with that "later"

• There is a "bug" with LETKF not being able to handle missing values the same way the var does. The workaround for this is to make sure the landmark qc filter is in place for all letkf obs (which they should have been anyway)

• Once I'm sure the CI tests are passing I'll run comparisons with a realistic domain and post some plots**

Dependencies

• requires matching branch in UFO, (https://github.com/JCSDA-internal/ufo/pull/2875)

[guillaumevernieres]

🎉

[travissluka]

note, I forgot to update 2 marine operators, I have updated the matching UFO PR. To make sure I have them all covered now, I updated the input obs files here to contain a single point at lat/lon 40.035, -105.243 which should definitely be land, otherwise the JCSDA building would be underwater.

[kbhargava]

Builds fine and ctests pass.

[travissluka]

Looking at SST increment with the old code (left) and new interpolation (right)

There are small difference that seem reasonable given the big change in interpolation method. (new - old)

[travissluka]

And looking at sst O-B for the old code (top) and new code (bottom) the "land ocean" points are now properly not having an hofx calculated. This should make diagnostics a lot easier going forward!

[kbhargava]

Looking at SST increment with the old code (left) and new interpolation (right)

There are small difference that seem reasonable given the big change in interpolation method. (new - old)

Nice!!!