Bug fix in boundary communication

[lroberts36]

PR Summary

While debugging Riot, Josh and I found that there was an issue in the logic for which variables to do flux correction on. Previously, there was an implicit assumption that variables that had Metadata::WithFluxes set also had Metadata::FillGhosts set. There were also potentially issues if only Metadata::FillGhosts was set. This PR fixes those issues, so that flux correction is only done for all variables with Metadata::WithFluxes while boundaries are still filled for all variables that have Metadata::FillGhosts.

@pgrete: You may want to try this with issue #659, although the appearance of that issue may have happened before the sparse boundary update.

@Yurlungur: I think this change shouldn't have any impact on how you are using ForEachBoundary in your prolongation PR, but the two probably require merging (sorry).

ps: I also made a couple of small changes to the Burger's equation benchmark to stop the linter from complaining.

PR Checklist

• Code passes cpplint
• Adds a test for any bugs fixed.
• Code is formatted
• Changes are summarized in CHANGELOG.md
• CI has been triggered on Darwin for performance regression tests.
• (@lanl.gov employees) Update copyright on changed files

[Yurlungur]

You may want to try this with issue #659, although the appearance of that issue may have happened before the sparse boundary update.

We may have found the source of @pgrete's bug while I was in Hamburg. It's different than this but also nasty. I will let @pgrete describe after he's done some checks to ensure we did find it.

[Yurlungur]

Sorry---beat you to it. Userspace prolongation is in. Up to you to do the merge. ;)

This looks entirely reasonable.

[pgrete]

I'll review once the merge conflict is resolved (I want you make sure that I don't break the updated logic here).

[lroberts36]

@pgrete, the merge is done. Should be ready for review whenever you get the chance.

[pgrete]

It took a while for my brain to parse the logic, but I now think that this looks good.

Out of curiosity, is there a use-case where Metadata::WithFluxes would be used without Metadata::FillGhosts?

Also, I didn't see any changes to the burgers benchmark so I assume that those changes are/were not required any more.

[lroberts36]

@pgrete: Thanks for the review.

I guess if your fluxes only depended on dependent variables, you could reconstruct only dependent variables (so they would just have FillGhosts) while independent, conserved variables would still require flux arrays (so they would just have WithFluxes set).

The burgers changes must have been fixed in another commit that got merged in before this one.