Adding runtime error related apis, metric aggregation, early bailout …

[dmlyubim]

Why are these changes needed?

Right now, at least TFGPU Multi (used in PPO) takes non-numeric metrics corresponding to first minibatch of an SGD step/last SGD step, the information for all other steps (or even minibatches in the last SGD step) is lost.

We want to be able to analyze and report runtime errors in any minibatch/sgd step. We also want to break SGD training early if we detected any fatal errors there.

So we add two apis

• to aggregate non-numeric metrics in ways other than just taking the one from last sgd iteration/first minibatch.
• to examine SGD evaluated fetches for runtime errors and break SGD training loop if we detect fatal errors (usually, loss errors that immediately would result to NaNs in weights of the policy, making further training impossible).

Related issue number

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[Edilmo]

LGTM