[Metrics] Add metrics on operand and subtask executions

[zhongchun]

What do these changes do?

• Add scheduling, band and op metrics

    mars.scheduling.submitted_subtask_count: The count of submitted subtasks to all bands.
    mars.scheduling.finished_subtask_count: The count of finished subtasks of all bands.
    mars.scheduling.canceled_subtask_count: The count of canceled subtasks of all bands.
    mars.band.submitted_subtask_number: The number of submitted subtask to a band.
    mars.band.unsubmitted_subtask_number: The number of unsubmitted subtask to a band.
    mars.operand.executed_number: The number of executed operands.

• Optimize metrics arguments check

Related issue number

Fixes #xxxx

Check code requirements

• tests added / passed (if needed)
• Ensure all linting tests pass, see here for how to run them

[qinxuye]

Some metrics named with suffix count, but some with number, looks a bit weird though, is it possible to unify the naming?

[zhongchun]

Some metrics named with suffix count, but some with number, looks a bit weird though, is it possible to unify the naming?

Thanks for you suggestion.
_count means metric type is Counter, _number means its type is Gauge. Just to distinguish them from their types.

[qinxuye]

Some metrics named with suffix count, but some with number, looks a bit weird though, is it possible to unify the naming?

Thanks for you suggestion. _count means metric type is Counter, _number means its type is Gauge. Just to distinguish them from their types.

OK, can we have a document about the naming specification about metrics? Just add it to https://github.com/mars-project/mars/tree/master/docs/source/development ?

[qinxuye]

LGTM overall, like I said, we can add a document about adding metrics.

[zhongchun]

LGTM overall, like I said, we can add a document about adding metrics.

Ok, I'll add docs soon.

[zhongchun]

Some metrics named with suffix count, but some with number, looks a bit weird though, is it possible to unify the naming?

Thanks for you suggestion. _count means metric type is Counter, _number means its type is Gauge. Just to distinguish them from their types.

OK, can we have a document about the naming specification about metrics? Just add it to https://github.com/mars-project/mars/tree/master/docs/source/development ?

Added in #2955, please take a look.

[wjsi]

The number of what kind of objects executed? Please make the name of the counter and the variable more explicit.

[zhongchun]

How about mars.operator_executed_number? And the description is The number of executed operators.

[wjsi]

Sorry but I've ignored the prefix.. Already merged.

[qinxuye]

LGTM

[wjsi]

LGTM