Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Commit

Permalink
[MXNET-1399] multiclass-mcc metric enhancements (#14874)
Browse files Browse the repository at this point in the history
* multiclass-mcc metric enhancements

 * Rename metric from "PCC" to "mMCC" because though the math is derived
   from Pearson CC, it's utility is as a multiclass extension of Mathews CC.
 * Harden mx.metric.mMCC.update to more variations of input format, similar to
   mx.metric.Accuracy.update.

* Harden mx.metric.PCC.update to more variations of input format, similar to mx.metric.Accuracy.update.

* Enhance testcases for mx.metric.PCC.
  • Loading branch information
tlby authored and lanking520 committed Aug 29, 2019
1 parent 2d86c70 commit 196d1f4
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 1 deletion.
6 changes: 5 additions & 1 deletion python/mxnet/metric.py
Original file line number Diff line number Diff line change
Expand Up @@ -1572,7 +1572,11 @@ def update(self, labels, preds):
# update the confusion matrix
for label, pred in zip(labels, preds):
label = label.astype('int32', copy=False).asnumpy()
pred = pred.asnumpy().argmax(axis=1)
pred = pred.asnumpy()
if pred.shape != label.shape:
pred = pred.argmax(axis=1)
else:
pred = pred.astype('int32', copy=False)
n = max(pred.max(), label.max())
if n >= self.k:
self._grow(n + 1 - self.k)
Expand Down
7 changes: 7 additions & 0 deletions tests/python/unittest/test_metric.py
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,13 @@ def test_pcc():
_, pear = met_pear.get()
np.testing.assert_almost_equal(pcc, pear)

# pcc should also accept pred as scalar rather than softmax vector
# like acc does
met_pcc.reset()
met_pcc.update(labels, [p.argmax(axis=1) for p in preds])
_, chk = met_pcc.get()
np.testing.assert_almost_equal(pcc, chk)

# check multiclass case against reference implementation
CM = [
[ 23, 13, 3 ],
Expand Down

0 comments on commit 196d1f4

Please sign in to comment.