Fix WIT multiclass confusion matrix #2256
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@tolga-b please review |
tolga-b
approved these changes
May 21, 2019
stephanwlee
reviewed
May 22, 2019
| @@ -3176,8 +3176,8 @@ <h2>Show similarity to selected datapoint</h2> | |||
| const modelInferenceValueStr = | |||
| this.strWithModelName_(inferenceValueStr, modelInd); | |||
| let count = statsActual[item[modelInferenceValueStr]]; | |||
| allLabels.add(item[modelInferenceValueStr]); | |||
| allLabels.add(item[modelInferenceValueStr]); | |||
| allLabels.add(String(item[modelInferenceValueStr])); | |||
There was a problem hiding this comment.
Curious: what happens if you don't explicitly cast it to string?
There was a problem hiding this comment.
For users that provide "ground truth class ID" as a number to WIT, you may get a confusion matrix where rows and columns are doubled, that is that there is a row for Predicted class 0 (as an int) and a row for Prediction class 0 (as a string) separately. They both contain the same correct numbers for those confusion matrix cells, but the rows and columns are duplicated leading to an overly large and confusing confusing matrix.
stephanwlee
approved these changes
May 22, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In certain cases the confusion matrix was missing entries.
Correctly add both the inferred label and the true label to the set of all labels to show in the multiclass confusion matrix, including coercing all labels to strings.
Verified continued correct working in multiclass demo, and verified correct behavior in multiclass colab notebook that displayed this issue.