Data Imputation Fix in erroranalysis
#2436
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Advitya17
requested review from
gaugup,
imatiach-msft,
RubyZ10,
vinuthakaranth,
tongyu-microsoft,
xuke444 and
hawestra
as code owners
gaugup
reviewed
Dec 4, 2023
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2436 +/- ##
==========================================
+ Coverage 89.74% 92.40% +2.66%
==========================================
Files 122 108 -14
Lines 6747 5415 -1332
==========================================
- Hits 6055 5004 -1051
+ Misses 692 411 -281
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
2 similar comments
3 similar comments
1 similar comment
imatiach-msft
approved these changes
Dec 13, 2023
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Generating the multilabel & OD dashboards from notebooks would result in an uncaught error statement:
Input X contains NaN. Traceback (most recent call last): File "c:\workspace\rai\responsible-ai-toolbox\erroranalysis\erroranalysis\analyzer\error_analyzer.py", line 480, in compute_importances importances = self._compute_error_correlation( File "c:\workspace\rai\responsible-ai-toolbox\erroranalysis\erroranalysis\analyzer\error_analyzer.py", line 519, in _compute_error_correlation return mutual_info_classif( File "c:\Users\agemawat\Anaconda3\envs\rai\lib\site-packages\sklearn\utils_param_validation.py", line 211, in wrapper return func(*args, **kwargs) File "c:\Users\agemawat\Anaconda3\envs\rai\lib\site-packages\sklearn\feature_selection_mutual_info.py", line 493, in mutual_info_classif return _estimate_mi(X, y, discrete_features, True, n_neighbors, copy, random_state) File "c:\Users\agemawat\Anaconda3\envs\rai\lib\site-packages\sklearn\feature_selection_mutual_info.py", line 258, in _estimate_mi X, y = check_X_y(X, y, accept_sparse="csc", y_numeric=not discrete_target) File "c:\Users\agemawat\Anaconda3\envs\rai\lib\site-packages\sklearn\utils\validation.py", line 1147, in check_X_y X = check_array( File "c:\Users\agemawat\Anaconda3\envs\rai\lib\site-packages\sklearn\utils\validation.py", line 959, in check_array _assert_all_finite( File "c:\Users\agemawat\Anaconda3\envs\rai\lib\site-packages\sklearn\utils\validation.py", line 124, in _assert_all_finite _assert_all_finite_element_wise( File "c:\Users\agemawat\Anaconda3\envs\rai\lib\site-packages\sklearn\utils\validation.py", line 173, in _assert_all_finite_element_wise raise ValueError(msg_err) ValueError: Input X contains NaN.
This was because the imputer in
erroranalysiswas not replacing nans due to non-numeric dtype. This PR enforces a numeric dtype for successful imputation and to remove this uncaught exception.Copilot:
This pull request to the
erroranalysis/erroranalysisrepository adds code to convert theinput_dataarray to thefloatdata type in thecompute_importancesmethod of theErrorAnalyzerclass. This change ensures error-free calculation and imputation of numerical data types.erroranalysis/erroranalysis/analyzer/error_analyzer.py: Added code to convertinput_dataarray tofloatdata type in thecompute_importancesmethod of theErrorAnalyzerclass.Description
Checklist