-
Notifications
You must be signed in to change notification settings - Fork 126
Passed Predict Params to RAPS for conformity score calculation and EnsembleClassifier #784
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -52,4 +52,5 @@ Contributors | |
| * Faustin Pulvéric <[email protected]> | ||
| * Chaoqi Zhang <[email protected]> | ||
| * Leena Kamran Qidwai | ||
| * Aman Vishnoi <[email protected]> | ||
| To be continued ... | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1764,6 +1764,42 @@ def test_predict_parameters_passing() -> None: | |
| np.testing.assert_equal(y_pred, 0) | ||
|
|
||
|
|
||
| def test_raps_with_predict_params() -> None: | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This test does not seem to test what we want. It does not test if predict params are passed into RAPS and actually this test passes on the current version of MAPIE (i.e., without your modifications). Furthermore, the code is too complex: there is no need to recreate a dataset. Please refer to e.g., |
||
| """Test that predict_params are correctly passed when using RAPS.""" | ||
| X, y = make_classification( | ||
| n_samples=500, | ||
| n_features=10, | ||
| n_informative=3, | ||
| n_classes=3, | ||
| random_state=random_state, | ||
| ) | ||
| X_train, X_test, y_train, y_test = train_test_split( | ||
| X, y, test_size=0.2, random_state=random_state | ||
| ) | ||
| estimator = CustomGradientBoostingClassifier(random_state=random_state) | ||
| predict_params = {"check_predict_params": True} | ||
| mapie_clf = _MapieClassifier( | ||
| estimator=estimator, | ||
| conformity_score=RAPSConformityScore(size_raps=0.2), | ||
| cv="split", | ||
| test_size=0.2, | ||
| random_state=random_state, | ||
| ) | ||
|
|
||
| mapie_clf.fit(X_train, y_train, predict_params=predict_params) | ||
|
|
||
| y_pred, y_ps = mapie_clf.predict( | ||
| X_test, | ||
| alpha=0.1, | ||
| include_last_label="randomized", | ||
| agg_scores="mean", | ||
| **predict_params, | ||
| ) | ||
| # Ensure the output shapes are correct | ||
| assert y_pred.shape == (X_test.shape[0],) | ||
| assert y_ps.shape == (X_test.shape[0], len(np.unique(y)), 1) | ||
|
|
||
|
|
||
| def test_with_no_predict_parameters_passing() -> None: | ||
| """ | ||
| Test passing with no predict parameters from the | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't you only need to add
**predict_paramsto the function? i.e,:self.y_pred_proba_raps = self.predictor.single_estimator_.predict_proba(self.X_raps, **predict_params)like you did in theclassifier.pyfile.