check_estimator fails for XGBClassifier

[lrjball]

The following checks fail for me using xgboost==1.0.2:

from xgboost import XGBClassifier
from sklearn.utils.estimator_checks import check_estimator
check_estimator(XGBClassifier)

This means that XGBClassifier doesn't fully comply with the sklearn estimator API, so some functionality in sklearn (like gridsearches) may not work properly or give unexpected results. See here for more info from the sklearn docs.

Is this something which has been encountered before? I'm happy to have a go at a PR for this, unless it is something which is known and has been decided intentionally (for example to support **kwargs in the classifier).

[hcho3]

It is actually not possible to pass check_estimator() check entirely, because XGBoost allows NaN as part of input whereas scikit-learn estimators do not. I tried complying with the check in #3674 but found it untenable, so I reverted it.

[StrikerRUS]

@hcho3 It's actually possible but with some tricks (pass all but one-two checks from all that come from check_estimator()). Refer to https://github.com/microsoft/LightGBM/blob/18c706dc840c72341a2a85e05dfb5fad6366902a/tests/python_package_test/test_sklearn.py#L284-L305

Also, in recent versions of scikit-learn it is possible to use generators or parametrization for checks: microsoft/LightGBM#2628 (comment) and microsoft/LightGBM#2947 respectively.

But another problem is that even passing that test is not sufficient to be sure that your estimator is fully compatible with scikit-learn. Refer to scikit-learn/scikit-learn#15392 (comment) and microsoft/LightGBM#2966 (comment).