[python-package] ignore scikit-learn 'check_sample_weight_equivalence' estimator check (fixes #6678)

[jameslamb]

Fixes #6678

Fixes #6680

As of scikit-learn/scikit-learn#29818 (which will be in scikit-learn 1.6), scikit-learn contains an estimator check that enforces the following behavior:

check that setting sample_weight to zero / integer is equivalent to removing / repeating corresponding samples

This new check is breaking CI here... LightGBM does not work that way.

• weight=0.0 samples' feature values still contribute to feature distributions and therefore bin boundaries in Dataset histograms (How observations with sample_weight of zero influence the fit of LGBMRegressor #5553)
• every sample counts as exactly 1 sample for the perspective of count-based hyper-parameters like min_data_in_leaf ([R-package] Weighted Training - Different results (cross entropy) when using .weight column Vs inputting the expanded data (repeated rows = .weight times) #5626 (comment))

This PR proposes skipping that estimator check, just as scikit-learn is currently doing for HistGradientBoosting* estimators (code link).

Notes for Reviewers

We could modify LightGBM's scikit-learn estimators to match this expected behavior by excluding rows with weight=0 and creating copies of rows with int(weight)>=2 before passing it through to Dataset here:

LightGBM/python-package/lightgbm/sklearn.py

Lines 938 to 947 in bbeecc0

	train_set = Dataset(
	data=_X,
	label=_y,
	weight=sample_weight,
	group=group,
	init_score=init_score,
	categorical_feature=categorical_feature,
	feature_name=feature_name,
	params=params,
	)

I don't support doing that... I think it'd add significant complexity (do count-based hyperparameters need to be modified? how does this affect Dask estimators?) for questionable benefit.

I think just skipping this compliance check is the right thing for lightgbm right now.

[jameslamb]

While working on this, I discovered #6680.

Have reported that issue upstream: conda-forge/pyparsing-feedstock#48

For LightGBM's purposes, testing on an old version of Python, let's just pin it as we do other libraries in these environments.

[borchero]

Thanks!

[StrikerRUS]

I think just skipping this compliance check is the right thing for lightgbm right now.

Agree with you!

Thanks for the fast fix!