Refactor RF objectives by objective family

[RAMitchell]

Summary

Refactors the batched-level random forest objective dispatch from one objective class/kernel instantiation per split criterion to one objective class/kernel instantiation per objective family.

This reduces the explicit RF objective kernel instantiations from 12 criterion-specific files to 4 family-specific files:

• classification float/double
• regression float/double

The individual gain formulas remain separate named functions, but the outer split-bin iteration is shared within each objective family and dispatches on the criterion at runtime.

Motivation

This reduces compile-time work and prepares the objective layer for future sample-weight support without introducing weight semantics in this PR.

From the local Ninja build log, summed compile time for the RF objective instantiation objects dropped from approximately:

• before: 669.99s across 12 criterion-specific objects
• after: 110.36s across 4 family-specific objects

That is roughly an 84% reduction in per-object objective-instantiation compile work. This is not a full clean-build wall-clock measurement, but it captures the targeted compile-time improvement from reducing these explicit instantiations.

Details

• Adds ClassificationObjectiveFunction for Gini and entropy.
• Adds RegressionObjectiveFunction for MSE, Poisson, Gamma, and Inverse Gaussian.
• Passes split_criterion into the objective object from the tree builder.
• Updates decision tree fit dispatch to select classification vs regression objective families.
• Replaces criterion-specific kernel instantiation files with family-specific instantiation files.

This PR does not change sample-weight behavior or add weighted histogram bins.

Verification

• git diff --check upstream/main..HEAD
• cmake --build cpp/build-ninja-gcc12 --target cuml_objs -j 8

[RAMitchell]

@switch527 thanks for motivating this. I think I have the solution now and will implement weights faithfully to sklearn.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 89301b39-b9e9-4dcd-a1ab-5cfe66907811

📥 Commits

Reviewing files that changed from the base of the PR and between 57d5c78 and d264d94.

📒 Files selected for processing (1)

• cpp/tests/sg/rf_test.cu

🚧 Files skipped from review as they are similar to previous changes (1)

• cpp/tests/sg/rf_test.cu

---

📝 Walkthrough

Summary by CodeRabbit

• Refactor

  • Unified decision-tree split scoring: classification and regression now use a single, criterion-driven implementation that selects scoring at runtime.
  • Streamlined kernel variants and registrations to align with the unified objective flow.

• Tests

  • Updated objective tests and configurations to match the new criterion-driven dispatch.

Walkthrough

Consolidates per-criterion objective implementations into ClassificationObjectiveFunction and RegressionObjectiveFunction, wires the runtime split_criterion through builder/fit/kernels, updates tests to parameterize by criterion, and replaces per-criterion kernel sources with consolidated classification/regression kernel files.

Changes

Decision Tree Objective Consolidation

Layer / File(s)	Summary
Classification objectives refactoring cpp/src/decisiontree/batched-levelalgo/objectives.cuh	ClassificationObjectiveFunction unifies GINI/ENTROPY scoring with stored criterion, adds GiniGain/EntropyGain, CountLeft, and criterion-dispatched Gain iteration.
Regression objectives refactoring cpp/src/decisiontree/batched-levelalgo/objectives.cuh	RegressionObjectiveFunction stores criterion and eps_, consolidates MSE/Poisson/Gamma/InverseGaussian gains with per-criterion helpers and eps_ validity checks, and dispatches via GainPerSplit.
Kernel entry-point updates cpp/src/decisiontree/batched-levelalgo/kernels/classification-{double,float}.cu, cpp/src/decisiontree/batched-levelalgo/kernels/regression-{double,float}.cu, removed/emptied per-criterion kernels	Classification kernels now alias _ObjectiveT = ClassificationObjectiveFunction; regression kernels alias _ObjectiveT = RegressionObjectiveFunction; multiple per-criterion kernel files were removed or emptied and file headers updated where present.
Builder wiring of criterion parameter cpp/src/decisiontree/batched-levelalgo/builder.cuh	computeSplit and SetLeafPredictions construct ObjectiveT with params.split_criterion.
High-level fit dispatch consolidation cpp/src/decisiontree/decisiontree.cuh	DecisionTree::fit replaces per-criterion dispatch with two runtime branches that instantiate builders with ClassificationObjectiveFunction or RegressionObjectiveFunction based on types and split_criterion.
Build system kernel source update cpp/CMakeLists.txt	decisiontree_algo target sources updated to include classification-{double,float}.cu and regression-{double,float}.cu and to remove per-criterion kernel files.
Tests refactor for criterion-driven objectives cpp/tests/sg/rf_test.cu	DT::ObjectiveTest parameterized by ObjectiveTestConfig<ObjectiveT, CRITERION>; tests construct objectives with the criterion, use shared eps_, and dispatch ground-truth via compile-time splitCriteria; test instantiations updated per criterion.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related issues

• [FEA] Add class_weight + sample_weight to RandomForest estimators #8093 — touches the same batched-levelalgo objective/kernel implementation and may overlap with these objective/builder changes.

Suggested reviewers

• hcho3
• divyegala
• dantegd

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Refactor RF objectives by objective family' clearly and specifically summarizes the main change: reorganizing objective dispatch from per-criterion to per-family instantiation.
Description check	✅ Passed	The description is directly related to the changeset, explaining the refactoring from 12 criterion-specific files to 4 family-specific files, implementation details, motivation (compile-time reduction), and verification steps.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/tests/sg/rf_test.cu`:
- Around line 1392-1403: The test currently calls compile-time helpers like
GainPerSplit<ObjectiveConfig::splitCriteria> and MSEGroundTruthGain directly, so
add at least one assertion that exercises the runtime dispatch path: construct
an objective/family instance that stores the criterion (the same enum value used
in the test), call its runtime Gain(data, split_bin_index) (or Gain() method on
the objective object that switches on objective.criterion), and assert the
result equals the compile-time GainPerSplit result (e.g., compare
objective.Gain(...) to GainPerSplit<ObjectiveConfig::splitCriteria>(...)). Do
this for at least one criterion (reuse MSE or another from the existing list) so
the test validates that the stored criterion drives the same gain formula as the
compile-time helpers. Ensure you reference the same data and split_bin_index
values used in the existing assertions.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f56f43f4-e32a-4c49-a407-6cedf732b525

📥 Commits

Reviewing files that changed from the base of the PR and between c048c5d and 66efd8c.

📒 Files selected for processing (17)

• cpp/CMakeLists.txt
• cpp/src/decisiontree/batched-levelalgo/builder.cuh
• cpp/src/decisiontree/batched-levelalgo/kernels/classification-double.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/classification-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/gamma-double.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/gamma-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/gini-double.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/gini-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/inverse_gaussian-double.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/inverse_gaussian-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/poisson-double.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/poisson-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/regression-double.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/regression-float.cu
• cpp/src/decisiontree/batched-levelalgo/objectives.cuh
• cpp/src/decisiontree/decisiontree.cuh
• cpp/tests/sg/rf_test.cu

💤 Files with no reviewable changes (8)

• cpp/src/decisiontree/batched-levelalgo/kernels/gini-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/poisson-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/inverse_gaussian-double.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/poisson-double.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/inverse_gaussian-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/gamma-double.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/gamma-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/gini-double.cu

🚧 Files skipped from review as they are similar to previous changes (8)

• cpp/CMakeLists.txt
• cpp/src/decisiontree/batched-levelalgo/kernels/classification-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/regression-float.cu
• cpp/src/decisiontree/batched-levelalgo/kernels/regression-double.cu
• cpp/src/decisiontree/decisiontree.cuh
• cpp/src/decisiontree/batched-levelalgo/builder.cuh
• cpp/src/decisiontree/batched-levelalgo/kernels/classification-double.cu
• cpp/src/decisiontree/batched-levelalgo/objectives.cuh

[switch527]

@RAMitchell, I appreciate you picking it up. This is going to make forest estimators much more powerful for my use cases when everything is buttoned up.

[dantegd]

The compile-time improvement is clear and great to see, didn't find any correctness issues. One thing that would be helpful to document is a quick train-time sanity check for common Gini/MSE cases. A small before/after would make any compile-time/runtime tradeoff explicit.

[dantegd]

What do you think about driving this through objective.Gain(...), or adding a focused test for Gain, now that the shared split-bin loop lives there? Calling GainPerSplit still validates the per-criterion formula dispatch, but it bypasses the min-samples guard and the shared outer iteration that this PR is mainly refactoring.

[RAMitchell]

I wanted to do this but objective.Gain is a device function so it means rewriting all of the tests to go through a kernel. The min samples parameter is definitely tested elsewhere.

[RAMitchell]

Benchmarking shows only minor differences.

Case	Before	After	Delta
Gini, 2-class, 128 bins	2628.32 ms mean, 2622.96 ms median (n=5)	2648.33 ms mean, 2648.97 ms median (n=5)	+0.76%
MSE, 128 bins	9155.68 ms mean, 9143.93 ms median (n=3)	9339.67 ms mean, 9337.94 ms median (n=3)	+2.01%

[RAMitchell]

/merge