[CUDA] Random number generation on CUDA

[shiyu1994]

Summary

Support generation of random numbers on CUDA.

Motivation

Sometimes random number generation is repeatedly called during the training process. For example, in the rank_xendcg objective. Supporting generation of random numbers on CUDA can accelerates these processes.

References

See the code snippet below for random number generation in boosting with rank_xendcg

LightGBM/src/objective/rank_objective.hpp

Lines 301 to 328 in f0e3296

	inline void GetGradientsForOneQuery(data_size_t query_id, data_size_t cnt,
	const label_t* label, const double* score,
	score_t* lambdas,
	score_t* hessians) const override {
	// Skip groups with too few items.
	if (cnt <= 1) {
	for (data_size_t i = 0; i < cnt; ++i) {
	lambdas[i] = 0.0f;
	hessians[i] = 0.0f;
	}
	return;
	}
	// Turn scores into a probability distribution using Softmax.
	std::vector<double> rho(cnt, 0.0);
	Common::Softmax(score, rho.data(), cnt);
	// An auxiliary buffer of parameters used to form the ground-truth
	// distribution and compute the loss.
	std::vector<double> params(cnt);
	double inv_denominator = 0;
	for (data_size_t i = 0; i < cnt; ++i) {
	params[i] = Phi(label[i], rands_[query_id].NextFloat());
	inv_denominator += params[i];
	}
	// sum_labels will always be positive number
	inv_denominator = 1. / std::max<double>(kEpsilon, inv_denominator);

[shiyu1994]

Closed in favor of being in #2302. We decided to keep all feature requests in one place.

Welcome to contribute this feature! Please re-open this issue (or post a comment if you are not a topic starter) if you are actively working on implementing this feature.

[jameslamb]

Sorry, this was locked accidentally. Just unlocked it. We'd still love help with this feature!