Efficient MXNet sampling in the multinomial distribution

[zixuanweeei]

Description

This PR has solved the problem of low efficiency to sample from the multinomial distribution, especially the distribution with a large number of possible outcomes. As it was described in #15231, it costs ~44 s using mx.nd.random.multinomial, while np.random.multinomial costs ~0.019 s. After our optimization, it decreases to ~0.054 s.

Feature changes

New features

• Efficient sampling strategy for the multinomial distribution using binary search.
• Use double type to define some cumulative variables for high precision with very small probabilities.
• Reserve the CDF array for sampling many times without duplicated addition operations.

Performance

We compared the time costs of mx.nd.random.multinomial on branch master(b8b352d) and our PR shown in the table below. The result shows that the time cost of the original sampling strategy rises much more rapidly than that of our PR with the increasing number of outcomes.

Number of outcomes	Master(b8b352d) Costs (ms)	PR(6c3c49b) Costs (ms)
10	0.125	0.066
100	0.270	0.058
1000	0.729	0.130
10000	62.717	0.996
100000	5372.306	12.125
1000000	-	173.534

Comments

@pengzhao-intel @ciyongch @TaoLv Please help me refine this PR and have some review on it. Thanks.

Check list

• Passed code style checking (cpplint).
• Unit test passed.
• Code is well-documented.

[pengzhao-intel]

@reminisce @wkcn for review.

[TaoLv]

Nice work! @zixuanweeei
@chinakook @stu1130 You might be interested.

[wkcn]

Great work. LGTM. Thank you!

[stu1130]

Brillant! LGTM

[chinakook]

Thanks, LGTM!

[wkcn]

Merged. Thanks for your contribution!