Adding categorical cross_entropy loss to yaml config.

[Eirikalb]

After reading the recent paper from Google Deepmind I am really tempted to try their methods out myself to benchmark some Online RL problems in Omniverse Isaac Gym. https://arxiv.org/abs/2403.03950.

I see that there is some mention of experimental support for it in the code but I also see in the changenotes that crossentropy loss has not been added to the yaml config yet. What is the easiest/ best path for me to try it out using this framework. All my current experiments are based on standard MSE loss, using standard network architectures.

[Denys88]

Hi, I experimented with it here

rl_games/rl_games/algos_torch/network_builder.py

Line 171 in cba782c

def _build_value_layer(self, input_size, output_size, value_type='legacy'):

yes it is not in config. I tested TwoHot encoding. Feel free to try more options. I can add it to the yaml config.