Add optional normalization (cont.)

[maxreciprocate]

• change scaling reward to have three options: ref/running/none
• add single thread test of running mean calculations

https://wandb.ai/sorry/public/runs/2x8totg2

[Dahoas]

I just noticed all the unittests are no longer being run upon pr submission. Do we know why?

[Dahoas]

Things look good. Can you just provide a way for users to optionally specify a ref_mean and ref_var in the ppo config? You can make them Optional[float] parameters defaulting to false in the PPOMethodConfig

[Dahoas]

Also louis will want you to address all his comments from before

[maxreciprocate]

I just noticed all the unittests are no longer being run upon pr submission. Do we know why?

#92 was the last pr on which unittests ran, I can fix it in this pr
it's about master->main change

trlx/.github/workflows/build.yml

Line 7 in 12598e9

branches: [ master ]

Also louis will want you to address all his comments from before

I think I have haven't I, i.e. running var calculation and storing ref_*

[Dahoas]

@LouisCastricato Are you happy with the comments?

Once the ppoconfig is modified to accept pre-computed ref_mean, ref_var I'm good to merge

[LouisCastricato]

I think this is ok to merge