New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update order of actor and critic #43

Open

hccz95 opened this issue Sep 19, 2022 · 1 comment

hccz95 commented Sep 19, 2022

It seems that you update critic before actor.

As far as I know, the actor_loss is calculated through critic network, so the backward of actor_loss will influence the grad of critic parameters.

Should we update actor first, and then update critic using both actor_loss and critic_loss?

zhuzihan728 commented Apr 1, 2023

In the original paper, they update critic first tho.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment