Updat value function with different action types, why?

[tessavdheiden]

Hi Shariq,

In your code you update the value function with actions computed by:

1. gumbel_softmax
2. onehot_from_logits

As far as I know, 1) has the gradient attached, while 2) does not.

Why did you implemented it this way?

[uhlajs]

Hi @tessavdheiden,
I believe that in one call of update you want to update actor just for one actor (namely for actor attach to agent_i). That is the reason why you send one action sample with gradient attached (representing the action of agent agent_i) and others without.