[rllib]Change in policy.py

[Emily0219]

What do these changes do?

I change the setup_loss function and correct the error about action space.

Related issue number

#1972

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5140/
Test PASSed.

[richardliaw]

Hm now I think about it, you might want something like

        if isinstance(action_space, gym.spaces.Box):
            self.ac = tf.placeholder(tf.float32, [None] +  list(action_space.shape), name="ac")
        elif isinstance(action_space, gym.spaces.Discrete):
            self.ac = tf.placeholder(tf.int64, [None], name="ac")
        else:
            raise NotImplementedError(
                "action space" + str(type(action_space)) +
                "currently not supported")

Can you test if this works for your use case?

[Emily0219]

Yes, I think you are right. I find the code is the same with that in /a3c/tfpolicy.py.

[richardliaw]

@Emily0219 would you be so kind to update the PR to match our above discussion?

Thanks!

[richardliaw]

Closed via #2012