Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ActorCritic and DDPG #118

Merged
merged 49 commits into from
Apr 19, 2017
Merged

ActorCritic and DDPG #118

merged 49 commits into from
Apr 19, 2017

Conversation

lgraesser
Copy link
Collaborator

@lgraesser lgraesser commented Apr 8, 2017 •
edited by kengz
Loading

New Algorithms

ActorCritic

  • add ActorCritic agent
  • add its policies, Discrete: ArgmaxPolicy, SoftmaxPolicy; Continuous: BoundedPolicy, GaussianPolicy
  • add basic specs, solve Cartpole-v0, Cartpole-v1, yet to solve the others

DDPG

  • add DDPG agent with custom tensorflow ops
  • add its policies (only Continuous now): NoNoisePolicy, LinearNoisePolicy, GaussianWhiteNoisePolicy, OUNoisePolicy
  • add basic specs, solve Pendulum-v0

Improvements/Bug Fixes

  • use logger.warn instead of raise error when component locks are violated
  • fix matplotlib backend  #114, add OS X matplotlib backend #115 matplotlib backend setting issue. now single trial will live-plot and render
  • mute DoubleDQN as it breaks; instead revert to the single-model recompile from DQN

lgraesser and others added 30 commits April 8, 2017 17:17
@lgraesser
Working discrete actor critic model
a665bf3
@lgraesser
Uncommenting analyze data
c42d991
@kengz
style fix, scehdule ac experiment
6f76d42
@kengz
stylefix
2f5c642
@lgraesser
Adding AC specs
0c4bf3a
@kengz
Merge remote-tracking branch 'origin/master' into policy-gradient
6041209
@kengz
add component locks for ActorCritic and DDPG
1be0cb3
@kengz
schedule ac on cartpole and pendulum
e8aab7c
@kengz
reorder component locks
28d3cc8
@kengz
add ac discrete component lock, fix and check all ac specs
1565a7a
@kengz
add variance to pendulum gaussian search, narrow search space
e9ad662
@kengz
add action bounds to env_spec
c3c4ffb
@kengz
fix boundedpolicy to auto-bound from env-spec
7eaed26
@lgraesser
Adding Acrobot specs
34b1b58
@lgraesser
Merge branch 'policy-gradient' of https://github.com/kengz/openai_lab …
9b415c3 
…into policy-gradient
@kengz
schedule other ac experiments
3bd227f
@kengz
Merge branch 'policy-gradient' of https://github.com/kengz/openai_lab …
d11abfd 
…into policy-gradient
@lgraesser
Fixing mem len param
e8e6877
@lgraesser
Merge branch 'policy-gradient' of https://github.com/kengz/openai_lab …
3d553ee 
…into policy-gradient
@kengz
add ddpg fix attempt
75198e4
@kengz
Merge remote-tracking branch 'origin/master' into policy-gradient
edd2276
@kengz
ddpg with bounded actions
47a5f8c
@kengz
Merge remote-tracking branch 'origin/master' into policy-gradient
ee011b1
@kengz
permami broken with reshape to len manually
56e7f95
@kengz
fixing permami shape one at a time; absolutely disgusting code
6215058
@kengz
disgusting ddpg hack running
434de8b
@kengz
comment out print
4fa1dbc
@kengz
fucking got it, culprit was predicted_q_val shape
a691401
@kengz
fix permami typo
5a827c3
@kengz
runnable ddpg2 from permami, still not working yet
fd17088
@kengz kengz mentioned this pull request Apr 18, 2017
Closed
5 tasks
@kengz kengz changed the title Working discrete actor critic model ActorCritic and DDPG Apr 18, 2017
@kengz kengz merged commit 29bd213 into master Apr 19, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

matplotlib backend
2 participants
@lgraesser @kengz