-
Notifications
You must be signed in to change notification settings - Fork 285
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Add CQL #1239
[Feature] Add CQL #1239
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work! Exciting stuff
A few comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work! Exciting stuff
A few comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesoooome!
let's wait for the tests to pass :)
I'm getting an error now regarding the in_keys of the objective, checking about that! Fixed! |
Description
Implements the Conservative Q-Learning (CQL) objective for Offline Reinforcement Learning (https://arxiv.org/abs/2006.04779)
and adds examples for online and offline training.
Performance tested for Pendulum:
![image](https://private-user-images.githubusercontent.com/29492081/244328088-ba22f80f-ce5b-4496-952b-5a8ece54b6a8.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ1MzY5MjcsIm5iZiI6MTcyNDUzNjYyNywicGF0aCI6Ii8yOTQ5MjA4MS8yNDQzMjgwODgtYmEyMmY4MGYtY2U1Yi00NDk2LTk1MmItNWE4ZWNlNTRiNmE4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI0VDIxNTcwN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPThhMzAxNWMwODc2ZTVjNGRhYzBhMTVhMjAzN2MyN2VmZmZjNDkxMjFmMDIwNDJhYTUxYTQ1N2RkNzI0NjFlMzMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ezwj6fo1QK7QwmvrQqf9fjJ7dfNEzYUYP_Xi00Hgr5Q)
Types of changes
What types of changes does your code introduce? Remove all that do not apply:
Checklist
Go over all the following points, and put an
x
in all the boxes that apply.If you are unsure about any of these, don't hesitate to ask. We are here to help!