Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SAC-discrete implementation #270

Merged
merged 42 commits into from
Jan 13, 2023
Merged

SAC-discrete implementation #270

merged 42 commits into from
Jan 13, 2023

Commits on Aug 29, 2022

  1. add draft of SAC discrete implementation

    @timoklein
    timoklein committed Aug 29, 2022
    Configuration menu
    Copy the full SHA
    18b643b View commit details
    Browse the repository at this point in the history

  2. run pre-commit

    @timoklein
    timoklein committed Aug 29, 2022
    Configuration menu
    Copy the full SHA
    c3c98bd View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2022

  1. Use log softmax instead of author's log-pi code

    @timoklein
    timoklein committed Aug 31, 2022
    Configuration menu
    Copy the full SHA
    ec31dc4 View commit details
    Browse the repository at this point in the history

  2. Revert to cleanrl SAC delay implementation (it's more stable)

    @timoklein
    timoklein committed Aug 31, 2022
    Configuration menu
    Copy the full SHA
    deb37e8 View commit details
    Browse the repository at this point in the history

  3. Remove docstrings and duplicate code

    @timoklein
    timoklein committed Aug 31, 2022
    Configuration menu
    Copy the full SHA
    a1fdd2b View commit details
    Browse the repository at this point in the history

  4. Use correct clipreward wrapper

    @timoklein
    timoklein committed Aug 31, 2022
    Configuration menu
    Copy the full SHA
    977a83a View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2022

  1. fix bug in log softmax calculation

    @timoklein
    timoklein committed Sep 6, 2022
    Configuration menu
    Copy the full SHA
    f2ea3e6 View commit details
    Browse the repository at this point in the history

  2. adhere to cleanrl log_prob naming

    @timoklein
    timoklein committed Sep 6, 2022
    Configuration menu
    Copy the full SHA
    48af04c View commit details
    Browse the repository at this point in the history

  3. fix bug in entropy target calculation

    @timoklein
    timoklein committed Sep 6, 2022
    Configuration menu
    Copy the full SHA
    b2a09a0 View commit details
    Browse the repository at this point in the history

  4. change layer initialization to match existing cleanrl codebase

    @timoklein
    timoklein committed Sep 6, 2022
    Configuration menu
    Copy the full SHA
    89680c7 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2022

  1. working minimal diff version

    @timoklein
    timoklein committed Sep 19, 2022
    Configuration menu
    Copy the full SHA
    b1d7d44 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2022

  1. implement original learning update frequency

    @timoklein
    timoklein committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    61e1c74 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2022

  1. parameterize the entropy scale for autotuning

    @timoklein
    timoklein committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    7cd1e3a View commit details
    Browse the repository at this point in the history

  2. add benchmarking script

    @timoklein
    timoklein committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    61c46fc View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2022

  1. rename target entropy factor and set new default value

    @timoklein
    timoklein committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    4915e4c View commit details
    Browse the repository at this point in the history

Commits on Nov 5, 2022

  1. add docs draft

    @timoklein
    timoklein committed Nov 5, 2022
    Configuration menu
    Copy the full SHA
    6f7251f View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2022

  1. fix SAC-discrete links to work pre merge

    @timoklein
    timoklein committed Nov 10, 2022
    Configuration menu
    Copy the full SHA
    23b60ff View commit details
    Browse the repository at this point in the history

  2. add preliminary result table for SAC-discrete

    @timoklein
    timoklein committed Nov 10, 2022
    Configuration menu
    Copy the full SHA
    10ee9f0 View commit details
    Browse the repository at this point in the history

  3. clean up todos and add header

    @timoklein
    timoklein committed Nov 10, 2022
    Configuration menu
    Copy the full SHA
    8430fd8 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2022

  1. minimize diff between sac_atari and sac_continuous

    @timoklein
    timoklein committed Nov 11, 2022
    Configuration menu
    Copy the full SHA
    a17768c View commit details
    Browse the repository at this point in the history

  2. add sac-discrete end2end test

    @timoklein
    timoklein committed Nov 11, 2022
    Configuration menu
    Copy the full SHA
    d6a507c View commit details
    Browse the repository at this point in the history

  3. SAC-discrete docs rework

    @timoklein
    timoklein committed Nov 11, 2022
    Configuration menu
    Copy the full SHA
    a7ea6f4 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2022

  1. Update SAC-discrete @100k results

    @timoklein
    timoklein committed Nov 12, 2022
    Configuration menu
    Copy the full SHA
    9f6493c View commit details
    Browse the repository at this point in the history

  2. Fix doc links and unify naming in code

    @timoklein
    timoklein committed Nov 12, 2022
    Configuration menu
    Copy the full SHA
    59a6d00 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2022

  1. update docs

    @vwxyzjn
    vwxyzjn committed Nov 13, 2022
    Configuration menu
    Copy the full SHA
    1304b7a View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2022

  1. fix target update frequency (see PR vwxyzjn#323)

    @timoklein
    timoklein committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    3a3f41b View commit details
    Browse the repository at this point in the history

  2. clarify comment regarding CNN encoder sharing

    @timoklein
    timoklein committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    80187ad View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2022

  1. Merge remote-tracking branch 'upstream/master' into sac-discrete

    @timoklein
    timoklein committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    e9cb494 View commit details
    Browse the repository at this point in the history

  2. fix benchmark installation

    @timoklein
    timoklein committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    e199e39 View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2022

  1. fix eps in minimal diff version and improve code readability

    @timoklein
    timoklein committed Dec 3, 2022
    Configuration menu
    Copy the full SHA
    bb27fa1 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2022

  1. add docs for eps and finalize code

    @timoklein
    timoklein committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    6a46632 View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2022

  1. use no_grad for actor Q-vals and re-use action-probs & log-probs in a… 

    …lpha loss
    @timoklein
    timoklein committed Dec 7, 2022
    Configuration menu
    Copy the full SHA
    cad5fff View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2022

  1. update docs for new code and settings

    @timoklein
    timoklein committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    0cf47f1 View commit details
    Browse the repository at this point in the history

  2. fix links to point to main branch

    @timoklein
    timoklein committed Dec 14, 2022
    Configuration menu
    Copy the full SHA
    61988c4 View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2022

  1. update sac-discrete training plots

    @timoklein
    timoklein committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    6e17005 View commit details
    Browse the repository at this point in the history

  2. new sac-d training plots

    @timoklein
    timoklein committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    33b00f3 View commit details
    Browse the repository at this point in the history

  3. update results table and fix link

    @timoklein
    timoklein committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    5dabafb View commit details
    Browse the repository at this point in the history

  4. fix pong chart title

    @timoklein
    timoklein committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    90b2fd5 View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2023

  1. add Jimmy Ba name as exception to code spell check

    @timoklein
    timoklein committed Jan 13, 2023
    Configuration menu
    Copy the full SHA
    a763994 View commit details
    Browse the repository at this point in the history

  2. change target_entropy_scale default value to same value as experiments

    @timoklein
    timoklein committed Jan 13, 2023
    Configuration menu
    Copy the full SHA
    071cdbb View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/master' into sac-discrete

    @timoklein
    timoklein committed Jan 13, 2023
    Configuration menu
    Copy the full SHA
    dcc2633 View commit details
    Browse the repository at this point in the history

  4. remove blank line at end of pre-commit

    @timoklein
    timoklein committed Jan 13, 2023
    Configuration menu
    Copy the full SHA
    c671a92 View commit details
    Browse the repository at this point in the history