-
Notifications
You must be signed in to change notification settings - Fork 285
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Skip existing for advantage modules #1070
Conversation
What about the other way around? by default not skipping. In that case we should make sure in the advantage module the value is computed without gradients. |
Could you explain a bit more why your example constitutes a problem? Also I wonder if this is a small step in the direction of #1000. PS in #1000 advantages would not even have access to neural networks. Only losses will. and thus it will make all this even simpler |
you can perfectly not pass a network and it will work fine.
instead of
|
Yep i didn't see that we could pass None, that works perfectly. Thinking about this more, if losses start skipping existing keys by default, this may have some bad consequences if collected data (from collectors or buffers) has some of the loss keys of interest (cause they might be old). But thinking of #1000 we definitely need a way to precomupte keys for losses (values and targets) and make losses use those precomputed keys. I think it would be fine to use skip_existing by default also in the losses and if users feed the losses data with old values it is their problem.
Alternatively, what @albertbou92 says also makes sense, the losses could recompute everything by default and we would need to
I am leaning more towards the first option tho |
yes this is why I set it to False, and why I was asking you guys what you were thinking. I guess we'll leave it as it is.
if you don't pass the value net to the module an error will be raised bc the entry is missing.
yep that's the idea.
or
|
I am more for the first of the 2 since you are not bound to smth at init time. Regarding the default of skip_existing i am leaning more towards True but I am fine with anything |
then use that one 😉
Let's put it that way: |
ahah i thought we were choosing alernatively but both is better ahaha
ok im sold |
I like the idea of having something that works out of the box, even if there is margin for improvement in terms of performance. Beginner users will be able to train and more advanced users will be able to adjust to code for better performance. |
Allows to ask the advantage modules to skip the value network if its output is already present.
@matteobettini @albertbou92
Curious to hear your thoughts about this:
Should all modules skip existing keys by defaults? This could have some unexpected consequences if someone does this