Start inject_hyperparams step count at 0. #416
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Start inject_hyperparams step count at 0.
This fixes issue #415. Previously,
inject_hyperparams
started withstep_count=1
in the first update when using schedules (it incremented beforepassing it to the schedule) whereas
scale_by_schedule
started withstep_count=0
. To make this consistent, this PR changesinject_hyperparams
to also start at 0, i.e. increment the count only after passing it to the
schedule.
The PR comes with a test that breaks without the change. Furthermore, the step
counts in the existing tests of
inject_hyperparams
had to be decremented byone in order for the tests to pass.