Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Final -- Improve handling of nested dependencies #181

Merged
merged 16 commits into from
Dec 13, 2021
Merged

Conversation

ncilfone
Copy link
Contributor

What does this PR do?

This PR uses simple graph methods to better handle nested dependencies. Also contains a fairly big refactor of some original spaghetti backend code into a much cleaner interface. Mainly contributed by @gbmarc1 via fork. This should close #152.

Gives attribution to @gbmarc1

Checklist

  • Did you adhere to PEP-8 standards?
  • Did you run black and isort prior to submitting your PR?
  • Does your PR pass all existing unit tests?
  • Did you add associated unit tests for any additional functionality?
  • Did you provide documentation (Numpy Docstring format) whenever possible, even for simple functions or classes?

ncilfone and others added 16 commits October 28, 2021 13:34
…ssuming to fall back on defaults instead of recursing the config space to set the correct parameters.
…ng recursed to set config arg values within nested classes.
…ctor should be thought about to handle all the parent/child relations in a cleaner manner
…. Probably a bunch of vestigial code still to remove -- base tests still only 43/48
* fixing some tests

* some interesting work but breaking the tests

* test_command_line pass

* some cleaning and refactoring

* fix when optional nested value is None

* clearner refactor to handle type optionals

* "config" cannot be a general argument

* fix some more tests and windows path compatible

* s3 tests pass on windows

* all tests now are passing

* dont need the graph in builde space

* renamed everything to builder_space
and doc string

* some cleaning

Co-authored-by: mbelanger_explorance <[email protected]>
* Nested values edge case fix (#160)

* fixes issues wrt more than 2 levels of class nesting references. was ssuming to fall back on defaults instead of recursing the config space to set the correct parameters.

* linted

* updated docs for cmd line overrides on nested classes

* fixed edge case from #152 where the default attrs object wasn't getting recursed to set config arg values within nested classes.

* adding test conf file

* fixing some tests

* some interesting work but breaking the tests

* test_command_line pass

* some cleaning and refactoring

* fix when optional nested value is None

* clearner refactor to handle type optionals

* "config" cannot be a general argument

* fix some more tests and windows path compatible

* s3 tests pass on windows

* all tests now are passing

* dont need the graph in builde space

* renamed everything to builder_space
and doc string

* some cleaning

* removing networkx dep

* linted

* removed dataclasses dep by swapping to a namedtuple

* fix-up of some tests. some were missing correct spock classes as *args. Some now raise a different exception with the new refactor

* moved help functionality to separate file for readability of the builder class

* documentation. almost finished

* finished doc strings. linted

* files for extra tests

* fixing issues with python3.6 which doesn't have the typing alias 'list' yet only 'typing.List'

* linted

Co-authored-by: mbelanger_explorance <[email protected]>
@ncilfone
Copy link
Contributor Author

@gbmarc1 Sorry for all the mentions. This should resolve attribution. Figured it was better to make sure your code contributions were in the blame like it should be.

Sorry again for the screw up on my end!

@coveralls
Copy link

Pull Request Test Coverage Report for Build 1574822357

  • 421 of 437 (96.34%) changed or added relevant lines in 12 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage increased (+2.3%) to 95.295%

Changes Missing Coverage Covered Lines Changed/Added Lines %
spock/backend/spaces.py 23 24 95.83%
spock/args.py 52 56 92.86%
spock/graph.py 58 62 93.55%
spock/backend/field_handlers.py 133 140 95.0%
Files with Coverage Reduction New Missed Lines %
spock/builder.py 1 94.25%
Totals Coverage Status
Change from base Build 1574762822: 2.3%
Covered Lines: 1600
Relevant Lines: 1679

💛 - Coveralls

@ncilfone ncilfone merged commit ba4330c into master Dec 13, 2021
@ncilfone ncilfone deleted the dag_nesting branch January 4, 2022 15:35
@ncilfone ncilfone mentioned this pull request Jan 10, 2022
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Nested config values are not updated with provided value in --config file
3 participants