Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metaflow Configs #1962

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open

Metaflow Configs #1962

wants to merge 15 commits into from

Commits on Sep 10, 2024

  1. Change top-level flow decorator options to be prefixed by METAFLOW_FLOW_

    Previously, options like `branch` and `name` (injected by the project
    decorator for example) could be set using `METAFLOW_BRANCH`. They now
    need to be set using `METAFLOW_FLOW_BRANCH`.
    
    This change is made to prevent clashes between regular metaflow
    configuration settings and decorator level options.
    
    No other changes are made so `METAFLOW_RUN_MAX_WORKERS` still works
    as expected and `METAFLOW_PYLINT` as well.
    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    931d310 View commit details
    Browse the repository at this point in the history
  2. Initial Config object

    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    688b561 View commit details
    Browse the repository at this point in the history
  3. More WIP

    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    1b1c59b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    4f3cb39 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    e285785 View commit details
    Browse the repository at this point in the history
  6. Multiple fix plus sample flow

    Several fixes:
      - fixed an issue with default values
      - better handling of parameter defaults as configs
      - handle config defaults as functions
      - ConfigValue is more "dict"-like
      - made <myflow>.configs and <myflow>.steps work properly
      - renamed resolve_configs to init
    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    8b6214a View commit details
    Browse the repository at this point in the history
  7. Addressed comments. Added more documentation/explanation

    Specifically:
      - moved things out of the INFO file
      - added to_dict
      - renamed user_configs to config_parameters
    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    5f354b1 View commit details
    Browse the repository at this point in the history
  8. Added test, more cleanup

    Specifically:
      - made config values immutable
      - cleaned up state stored in FlowSpec
      - added a test exercising configs in various places
    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    bcdd987 View commit details
    Browse the repository at this point in the history
  9. Fixup conda decorator

    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    d00fe98 View commit details
    Browse the repository at this point in the history
  10. Whoops -- forgot one

    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    9592600 View commit details
    Browse the repository at this point in the history
  11. Fix the dreadful core tests and be a LOT less verbose

    Will only print stuff when there is an error and will print it in a non-interleaved manner.
    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    f6fdc69 View commit details
    Browse the repository at this point in the history
  12. Fix parallel tests

    romain-intel committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    6370ff2 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    39b31de View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    dad4707 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. Configuration menu
    Copy the full SHA
    f172725 View commit details
    Browse the repository at this point in the history