Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Use @hydra.main() decorator for multiple python scripts from one bash script #2949

Open
koch-kil opened this issue Sep 10, 2024 · 0 comments
Labels
enhancement Enhanvement request

Comments

@koch-kil
Copy link

🚀 Feature Request

Option to run subsequent python scripts through a bash script - each using the hydra configuration.

Motivation

Is your feature request related to a problem? Please describe.
We are using a bash script to run an ML inference pipeline. It consists of multiple python files (preparation & processing) and a docker command (ML inference).
When passing the hydra kwargs to the python scripts, however, we get a ValueError
ValueError: GlobalHydra is already initialized, call GlobalHydra.instance().clear() if you want to re-initialize
Even though we clear the instance before the call of the main function using these lines:

    if hydra.core.global_hydra.GlobalHydra.instance().is_initialized():
        hydra.core.global_hydra.GlobalHydra.instance().clear()
    main()

This seems not to solve the problem.
All my python scripts only use the @hydra.main() decorator on the main function which is only called using the if name=="main" entry point. Also, the python scripts are called like this:

python3 script.py --config-dir $configs_path $hydra_kwargs

Pitch

Describe the solution you'd like
I want to have an option to pass the arguments to each of the python scripts without creating an error - or an option to flag the @hydra.main() decorator to ignore the global hydra instance.
If there's a workaround, please let me know.

@koch-kil koch-kil added the enhancement Enhanvement request label Sep 10, 2024
@koch-kil koch-kil changed the title [Feature Request] [Feature Request] Use @hydra.main() decorator for multiple python scripts from one bash script Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhanvement request
Projects
None yet
Development

No branches or pull requests

1 participant