[Bug] relative config_path resolution is not consistent #2564
Comments
When constructing the config search path, allow callers to directly pass in a non-relative module name starting with `pkg://`. This allows callers to bypass the relative name lookup, and always get consistent behavior regardless of the current environment variables at runtime. This addresses issue facebookresearch#2564.
When constructing the config search path, allow callers to directly pass in a non-relative module name starting with `pkg://`. This allows callers to bypass the relative name lookup, and always get consistent behavior regardless of the current environment variables at runtime. This addresses issue facebookresearch#2564.
When constructing the config search path, allow callers to directly pass in a non-relative module name starting with `pkg://`. This allows callers to bypass the relative name lookup, and always get consistent behavior regardless of the current environment variables at runtime. This addresses issue facebookresearch#2564.
When constructing the config search path, allow callers to directly pass in a non-relative module name starting with `pkg://`. This allows callers to bypass the relative name lookup, and always get consistent behavior regardless of the current environment variables at runtime. This addresses issue #2564. Co-authored-by: Adam Simpkins <[email protected]> Co-authored-by: Jasha <[email protected]>
…esearch#2565) When constructing the config search path, allow callers to directly pass in a non-relative module name starting with `pkg://`. This allows callers to bypass the relative name lookup, and always get consistent behavior regardless of the current environment variables at runtime. This addresses issue facebookresearch#2564. Co-authored-by: Adam Simpkins <[email protected]> Co-authored-by: Jasha <[email protected]>
When constructing the config search path, allow callers to directly pass in a non-relative module name starting with `pkg://`. This allows callers to bypass the relative name lookup, and always get consistent behavior regardless of the current environment variables at runtime. This addresses issue #2564. Co-authored-by: Adam Simpkins <[email protected]> Co-authored-by: Jasha <[email protected]>
|
The automatic detection of the main module is a tricky part of Hydra. Off the top of my head, those environment variables are short-circuiting the detection of the main module and are always treated as absolute modules. Note that those were meant to be used primarily to detect the main module when the app is running from a package built in fbcode (xar, par). Please explain what inconsistency you are seeing in more details. |
🐛 Bug
Description
The config_path parameter to hydra.main() and hydra.experimental.initialize() must be a path name, or a relative module name.
If a relative module name is passed in, hydra does not behave consistently about what this module name is relative to. If any of HYDRA_MAIN_MODULE, FB_PAR_MAIN_MODULE, or FB_XAR_MAIN_MODULE is set, the argument is treated relative to the value of that environment variable. Otherwise hydra tries to determine the module that contains the task function, and the config_path is treated relative to that module.
This results in different behavior based on environment variables. In our code base this has resulted in hundreds of locations where users are manually checking these environment variables before calling hydra.main(), and passing in different config paths based on the value of these environment variables. (or, in a handful of cases, unsetting all of these variables before calling hydra.) It seems better to simply fix hydra to behave consistently rather than having hundreds of call sites working around the hydra behavior.
Note that in our situation this mainly matters for FB_XAR_MAIN_MODULE and FB_PAR_MAIN_MODULE, since these environment variables are set conditionally depending on the current build mode, so hydra behaves differently in different build modes.
To minimize the impact of any code depending on the current behavior, it would probably be easiest to provide some way for callers to pass in a top-level module path, and prevent hydra from treating it as relative. e.g., we could perhaps allow config_path to start with
pkg://to indicate that what follows is a top-level module name.#2395 added support for absolute paths, but there is no mechanism to pass in non-relative modules that I can tell.
Checklist
Expected Behavior
Provide a consistent behavior for config_path that does not depend on build mode.
System information
The text was updated successfully, but these errors were encountered: