Allow setting environment variable automatically for jobs

[omry]

This is low priority at the moment, please vote up if you will find it useful.

Control of env variables for launched jobs.
This could be based on a special node in the job config:

env:
  OMP_NUM_THREADS: 1 
  # copy env from running machine
  USER: ${env:USER} 

TO BE CLEAR:
You can already use environment variables in configs:
See the OmegaConf documentation.

This issue is about allowing easier control of the environment variable of Hydra apps for other purposes (for example, setting OMP_NUM_THREADS automatically to 1).

[sackh]

Will this also allow overriding config from environment variable? similar to overriding from the command line

[omry]

This is already possible (although not similar to overriding from the command line).
See the OmegaConf documentation.

[sackh]

I think it will be a good idea to make overriding config from the environment variable consistent with overriding using the command line approach.

[omry]

How can you make two things that are completely different consistent with each other?
If you want me to take this suggestion seriously, make it more concrete. I have no idea what kind of functionality you are thinking about.

regardless, please open a new issue against OmegaConf if you have a feature request. this is not the right place to discuss any such requests.

[deeplook]

OMG, this would be a super important feature. It's hard to believe it doesn't exist already...

[fcennecf]

Hi, I'm sorry. it's an important enhancement. if we looked at containers, the main workflow would be using environment variables.
It would be great to have this enhancement soon.

[omry]

1. This is a feature request to support automatically SETTING the environment variables for Hydra jobs.
2. You can already USE environment variables in the config (see my edit to the task description).
3. I am planning to add this functionality for the next Hydra version.
4. This is marked as "good first issue", meaning it's not hard to implement.

If this is so important to people, they can step up and help.

[fcennecf]

the environment variables for Hydra jobs.

Oh, miss that.

You can already USE environment variables in the config

Thank you

[omry]

It was not clear from the issue description, I updated it.
I recommend everyone here spend the time to read the OmegaConf documentation. there are many good features there you should know about if you use Hydra.

[omry]

See this for details.

[rr-hari-prakash]

See this for details.

page doesn't exist. Can we set ENV variables in hydra?

[Jasha10]

@rr-hari-prakash I've edited the link; it works now. Use the hydra.job.env_set variable.