Ability to set env vars in project admin

[agjohnson]

We should allow users to inject env vars that can be maintained outside of source code. The use case here would be inputting shared secrets like API access tokens which would then be injected into the build processes. Heroku does something similar.

Eventually, we might auto generate an auth token for our api and inject this in the same way, so that Sphinx can automatically have access to our API as the user. This particular feature will likely be an API v3 feature though.

Phase I

• Add modeling for project environment variables
• Add site admin UI for models
• Add API response that includes environment variables so builder can consume this
• Expose env variables to necessary build commands (all build commands?)

Phase II

• Add model forms for user facing interface
• Add CRUD UI to project admin -- copy from established CRUD UI pattern

[agjohnson]

@humitos what do you think of this feature + pypa/pip#3728 to support private repositories through pip? Maybe there is a way to also address git submodules?

It feels a bit hacky still. A GitHub user looks to be the prescribed path, as much as i hate it.

[humitos]

@humitos what do you think of this feature + pypa/pip#3728 to support private repositories through pip?

I like this because it's not something that we are hacking to support but it comes from upstream and also because it seems pretty easy to do, follow and understand what's happening.

Expose env variables to necessary build commands (all build commands?)

Do you see any potential problem by exposing all the ENV vars to all the commands?

I see as cons that we may end up sending secret data to places that should not be sent. On the other hand, as pros I think this will help users to expand other commands that we don't think to be expandable when implementing this: I see this a way of "allowing things from outside". For example, you can run custom code in conf.py without worries about exposing anything if we provide a ENV secret. Many doors (maybe hacky ones) are open there to the users.

But, if we don't want to expose all the ENV to all the commands we could match the ENV to a particular command:

class EnvVariable(models.Model):
   command = models.ChoiceField(choices=['sphinx-build', 'pip', 'git', ...])
   name = models.CharField()
   value = models.CharField()
   project = models.ForeignKey(Project)

with that model we could inject the variables only where they are needed. If it's a ChoiceField we will end up by duplicating the same variable for multiple commands, so it may needs to be a multiple choice field.

Maybe there is a way to also address git submodules?

This case is a little more complex to me. I found that what some CI do is to modify the submodule URL on the fly with a config withtout touching the user configs using insteadOf (more context from SSH management PR)

[humitos]

Adding this link that contains useful information about how travis handles this: https://docs.travis-ci.com/user/best-practices-security/#Steps-Travis-CI-takes-to-secure-your-data

[agjohnson]

But, if we don't want to expose all the ENV to all the commands we could match the ENV to a particular command:

I'd be -1 on this is think, it's too complex for the user.

[agjohnson]

We can disregard the SSH use case for now. Users will potentially be able to use this for accessing GitHub, but it's multiple levels of bad UX at this point.

[humitos]

Add CRUD UI to project admin -- copy from established CRUD UI pattern

Which pattern do we want to follow here?

1. list and add in the same page, as Domain
2. list in the first page and a different page for add/edit/delete, as Integration

I think it's clearer the one from Integrations.

[agjohnson]

Yeah, integrations is more recent, it's the pattern to follow.

[ericholscher]

This is now done. 🎉