Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execute checkout step respecting docker setting #6436

Merged
merged 5 commits into from
Jan 20, 2020

Conversation

stsewd
Copy link
Member

@stsewd stsewd commented Dec 4, 2019

Closes #5673

There is a couple of things with this:

  • Git commands using gitpython will use the git binary from the host
  • Git commands using git directly, will use the git binary from the
    container.

This may bring some surprises, since the git versions will be different.
But I'm not really worried about it, it should work.

Another thing is: To make a clone we will spin a build image, which are
big, probably we could add an image that only has git in it.

This should be merged after we have integrated this with the ssh-agent
in .com

Closes readthedocs#5673

There is a couple of things with this:

- Git commands using gitpython will use the git binary from the host
- Git commands using git directly, will use the git binary from the
  container.

This may bring some surprises, since the git versions will be different.
But I'm not really worried about it, it should work.

Another thing is: To make a clone we will spin a build image, which are
big, probably we could add an image that only has git in it.

This should be merged after we have integrated this with the ssh-agent
in .com
@stsewd stsewd added the Status: blocked Issue is blocked on another issue label Dec 4, 2019
@humitos
Copy link
Member

humitos commented Dec 5, 2019

I just tested this and it worked.

With this setup, I think we can spin up just one container and do everything there without the need to have setup_env and build_env, right?

@stsewd
Copy link
Member Author

stsewd commented Dec 5, 2019

With this setup, I think we can spin up just one container and do everything there without the need to have setup_env and build_env, right?

Nope, we still require two env. Because to have only one env, we'll need to know the image to be used, that is known after parsing the config file.

Copy link
Member

@humitos humitos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may bring some surprises, since the git versions will be different.
But I'm not really worried about it, it should work.

I agree. I don't think this will be a real problem unless the git from the container updates to a version that is not backward compatible. Although, we can easily pin the version that we want to use.

To make a clone we will spin a build image, which are big, probably we could add an image that only has git in it.

This is probably a good thing to do. Besides, it's really simple. I just did a test:

# Dockerfile
FROM ubuntu:18.04

RUN apt-get update
RUN apt-get install -y git

This docker image should have all the VCS that we support installed

and then create a class for the setup step:

class DockerSetupEnvironment(DockerBuildEnvironment)

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.container_image = 'readthedocs/git:latest'

I'm 👍 with this solution.

Copy link
Member

@ericholscher ericholscher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a reasonable solution to me. It's a bit hacky, but way less complexity than the other approaches we've tried.

stsewd added a commit to stsewd/readthedocs.org that referenced this pull request Dec 18, 2019
I guess is to get around any security implications.
But modifying the PATH to a local copy of the environments variables of
the host leads to errors.

This is needed for readthedocs#6436
@stsewd stsewd removed the Status: blocked Issue is blocked on another issue label Jan 8, 2020
@stsewd
Copy link
Member Author

stsewd commented Jan 8, 2020

This isn't blocked anymore, ssh is working with docker now.

Copy link
Contributor

@davidfischer davidfischer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this in development without issue! I'm excited to finally have this. While git doesn't frequently have security issues, I'm glad to at least have some mitigation in place for that.

@@ -448,7 +444,12 @@ def run_setup(self, record=True):

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nitpick: the comment above this line (which I can't comment on directly) should be updated.

@stsewd stsewd merged commit 0274ed7 into readthedocs:master Jan 20, 2020
@stsewd stsewd deleted the run-checkout-step-inside-docker branch January 20, 2020 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Execute all checkout steps inside docker
4 participants