Add support for docker proxy #69

otahi · 2014-05-25T07:49:37Z

Configure docker to use proxies if it is installed.

Docker daemon needs http_proxy environment when behind proxies in a docker config file as follows:

RHEL
- /etc/sysconfig/docker
Ubuntu
- /etc/default/docker ( < Ubuntu 14.04)
- /etc/default/docker.io ( >= Ubuntu 14.04 )
boot2docker
- /var/lib/boot2docker/profile

I could not find any official documents docker for proxy.
See following references.
docker issues 1852
Stack Overflow: Cannot download Docker images behind a proxy

dduportal · 2014-05-26T15:37:06Z

Hi !

Good works, i'm going to test while you roasted me at implementing this.

I'm currently using boot2docker, and it seems that, when you have vagrant-proxyconf which configure an /etc/profile.d/proxy.sh script, we just have to restart docker service, beacuse it'll use profile script.

So Docker support will mostly depends on which Os and which service management is used (init, systemd, etc.)

What do you think around that ?

otahi · 2014-05-26T22:18:49Z

Thanks for your comment!

I found /usr/local/etc/init.d/docker for boot2docker. It uses /var/lib/boot2docker/profile so we can add configure to that file.

I will add this to my pull request, so I changed this pull request to 'Work In Progress'.

tmatilai · 2014-05-27T08:42:43Z

Thanks a lot! Unfortunately I haven't had time to go through this yet, but hopefully have on Thursday.

Anyway, the different behavior of guest OSs should be implemented through guest capabilities. This is a design mistake from my part early on. I haven't got into refactoring it on all action classes, but here we could move the logic to capability classes anyway.

otahi · 2014-05-27T13:40:39Z

@tmatilai Thank you for your advice. If you give me a direction, i will try to move the logic. Can you give me hints or something?

otahi · 2014-05-27T22:04:27Z

@dduportal I have added docker proxy config on boot2docker. Please check it.

I think there are some kinds of implementations around booting of docker, we need to care all individuals..

otahi · 2014-05-30T21:08:45Z

I fixed a problem for false values same as #54.

tmatilai · 2014-06-04T20:27:31Z

@otahi thanks again for all the work. I've been buried with other things, but hope to recover a bit in a week. If this seems to work for people I guess I could merge this then and we can think about refactoring later.

otahi · 2014-06-04T21:22:28Z

@tmatilai Thank you for your comment. I agree with your idea.
On my environment, it seems working OK.
Refactoring later is nice to me, because it is hard to find problems without lots of docker users.

BugRoger · 2014-06-12T15:20:02Z

+1 for this patch.

I tested the plugin and it works nicely with our corporate http proxy. The only issue I found is that the docker service needs to be restarted manually to pick up the new configuration.

otahi · 2014-06-12T22:26:23Z

@BugRoger Thank you for testing.
I understand the issue. I will fix it with restart docker after configuration.

otahi · 2014-06-13T23:03:38Z

@BugRoger @tmatilai I have added docker daemon restarting.
Please check it.

I have checked this change with the following guests, and it works.

Ubuntu 14.04
Centos 6.5
boot2docker 0.8.0

It works even if it is a first boot.

[vagrant@localhost ~]$ sudo strings /proc/`pgrep docker`/environ | grep HTTP_PROXY
HTTP_PROXY=http://192.168.0.2:3128/
[vagrant@localhost ~]$

johnbellone · 2014-06-15T14:36:14Z

@otahi Does docker use HTTP_PROXY instead of http_proxy? Most UNIX utilities tend to favor the latter.

dduportal · 2014-06-15T19:15:34Z

I'll take a look behind my company proxy this week (under Cloudstack, and on Windows 7 x64).

dduportal · 2014-06-16T06:37:37Z

Hi, excuse the ruby's noob question but : how to compile and run the plugin ?

I installed a basic container with ruby 2.0.0/gem from RVM :

FROM debian:wheezy
MAINTAINER [email protected]

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get install -y wget curl vim procps dnsutils telnet htop

# Install ruby from RVM
RUN curl -sSL https://get.rvm.io | bash -s stable --ruby=2.0.0

RUN apt-get install -y git

RUN /usr/local/bin/gem install bundler

# Configure
ENV PATH /usr/local/bin:$PATH

Where i cloned the two repositories (tmatilai's one and otahi's one) into.

Then i ran a bundle install into the repos, but what to do then ? I'm stuck, with no dev docs to begin with.

BugRoger · 2014-06-16T08:10:59Z

You need to install the plugin in Vagrant. I assume you don't have your Vagrant inside of a docker container. This would do it on the vagrant host:

git clone https://github.com/otahi/vagrant-proxyconf
cd vagrant-proxyconf
git checkout docker
bundle install
rake build
vagrant plugin install pkg/vagrant-proxyconf-1.3.2.dev.gem

dduportal · 2014-06-16T08:23:03Z

Well OK, i missed the rake release (didn't know where this tool was user in the build toolchain).

Thanks for the response ! I'll give a shoot these days.

(In fact, i use docker for sharing and reusing the compilation env => From git clone to rake release, all of that will be run into a container, ii's protable and versionnable. Then, i'll copy the generated artefacts to a physical host running vagrant, virtualbox, docker, etc... If i want to be tied to docker only, i can also test the plugin inside my container, with things like https://github.com/jpetazzo/dind, vagrant and only the docker provider :-) )

BugRoger · 2014-06-16T08:40:59Z

@johnbellone Docker just uses the http transport from Go's standard library. And that one doesn't care about the case of the env variables

http://golang.org/src/pkg/net/http/transport.go#L102
https://github.com/dotcloud/docker/blob/master/registry/registry.go#L50

tmatilai · 2014-06-16T08:41:19Z

Even though rake release also builds the gem, it's purpose is to release the version to rubygems.org. To only build the gem, execute bundle exec rake build.

But you don't even need to use rake and system ruby, as you can just use Vagrant's embedded ruby environment. The instructions can be found in the wiki.

dduportal · 2014-06-16T08:57:26Z

@tmatilai thanks a lot, i missed there was a wiki, shame on me !

dduportal · 2014-06-16T20:35:23Z

FROM debian:wheezy
MAINTAINER [email protected]

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get install -y wget curl vim procps dnsutils telnet htop

# Install ruby tools
RUN curl -sSL https://get.rvm.io | bash -s stable --ruby=2.0.0
RUN apt-get install -y git 
ENV PATH /usr/local/rvm/rubies/ruby-2.0.0-p481/bin/:$PATH
RUN gem install bundler
RUN mkdir -p /app


# Fetch and compile the project
WORKDIR /app
RUN git clone https://github.com/otahi/vagrant-proxyconf

WORKDIR /app/vagrant-proxyconf
RUN git checkout docker
RUN bundle install
RUN rake build

CMD cat $(find /app/vagrant-proxyconf/pkg -type f -name "*.gem")

Under Mac OS 10.9.3 with boot2docker 1.0.0
=>

docker build -t proxyconf-test .
docker run --rm proxyconf > vagrant-proxyconf.dev.gem
vagrant plugin install vagrant-proxyconf.dev.gem

First test with a local proxy (basic polipo image) works fine.

I'll try under Windows Wednesday.

Anyway, good job !

otahi · 2014-06-16T22:05:18Z

@johnbellone @BugRoger Thank you for your check and giving a reference.
I have confirmed http_proxy has same result.

[vagrant@localhost ~]$ sudo strings /proc/`pgrep docker`/environ | grep http_proxy
http_proxy=http://192.168.0.2:3128/
[vagrant@localhost ~]$

Anyway, I have added unnecessary environments like ftp_proxy and so on. Go library uses only http_proxy and no_proxy, so I think I should delete unnecessary environments. What do you think? > @tmatilai

If it is better to delete, I will modify like this.

        def docker_sed_script
          <<-SED.gsub(/^\s+/, '')
            /^export HTTP_PROXY=/ d
            /^export HTTPS_PROXY=/ d
            /^export FTP_PROXY=/ d
            /^export NO_PROXY=/ d
            /^export http_proxy=/ d
            /^export https_proxy=/ d
            /^export ftp_proxy=/ d
            /^export no_proxy=/ d
          SED
        end

⬇️

        def docker_sed_script
          <<-SED.gsub(/^\s+/, '')
            /^export HTTP_PROXY=/ d
            /^export NO_PROXY=/ d
            /^export http_proxy=/ d
            /^export no_proxy=/ d
          SED
        end

otahi · 2014-06-16T22:06:21Z

@dduportal Thank you for testing! I want to hear good news from you! 😄

otahi · 2014-06-17T20:53:42Z

I have deleted unused environment variables at d8755f4, and it still works on my environment.

dduportal · 2014-06-18T10:57:57Z

@otahi @tmatilai All working for me : company corporate proxy (dedicated physical appliance), serving only HTTP and HTTPS (FTP thru HTTP supported but no SSH over HTTPS).

I installed the dev gem :

vagrant plugin list
vagrant-cachier (0.7.2)
vagrant-librarian-chef (0.1.5)
vagrant-login (1.0.1, system)
vagrant-proxyconf (1.3.2.dev)
  - Version Constraint: 1.3.2.dev
vagrant-share (1.0.1, system)

Adn then ran my custom boot2docker :

vagrant init dduportal/boot2docker
vagrant up
vagrant ssh
$ docker search nginx => OK
$ docker pull debian:wheezy => OK

My config was a Windows 7 x64, with vagrant 1.6.2

It's all OK for me !

otahi · 2014-06-18T13:29:57Z

@dduportal Thank you for your time! That's nice!

@tmatilai Can we merge this change?

tmatilai · 2014-06-18T16:05:57Z

@otahi sure! I'll go through it once more and merge. But can't promise if I manage to do it today.

Add support for docker proxy

tmatilai · 2014-06-22T07:59:57Z

Merged! Thanks to everyone involved!

otahi · 2014-06-22T11:55:03Z

Thank you @tmatilai and everyone!!

dduportal · 2014-09-28T16:44:49Z

👍

adds docker proxy config

0e6716c

otahi changed the title ~~Add support for docker proxy~~ [WIP] Add support for docker proxy May 26, 2014

adds docker proxy config on boot2docker

fc31102

otahi changed the title ~~[WIP] Add support for docker proxy~~ Add support for docker proxy May 27, 2014

Fix docker config for empty end false values

69909d4

adds docker daemon restarting to reflect proxy configures

421c477

deletes unused environment variables by go library

d8755f4

tmatilai added a commit that referenced this pull request Jun 22, 2014

Merge pull request #69 from otahi/docker

2e0bbaf

Add support for docker proxy

tmatilai merged commit 2e0bbaf into tmatilai:master Jun 22, 2014

otahi deleted the docker branch June 22, 2014 12:30

jperville mentioned this pull request Oct 29, 2014

Explicitly disable a specific proxy, eg docker. #85

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for docker proxy #69

Add support for docker proxy #69

otahi commented May 25, 2014

dduportal commented May 26, 2014

otahi commented May 26, 2014

tmatilai commented May 27, 2014

otahi commented May 27, 2014

otahi commented May 27, 2014

otahi commented May 30, 2014

tmatilai commented Jun 4, 2014

otahi commented Jun 4, 2014

BugRoger commented Jun 12, 2014

otahi commented Jun 12, 2014

otahi commented Jun 13, 2014

johnbellone commented Jun 15, 2014

dduportal commented Jun 15, 2014

dduportal commented Jun 16, 2014

BugRoger commented Jun 16, 2014

dduportal commented Jun 16, 2014

BugRoger commented Jun 16, 2014

tmatilai commented Jun 16, 2014

dduportal commented Jun 16, 2014

dduportal commented Jun 16, 2014

otahi commented Jun 16, 2014

otahi commented Jun 16, 2014

otahi commented Jun 17, 2014

dduportal commented Jun 18, 2014

otahi commented Jun 18, 2014

tmatilai commented Jun 18, 2014

tmatilai commented Jun 22, 2014

otahi commented Jun 22, 2014

dduportal commented Sep 28, 2014

Add support for docker proxy #69

Add support for docker proxy #69

Conversation

otahi commented May 25, 2014

dduportal commented May 26, 2014

otahi commented May 26, 2014

tmatilai commented May 27, 2014

otahi commented May 27, 2014

otahi commented May 27, 2014

otahi commented May 30, 2014

tmatilai commented Jun 4, 2014

otahi commented Jun 4, 2014

BugRoger commented Jun 12, 2014

otahi commented Jun 12, 2014

otahi commented Jun 13, 2014

johnbellone commented Jun 15, 2014

dduportal commented Jun 15, 2014

dduportal commented Jun 16, 2014

BugRoger commented Jun 16, 2014

dduportal commented Jun 16, 2014

BugRoger commented Jun 16, 2014

tmatilai commented Jun 16, 2014

dduportal commented Jun 16, 2014

dduportal commented Jun 16, 2014

otahi commented Jun 16, 2014

otahi commented Jun 16, 2014

otahi commented Jun 17, 2014

dduportal commented Jun 18, 2014

otahi commented Jun 18, 2014

tmatilai commented Jun 18, 2014

tmatilai commented Jun 22, 2014

otahi commented Jun 22, 2014

dduportal commented Sep 28, 2014