Portus - NGINX error

[ghost]

I have been using Portus for about 6 months now and it has been flawless. I am attempting to redeploy on new infrastructure and Rancher 1.1.2, Docker 1.12. I am using my own "real" certificates.

The SSLProxy container fails to launch with this error.
nginx: [emerg] host not found in upstream "portus" in /etc/nginx/conf.d/portus.conf:9

This is my docker compose file:

sslproxy:
  labels:
    io.rancher.scheduler.affinity:container_label_soft: registry.portus.db=1
    io.rancher.scheduler.affinity:host_label: name=rancherpool-pvt
  tty: true
  image: nginx:1.9.9
  links:
  - portus:portus
  volumes:
  - /efs/data/registry/certs:/etc/nginx/certs:ro
  - /efs/data/registry/proxy:/etc/nginx/conf.d:ro  
  stdin_open: true
lb:
  ports:
  - 5000:5000/tcp
  - 443:443/tcp
  labels:
    io.rancher.scheduler.global: 'false'
    io.rancher.loadbalancer.target.sslproxy: 443=443
    io.rancher.scheduler.affinity:host_label: name=rancherpool-pvt
    io.rancher.loadbalancer.target.registry: 5000=5000
  tty: true
  image: rancher/load-balancer-service
  links:
  - registry:registry
  - sslproxy:sslproxy
  stdin_open: true
registry:
  environment:
    REGISTRY_AUTH: token
    REGISTRY_AUTH_TOKEN_ISSUER: reg.domain.com
    REGISTRY_AUTH_TOKEN_REALM: https://reg.domain.com:443/v2/token
    REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE: /certs/registry.crt
    REGISTRY_AUTH_TOKEN_SERVICE: reg.domain.com:5000
    REGISTRY_HTTP_SECRET: httpsecret
    REGISTRY_HTTP_TLS_CERTIFICATE: /certs/registry.crt
    REGISTRY_HTTP_TLS_KEY: /certs/registry.key
    REGISTRY_LOG_LEVEL: warn
    REGISTRY_NOTIFICATIONS_ENDPOINTS: |-
      - name: portus
        url: http://portus:3000/v2/webhooks/events
        timeout: 500
        threshold: 5
        backoff: 1
    REGISTRY_STORAGE_DELETE_ENABLED: 'true'
  log_driver: ''
  labels:
    io.rancher.scheduler.affinity:host_label: name=rancherpool-pvt
  log_opt: {}
  image: registry:2.3.1
  links:
  - portus:portus
  volumes:
  - /efs/data/registry/certs:/certs:rw
  - /efs/data/registry/data:/var/lib/registry:rw
portus:
  environment:
    PORTUS_CHECK_SSL_USAGE_ENABLED: 'true'
    PORTUS_GRAVATAR_ENABLED: 'true'
    PORTUS_KEY_PATH: /certs/registry.key
    PORTUS_MACHINE_FQDN: reg.domain.com
    PORTUS_PASSWORD: password$$
    PORTUS_PORT: '443'
    PORTUS_PRODUCTION_DATABASE: portus
    PORTUS_PRODUCTION_HOST: aws.database.rds.amazonaws.com
    PORTUS_PRODUCTION_PASSWORD: password$$
    PORTUS_PRODUCTION_USERNAME: username
    PORTUS_SECRET_KEY_BASE: password$$
    PORTUS_SMTP_ENABLED: 'false'
    REGISTRY_HOSTNAME: reg.domain.com
    REGISTRY_NAME: Registry
    REGISTRY_PORT: '5000'
    REGISTRY_SSL_ENABLED: 'true'
  log_driver: ''
  labels:
    registry.portus.app: '1'
    io.rancher.container.pull_image: always
    io.rancher.scheduler.affinity:host_label: name=rancherpool-pvt
  log_opt: {}
  image: sshipway/portus:2.0.5
  volumes:
  - /efs/data/registry/certs:/certs:rw
  - /efs/data/registry/proxy:/etc/nginx/conf.d:rw

[sshipway]

This might be caused by different compose parsing under Docker 1.12? I do not have a Docker 1.12 environment to test this under though. The nginx container definitely has the links: definition to define portus; however maybe the portus container needs to come first in the compose file order?
Can you try reordering the file so that you get portus - registry - sslproxy - lb (as this is the dependency order) and see if things work

[ghost]

I watched the order it was coming up. Portus, Registry both successful.
Then SSLproxy just keeps trying to load.
I added an affinity rule to make sure that Portus and SSLProxy are launched
on the same host, but that did not fix the problem.

This is the contents of my portus.conf file.

     server {
        listen 443 ssl;
        ssl_certificate     certs/registry.crt;
        ssl_certificate_key certs/registry.key;
        location / {
          proxy_set_header Host reg.domain.com;
          proxy_set_header X-Forwarded-Proto https;
          proxy_set_header X-Forwarded-Host reg.panosoft.com:443;
          proxy_pass http://portus:3000/;
          proxy_http_version 1.1;
          proxy_set_header Connection "upgrade";
          proxy_read_timeout 900s;
        }
      }

[ghost]

I don't know what else to look at.

[sshipway]

@gitlab-pano You could open a shell in the haproxy container, and test to see if 'portus' can resolve (use 'ping portus'). If it can't then this would indicate an issue with the linkages passing on into DNS, and this should be reported to Rancher. I don't know if they officially support Docker 1.12, it may be that Rancher needs 1.10 or 1.11

[ghost]

Unfortunately the container does not stay running. It dies. I verified that the version of rancher I am using is compatible with 1.12. I have other stacks that use container links and they work just fine.

[ghost]

Someone on the rancher forum suggested that this line should be added to the the portus.conf file:

resolver 169.254.169.250 valid=5s ipv6=off;

How can I pass that in? The conf file gets re-written every time I launch.

[sshipway]

The portus.conf file (for configuring NGINX) is created by the startup.sh script in the Portus container; this is on a shared mount that is subsequently loaded by the proxy container. If you want to modify this, then you have two options.
First, you can download the source code here, modify startup.sh, and then rebuild the container for your personal use.
Secondly, if the mounted directory is persistent, you could create a second file (eg, portus2.conf) in there to hold any additional global configuration items.

I notice the 'ipv6=off' they are specifying; it is possible that the new versions of docker/rancher are IPv6 aware but don't quite support the DNS or routing somehow and things are getting confused with nginx trying to use IPv6 when its not available?

[ghost]

I need to know what directory it writes to so I can be sure it is mounted. These are my mounts for Portus but I don't see a startup.sh in there.

• /efs/data/registry/certs:/certs:rw
• /efs/data/registry/proxy:/etc/nginx/conf.d:rw

[ghost]

Just to check, I launched some servers running Docker 1.10.3 which is the same as I used in the previous environment. Still the same issue.

[ghost]

I was looking for a docker compose example on your site and could not find
one that is even close to what I am using. Can you point me to an example
docker compose for the current version so that I can compare it to what I
have?

On Wed, Aug 31, 2016 at 2:17 PM, Steve Shipway [email protected]
wrote:

The portus.conf file (for configuring NGINX) is created by the startup.sh
script in the Portus container; this is on a shared mount that is
subsequently loaded by the proxy container. If you want to modify this,
then you have two options.
First, you can download the source code here, modify startup.sh, and then
rebuild the container for your personal use.
Secondly, if the mounted directory is persistent, you could create a
second file (eg, portus2.conf) in there to hold any additional global
configuration items.

I notice the 'ipv6=off' they are specifying; it is possible that the new
versions of docker/rancher are IPv6 aware but don't quite support the DNS
or routing somehow and things are getting confused with nginx trying to use
IPv6 when its not available?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#8 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ARA0TBKyEwUlzIFcWkt0iMWNaSAKqHNFks5qle9fgaJpZM4JwDTw
.

Laurie Kepford
Cloud DevOps Engineer
Panoramic Software
760-712-2266

[sshipway]

I only have the docker-compose and rancher-compose that are in the Rancher Template; this is what we used at the University. https://github.com/rancher/community-catalog/tree/master/templates/registry/2

[ghost]

Lets say I was a total newbie. What instructions would you give me for setting this up? Maybe if I go through the process from scratch I will find the issue.

[ghost]

Today I tried the catalog version you have listed. (Your catalog shows the "no compatible templates" when I tried to connect to it. So i just copied the compose files and added them to my private catalog. It also had the same problem.

So I have eliminated all possibilities that I can think of. Any suggestions? Can anyone else test it on Rancher 1.12?