Implementing Proxy CA Bundle validation.

[ricardomaraschini]

This patch adds a sidecar command line parameter to the registry operator. This sidecar will be deployed with the operator and will constantly monitor for changes on the mounted config map, restarting the operator every time the certificate content changes.

It is a big PR as it includes needed updates for the following packages:

1. github.com/prometheus/client_golang
2. k8s.io/apiserver
3. k8s.io/kube-openapi
4. sigs.k8s.io/structured-merge-diff

[adambkaplan]

Good start here - there have been a few updates in the design that need to be factored in.

[bparees]

fyi there's a canonical example of doing the "watch this file for changes and then restart the operator" here:
https://github.com/openshift/cluster-kube-controller-manager-operator/pull/254/commits

[ricardomaraschini]

/assign @adambkaplan

[ricardomaraschini]

/retest

[adambkaplan]

@ricardomaraschini two items before this is ready for review/merge:

1. Squash your commits with the exception of the "Update dependencies" commit.
2. Rename the update dependencies commit "bump(*):", with a short note explaining why the deps need to be updated (ex: "pulling in watchdog from library-go"). Keep this commit separate from "real" code changes.

[ricardomaraschini]

@adambkaplan Commits squashed/renamed as requested.

[ricardomaraschini]

As for reference, the deployment of the sidecar will be made as:

      - image: cluster-image-registry-operator
        imagePullPolicy: IfNotPresent
        name: cluster-image-registry-operator-watch
        volumeMounts:
        - mountPath: /path/to/certificates
          name: trusted-ca
        args:
        - --namespace=$(POD_NAMESPACE)
        - --process-name=cluster-image-registry-operator
        - --termination-grace-period=30s
        - --files=/path/to/certificate.pem
        command:
        - cluster-image-registry-operator-watch
        - file-watcher-watchdog
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace

We also need to make sure both containers share the same process namespace by specifying shareProcessNamespace: true on the Deployment. This way watcher can see the operator and kill it when necessary.

[adambkaplan]

A few nits.

I'd also go ahead and add the sidecar container to the image registry operator. This should watch the file /usr/share/pki/ca-trust-source/ca-bundle.pem (or watch the entire directory if that is an option).

EDIT: this can be done in the manifest YAML manifests/07-operator.yaml

[adambkaplan]

/approve

I'm holding this until the key name for the CA trust bundle is finalized. Otherwise looks good!

[ricardomaraschini]

/retest

[adambkaplan]

/hold cancel

We need to land this first. I'll then update #340 so that the operator and the file watcher share the trusted-ca mount.

[adambkaplan]

/lgtm

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adambkaplan, ricardomaraschini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [adambkaplan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[wking]

Upgrade passed in CI here, but this change seems to have broken ART upgrade tests in 4.2. I've hijacked rhbz#1727080 to cover this (the CVO doesn't do a good job bubbling the cause up, so this is almost certainly different from what that Bugzilla was originally about). You may want to revert until you have time to figure out and fix whatever's wrong with the ART nightlies.

[bparees]

It looks like the dockerfile change (which added the symlink) did not get sync'd to distgit. Will need to discuss w/ ART but i'll revert this for now to get us back to green.

[bparees]

@ricardomaraschini @adambkaplan this PR needed to update Dockerfile.rhel7 in addition to Dockerfile, that is why the ART release jobs (which use the .rhel7 file) were broken by this PR even though it passed CI (which, I guess, use the plain Dockerfile).

[danehans]

fyi @ricardomaraschini @adambkaplan I took a slightly different approach for adding trusted ca support to ingress operator: openshift/cluster-ingress-operator#334

/cc @bparees

[bparees]

fyi @ricardomaraschini @adambkaplan I took a slightly different approach for adding trusted ca support to ingress operator: openshift/cluster-ingress-operator#334

@danehans by which you mean you built a filewatcher into the operator process and have the operator terminate itself when the file changes (which will cause the pod to be restarted)? That seems fine to me.

[ricardomaraschini]

@danehans I like it!