Support TEST_KUBECONFIG for integration tests

[smarterclayton]

Uses the provided administrator kubeconfig and runs the tests against
that master. This allows the integration tests to be run against a stock
"openshift start master".

Another change in the master will make local dev cases easier so allowed
registries doesn't have to be customized.

@bparees @legionus

[smarterclayton]

Here's what the runner will look like:

    # Once admin.kubeconfig exists, executes shared tests
    - name: test
      image: ${LOCAL_IMAGE_TEST_BIN}
      volumeMounts:
      - name: shared-tmp
        mountPath: /tmp/shared
      - name: artifacts
        mountPath: /tmp/artifacts
      command:
      - /bin/bash
      - -c
      - |
        #!/bin/bash
        set -euo pipefail

        trap 'touch /tmp/shared/exit' EXIT
        trap 'kill $(jobs -p); exit 0' TERM

        # wait until the master job creates admin.kubeconfig
        while true; do
          if [[ ! -f /tmp/shared/admin.kubeconfig ]]; then
            sleep 5 & wait
            continue
          fi
          break
        done
        echo "Found kubeconfig"
        export KUBECONFIG=/tmp/shared/admin.kubeconfig
        
        ${COMMAND}

    # Start a standalone master
    - name: control-plane
      image: ${IMAGE_CONTROL_PLANE}
      volumeMounts:
      - name: shared-tmp
        mountPath: /tmp
      - name: artifacts
        mountPath: /tmp/artifacts
      env:
      - name: TYPE
        value: ${CLUSTER_TYPE}
      command:
      - /bin/bash
      - -c
      - |
        #!/bin/bash
        set -euo pipefail
        function teardown() {
          set +e
          echo "Gathering artifacts ..."
        }
        trap 'teardown' EXIT
        trap 'kill $(jobs -p); exit 0' TERM

        ( 
          while true; do
            if [[ -f /tmp/exit ]]; then
              echo "Shutting down"
              kill $$
            fi
            sleep 5 & wait
          done
        ) &

        mkdir /tmp/cluster && cd /tmp/cluster
        openshift start master --write-config=/tmp
        openshift start master --config=/tmp/master-config.yaml

[smarterclayton]

That runner would be shared between test-cmd and this integration test and anyone else who just needs a stock origin master. When we split origin master up, we'll update the shared runner so you don't have to worry about it.

[smarterclayton]

Linked the change to openshift to stop defaulting whitelisted registries, to be consistent with install.

[smarterclayton]

I linked in the integration job - if we're ok with these changes, then I'll test those and iterate, but integration should take ~10s with master reuse vs 1m or so.

[bparees]

so this is just establishing an abstraction where you can either run a master in a container (old behavior) or point the client to an existing cluster via an admin kubeconfig (new behavior), right?

looks reasonable to me as long as the old behavior wasn't customizing the master it started in any meaningful way.

[dmage]

Apparently we are not willing to add any integration tests for image pruning and for anything that touches the image collection.

Discussions about #78 were ended this the same conclusions. We need to run tests on a running cluster and make our best to avoid touching global resources. In this case for pruning we'll write only unit tests with a mocked client.

I don't know what we will do with signatures, perhaps we'll keep them without tests. Nobody knows how to use it.

I'll drop the old behaviour as a follow up as it'll be broken when openshift start will be gone.

So, LGTM (it have not tested yet with the new job, right?)

[smarterclayton]

Once this is merged I'll start testing the new job against my extra PR.

[dmage]

/lgtm

[smarterclayton]

Testing the job now

[smarterclayton]

Now waiting on openshift/origin#19829

[smarterclayton]

Two small changes - CrossMount now reuses the project helper (namespaces are guaranteed to be cleaned up when we exit the test) and the makefile for integration now uses test-go to get junit output and artifacts

[openshift-ci-robot]

@smarterclayton: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/prow/integration	777e4e1	link	/test integration

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[bparees]

/lgtm

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bparees, dmage, smarterclayton

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [bparees,dmage,smarterclayton]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-docker]

e2e is broken because of an accident with azure, fixing at the source.

…

On Thu, May 24, 2018 at 1:52 PM, Ben Parees ***@***.***> wrote: /lgtm — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#88 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG7fCvDMAC8wSZ-e7EVjI9SWY-BSkoyRks5t1vNXgaJpZM4ULHD_> .

[smarterclayton]

/test e2e