WIP: Use the kubelet on RHCOS instead of from origin-node image

[eparis]

This would break CL....

[wking]

/retest

#181 fixed the gofmt stuff. Also kick Jenkins for smoke tests:

retest this please

[wking]

Can we drop this from modules/tectonic/resources/tectonic-wrapper.sh too? I don't understand where that script is being run yet, but it would be nice to drop our dependency on the hyperkube image completely.

[eparis]

/retest
I can try to see what uses that script (and in turn image) as my next side project...

[eparis]

Can we merge this? If I promise to come back and look at the (unrelated) tectonic-wrapper.sh? and start ripping out CL entirely...

[wking]

The smoke test errors included:

 smoke_test.go:113: retrying in 10s
 smoke_test.go:112: failed with error: failed to connect to API server: the server could not find the requested resource
 smoke_test.go:113: retrying in 10s
 common_test.go:18: Failed to connect to API server in 15m0s.

I don't know if that's related or not. Maybe you need to bump the RHCOS AMI before using the system kubelet? [Edit: actually, the smoke tests don't use that config, so I have no idea what's broken :/]

[wking]

I don't know if that's related or not. Maybe you need to bump the RHCOS AMI before using the system kubelet? [Edit: actually, the smoke tests don't use that config, so I have no idea what's broken :/]

Now that I've finished my coffee, I've remembered ;). The smoke tests are currently running via Jenkins in a separate AWS account (Prow is in 460538899914, Jenkins/smoke is in 846518947292). The RHCOS AMIs aren't public, and only the Prow account has the access credentials for them. So the smoke-test failures are probably because the CoreOS AMI being used is missing a host kubelet, and everything after that hangs. In order to get this PR to work, we need to do one of:

a. Add a native kubelet to the CoreOS stable series.
b. Move the smoke tests over to Prow and/or the Prow AWS account, from which they'll have access to the RHCOS AMIs.
c. Drop the smoke tests in favor of more e2e-aws coverage. Work in progress on this front in openshift/release#1271.

In general, moving the default to use a private AMI stream is going to make it harder for devs and others to use the installer. Is there a public stream that is similiar enough to the RHCOS stream that we can use for development?

[abhinavdahiya]

Can we drop this from modules/tectonic/resources/tectonic-wrapper.sh too? I don't understand where that script is being run yet, but it would be nice to drop our dependency on the hyperkube image completely.

The tectonic-wrapper.sh allows tectonic.sh to run inside a container that has oc binary

installer/modules/tectonic/resources/tectonic.sh

Line 8 in 879a4c9

KUBECTL="oc --config=$KUBECONFIG"

[wking]

The tectonic-wrapper.sh allows tectonic.sh to run inside a container that has oc binary.

Hyperkube seems like a heavy dependency if that's all we'll need it for. Do the origin folks publish an image that has only oc, kubectl, and enough-of-POSIX (e.g. via BusyBox)? Effectively that would be an origin-client image. Or maybe I'm overestimating the size of hyperkube?

[abhinavdahiya]

Hyperkube seems like a heavy dependency if that's all we'll need it for.

We needed hyperkube on host for kubelet anyways; So it wasn't a problem. But moving forward as we would not need hyperkube image, i think we can move to minimal image.

But oc is required; kubectl does not work well with securitycontextconstraints objects.

[eparis]

https://github.com/coreos-inc/tectonic-operators/pull/447 would need to merge, and then this get updated.

[eparis]

/retest
now that my forked tectonic-operators are public

[sjenning]

fyi me @rphillips @derekwaynecarr

[abhinavdahiya]

@eparis https://jenkins-kube-lifecycle.prod.coreos.systems/job/tectonic-operators/job/master/418/display/redirect created new operators with changes from https://github.com/coreos-inc/tectonic-operators/pull/447 and https://github.com/coreos-inc/tectonic-operators/pull/451.

You can use b0e6febc1dfe4998f911c4ce2c911228a3fdcb38 tag instead of personal images

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: eparis
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: yifan-gu

If they are not already assigned, you can assign the PR to them by writing /assign @yifan-gu in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[eparis]

/retest

[openshift-ci-robot]

@eparis: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/prow/e2e-aws	86669fe	link	/test e2e-aws

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[crawford]

@eparis I'm working on stripping Container Linux from the installer. Do you mind if I close this out in favor of a future PR?