crio 1.12 and ssh-bastion in a pod #11203

vrutkovs · 2019-02-15T13:19:25Z

Supersedes #11167

Setup crio 1.12 and ssh bastion in a pod to reach scaled up RHEL nodes

Failing tests:

The bootstrap user should successfully login with password decoded from kubeadmin secret [Suite:openshift/conformance/parallel]
2x [sig-api-machinery] Namespaces [Serial] should ensure that all pods are removed when a namespace is deleted [Conformance] [Suite:openshift/conformance/serial/minimal] [Suite:k8s]

vrutkovs · 2019-02-15T16:01:49Z

/retest

DanyC97 · 2019-02-15T17:44:30Z

roles/container_runtime/tasks/package_crio.yml

-    backup: yes
+    section: "crio.image"
+    option: pause_command
+    value: ' "/usr/bin/pod"'


you didn't like the template ?

imo a template would give you a much better flexibility in the long run when things will get much more stable and you will realize you want to do some tuning.

We don't directly control the crio versions, unlike docker, so its best to modify just the required parts

right, i was coming from the angle where later the default CRI-O settings won't satisfy the needs of BYO .. i guess we can cross that bridge when we come to it

DanyC97 · 2019-02-15T17:46:59Z

Vadim, love this ssh bastion pod, is a killer man ! ;)

vrutkovs · 2019-02-15T17:53:40Z

/retest

mtnbikenc · 2019-02-15T18:27:25Z

Looks like a few things changed, need to update:
machinesets.machine.openshift.io -> machinesets
openshift-machine-api -> openshift-cluster-api

$ oc get machinesets -o name -n openshift-cluster-api
machineset.cluster.k8s.io/rteague-worker-us-east-2a
machineset.cluster.k8s.io/rteague-worker-us-east-2b
machineset.cluster.k8s.io/rteague-worker-us-east-2c

mtnbikenc · 2019-02-15T18:40:23Z

Also:
machines.machine.openshift.io -> machines

sdodson · 2019-02-15T18:57:15Z

test/aws/files/01_service.yml

@@ -0,0 +1,17 @@
+apiVersion: v1


If there's any chance we'll drop more stuff in here should we name them 01_ssh_service.yml, 02_ssh_serviceaccount.yml, etc?

Right, oc apply applies files in alphabetical order, so first the namespace has to be created, then a service etc.

vrutkovs · 2019-02-15T20:00:25Z

/test e2e-aws-scaleup

mtnbikenc · 2019-02-15T21:44:56Z

My two comments above about needing to change machines and machinesets was due to the fact I was using the 0.12 installer. After building the installer from master I didn't have those issues.

…beconfig_path is set

vrutkovs · 2019-02-18T16:17:39Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-18T17:13:31Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-18T18:29:26Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-19T12:35:07Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-19T12:48:22Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-19T13:16:05Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-19T14:24:38Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-19T16:27:57Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-19T17:45:07Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-19T21:10:37Z

/retest

vrutkovs · 2019-02-20T08:57:31Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-20T09:28:08Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-20T10:01:27Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-20T11:36:25Z

/test e2e-aws-scaleup

vrutkovs · 2019-02-20T13:13:53Z

/test e2e-aws-scaleup

sdodson · 2019-02-20T14:55:26Z

/lgtm

As a team we feel that this code is generally working as intended, however we're running into persistent e2e test failures that we'll need to address with other teams. Merging as is.

openshift-ci-robot · 2019-02-20T14:55:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sdodson, vrutkovs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [sdodson,vrutkovs]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2019-02-20T15:11:27Z

@vrutkovs: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/prow/e2e-aws-scaleup	`a20829b`	link	`/test e2e-aws-scaleup`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Vadim Rutkovsky added 5 commits February 14, 2019 14:14

test/aws: copy criotested cert if it was mounted by CI

18b85ab

Change only pause_image and pause_command in crio config

cb27569

Add kubelet and cri-o logs in the output

8c469f9

MCD port is now 22623

ae676d0

Wait for scaled up machines to become running

47b2466

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 15, 2019

openshift-ci-robot requested review from michaelgugino and sdodson February 15, 2019 13:19

openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 15, 2019

vrutkovs force-pushed the devel-40-sshbastion branch 2 times, most recently from 45d20ae to ce23381 Compare February 15, 2019 14:46

Setup an ssh bastion in the pod and use it instead of masters

6b2893c

vrutkovs force-pushed the devel-40-sshbastion branch from ce23381 to 6b2893c Compare February 15, 2019 15:04

ssh bastion is not stable, add 10 retries

4a12518

Vadim Rutkovsky added 2 commits February 15, 2019 18:01

Don't copy authorized file

d4d102e

check_def_equal: user_def should always be a dict

a01cb8a

openshift-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 15, 2019

DanyC97 reviewed Feb 15, 2019

View reviewed changes

vrutkovs changed the title ~~WIP crio 1.12 and ssh-bastion in a pod~~ crio 1.12 and ssh-bastion in a pod Feb 15, 2019

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 15, 2019

sdodson reviewed Feb 15, 2019

View reviewed changes

vrutkovs force-pushed the devel-40-sshbastion branch from aeb24f3 to aca40b5 Compare February 18, 2019 12:08

Construct openshift_bootstrap_endpoint from cluster config CRDs if ku…

a19f0cd

…beconfig_path is set

vrutkovs force-pushed the devel-40-sshbastion branch 3 times, most recently from 7ba1c1d to a1d622a Compare February 18, 2019 14:34

Read IdentityFile from openshift_aws_scaleup_key_path var

a20829b

vrutkovs force-pushed the devel-40-sshbastion branch from a1d622a to a20829b Compare February 18, 2019 14:34

openshift-ci-robot assigned sdodson Feb 20, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 20, 2019

sdodson merged commit 168f41f into openshift:devel-40 Feb 20, 2019

mtnbikenc mentioned this pull request Feb 20, 2019

Use MCO --once-from instead of custom ansible module #10958

Merged

6 tasks

vrutkovs deleted the devel-40-sshbastion branch February 20, 2019 15:52

crio 1.12 and ssh-bastion in a pod #11203

crio 1.12 and ssh-bastion in a pod #11203

Uh oh!

Conversation

vrutkovs commented Feb 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vrutkovs commented Feb 15, 2019

Uh oh!

DanyC97 Feb 15, 2019

Choose a reason for hiding this comment

Uh oh!

vrutkovs Feb 15, 2019

Choose a reason for hiding this comment

Uh oh!

DanyC97 Feb 16, 2019

Choose a reason for hiding this comment

Uh oh!

DanyC97 commented Feb 15, 2019

Uh oh!

vrutkovs commented Feb 15, 2019

Uh oh!

mtnbikenc commented Feb 15, 2019

Uh oh!

mtnbikenc commented Feb 15, 2019

Uh oh!

sdodson Feb 15, 2019

Choose a reason for hiding this comment

Uh oh!

vrutkovs Feb 15, 2019

Choose a reason for hiding this comment

Uh oh!

vrutkovs commented Feb 15, 2019

Uh oh!

mtnbikenc commented Feb 15, 2019

Uh oh!

vrutkovs commented Feb 18, 2019

Uh oh!

vrutkovs commented Feb 18, 2019

Uh oh!

vrutkovs commented Feb 18, 2019

Uh oh!

vrutkovs commented Feb 19, 2019

Uh oh!

vrutkovs commented Feb 19, 2019

Uh oh!

vrutkovs commented Feb 19, 2019

Uh oh!

vrutkovs commented Feb 19, 2019

Uh oh!

vrutkovs commented Feb 19, 2019

Uh oh!

vrutkovs commented Feb 19, 2019

Uh oh!

vrutkovs commented Feb 19, 2019

Uh oh!

vrutkovs commented Feb 20, 2019

Uh oh!

vrutkovs commented Feb 20, 2019

Uh oh!

vrutkovs commented Feb 20, 2019

Uh oh!

vrutkovs commented Feb 20, 2019

Uh oh!

vrutkovs commented Feb 20, 2019

Uh oh!

sdodson commented Feb 20, 2019

Uh oh!

openshift-ci-robot commented Feb 20, 2019

Uh oh!

openshift-ci-robot commented Feb 20, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

vrutkovs commented Feb 15, 2019 •

edited

Loading