Skip to content

Don't create dirs for crio#11746

Closed
mtnbikenc wants to merge 1 commit intoopenshift:masterfrom
mtnbikenc:crio-dirs
Closed

Don't create dirs for crio#11746
mtnbikenc wants to merge 1 commit intoopenshift:masterfrom
mtnbikenc:crio-dirs

Conversation

@mtnbikenc
Copy link
Member

crio 1.14.6 and above should not need these dirs created in order for
nodes to come online

Tracking crio fixes here: https://bugzilla.redhat.com/show_bug.cgi?id=1723295

Reverts: #11698

crio 1.14.6 and above should not need these dirs created in order for
nodes to come online.
@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mtnbikenc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 3, 2019
@openshift-ci-robot
Copy link

@mtnbikenc: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/prow/e2e-aws-scaleup-rhel7 684b651 link /test e2e-aws-scaleup-rhel7

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@vrutkovs
Copy link
Contributor

vrutkovs commented Jul 4, 2019

/hold

'Jul 03 20:51:48 ip-10-0-150-242.ec2.internal hyperkube[4866]: I0703 20:51:48.612918    4866 event.go:209] Event(v1.ObjectReference{Kind:"Pod", Namespace:"openshift-sdn", Name:"sdn-nf6rv", UID:"6223920b-9dd3-11e9-9530-12e052c89f14", APIVersion:"v1", ResourceVersion:"16728", FieldPath:"spec.containers{sdn}"}): type: ''Warning'' reason: ''Failed'' Error: container create failed: container_linux.go:336: starting container process caused "process_linux.go:399: container init caused \"rootfs_linux.go:58: mounting \\\"/var/lib/cni/bin\\\" to rootfs \\\"/var/lib/containers/storage/overlay/a9907778a4e9e2362486d0606e3a90982c6ea8f8dbea475e3572ddcd59588014/merged\\\" at \\\"/var/lib/containers/storage/overlay/a9907778a4e9e2362486d0606e3a90982c6ea8f8dbea475e3572ddcd59588014/merged/host/opt/cni/bin\\\" caused \\\"mkdir /var/lib/containers/storage/overlay/a9907778a4e9e2362486d0606e3a90982c6ea8f8dbea475e3572ddcd59588014/merged/host/opt/cni: read-only file system\\\"\""'

@openshift-ci-robot openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 4, 2019
@haircommander
Copy link
Member

Related cri-o/cri-o#2564
Will look into it tomorrow

@haircommander
Copy link
Member

for the time being, I would actually continue to create these directories (and not make the changes in this PR).

This is actually partially a problem in kubernetes, partially a problem of runc not properly reporting the issue. What (seems) to be happening is the sdn pod is attempting to mount the directories as a volume. It seems this volume is of type hostPath, and of the default type (empty string). The empty string means there's no verification that happens before the volume is mounted (ref: https://kubernetes.io/docs/concepts/storage/volumes/#hostpath). runc then reports that error when the mount failed, because there's no directory to mount from

I have yet to reproduce myself, but given what I know, I don't see a graceful work around in CRI-O yet

@mtnbikenc
Copy link
Member Author

Per #11746 (comment)
/close

@openshift-ci-robot
Copy link

@mtnbikenc: Closed this PR.

Details

In response to this:

Per #11746 (comment)
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mtnbikenc mtnbikenc deleted the crio-dirs branch July 8, 2019 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants