ci-operator/step-registry/upi: Fix ShellCheck quoting, fgrep, etc. #10113

wking · 2020-07-08T15:52:50Z

Addressing:

$ sudo podman run --rm --volume "${PWD}:/workdir:ro,z" --workdir /workdir registry.svc.ci.openshift.org/ci/shellcheck sh -c 'find ci-operator/step-registry/upi -name "*.sh" -print0 | xargs -0 -n1 shellcheck'
In ci-operator/step-registry/upi/conf/gcp/upi-conf-gcp-commands.sh line 32:
  HOST_PROJECT=$(jq -r '.hostProject' ${SHARED_DIR}/xpn.json)
                                      ^-----------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean:
  HOST_PROJECT=$(jq -r '.hostProject' "${SHARED_DIR}"/xpn.json)
...
In ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh line 46:
        echo "command failed, retrying in $(( 2 ** $attempt )) seconds"
                                                   ^------^ SC2004: $/${} is unnecessary on arithmetic variables.
...
In ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh line 290:
IMAGE_SOURCE=$(cat /var/lib/openshift-install/rhcos.json | jq -r .gcp.url)
                   ^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.
...
In ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh line 365:
fgrep -e '-master-0' 05_control_plane.py
^---^ SC2197: fgrep is non-standard and deprecated. Use grep -F instead.
...
For more information:
  https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
  https://www.shellcheck.net/wiki/SC2197 -- fgrep is non-standard and depreca...
  https://www.shellcheck.net/wiki/SC2002 -- Useless cat. Consider 'cmd < file...

More groundwork for #9772.

Addressing: $ sudo podman run --rm --volume "${PWD}:/workdir:ro,z" --workdir /workdir registry.svc.ci.openshift.org/ci/shellcheck sh -c 'find ci-operator/step-registry/upi -name "*.sh" -print0 | xargs -0 -n1 shellcheck' In ci-operator/step-registry/upi/conf/gcp/upi-conf-gcp-commands.sh line 32: HOST_PROJECT=$(jq -r '.hostProject' ${SHARED_DIR}/xpn.json) ^-----------^ SC2086: Double quote to prevent globbing and word splitting. Did you mean: HOST_PROJECT=$(jq -r '.hostProject' "${SHARED_DIR}"/xpn.json) ... In ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh line 46: echo "command failed, retrying in $(( 2 ** $attempt )) seconds" ^------^ SC2004: $/${} is unnecessary on arithmetic variables. ... In ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh line 290: IMAGE_SOURCE=$(cat /var/lib/openshift-install/rhcos.json | jq -r .gcp.url) ^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead. ... In ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh line 365: fgrep -e '-master-0' 05_control_plane.py ^---^ SC2197: fgrep is non-standard and deprecated. Use grep -F instead. ... For more information: https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ... https://www.shellcheck.net/wiki/SC2197 -- fgrep is non-standard and depreca... https://www.shellcheck.net/wiki/SC2002 -- Useless cat. Consider 'cmd < file...

wking · 2020-07-08T15:56:34Z

/assign @jstuever

Because most touches are to the GCP steps, and that's your space :).

wking · 2020-07-08T18:14:34Z

e2e-gcp-upi-xpn died on e2e test-case failures, so not a result of step script changes (or at least, very unlikely ;).

e2e-snc died on install:

level=fatal msg="failed to initialize the cluster: Working towards 0.0.1-2020-07-08-155929: 87% complete, waiting on cluster-autoscaler, console"
failed to create the cluster, but that is expected.  We will block on a successful cluster via a future wait-for.
...
error: unable to recognize "kubelet-bootstrap-cred-manager-ds.yaml": Get https://api.crc.testing:6443/api?timeout=32s: dial tcp 192.168.126.11:6443: connect: connection refused
error: failed to execute wrapped command: exit status 1

Also seems unlikely to be a result of my changes.

e2e-gcp-upi died in install:

ERROR: (gcloud.deployment-manager.deployments.create) Error in Operation [operation-1594226118523-5a9f0b17c7c1e-7ba73bc2-773aaec7]: errors:
- code: RESOURCE_ERROR
  location: /deployments/ci-op-8di1zriy-fbda7-mcqrx-control-plane/resources/ci-op-8di1zriy-fbda7-mcqrx-master-2
  message: "{\"ResourceType\":\"compute.v1.instance\",\"ResourceErrorCode\":\"403\"\
    ,\"ResourceErrorMessage\":{\"code\":403,\"message\":\"Quota exceeded for quota\
    \ group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com'\
    \ for consumer 'project_number:1053217076791'.\",\"status\":\"PERMISSION_DENIED\"\
    ,\"details\":[{\"@type\":\"type.googleapis.com/google.rpc.Help\",\"links\":[{\"\
    description\":\"Google developer console API key\",\"url\":\"https://console.developers.google.com/project/1053217076791/apiui/credential\"\
    }]}],\"statusMessage\":\"Forbidden\",\"requestPath\":\"https://www.googleapis.com/compute/v1/projects/openshift-gce-devel-ci/zones/us-east1-d/instances/ci-op-8di1zriy-fbda7-mcqrx-master-2\"\
    ,\"httpMethod\":\"GET\"}}"

followed by a number of later steps failing because the cluster wasn't up. Not related to my changes either. So I think we're good to go for this PR.

jstuever · 2020-07-10T20:56:40Z

/hold
e2e-gcp-upi-xpn has never succeeded here.
I don't know what e2e-sync is.
Let's at least get a successful run on e2e-gcp-upi.

jstuever · 2020-07-10T20:56:55Z

/lgtm

jstuever · 2020-07-10T20:57:09Z

/test pj-rehearse

openshift-ci-robot · 2020-07-10T20:57:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jstuever, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/step-registry/upi/conf/gcp/OWNERS~~ [jstuever,wking]
~~ci-operator/step-registry/upi/deprovision/gcp/OWNERS~~ [jstuever,wking]
~~ci-operator/step-registry/upi/gcp/nested/post/OWNERS~~ [jstuever,wking]
~~ci-operator/step-registry/upi/gcp/nested/pre/OWNERS~~ [jstuever,wking]
~~ci-operator/step-registry/upi/install/gcp/OWNERS~~ [jstuever,wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jstuever · 2020-07-15T21:35:24Z

/hold cancel
e2e-gcp-upi succeeded.

openshift-bot · 2020-07-15T22:28:36Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot · 2020-07-15T22:36:39Z

@wking: Updated the following 2 configmaps:

step-registry configmap in namespace ci at cluster api.ci using the following files:
- key upi-conf-gcp-commands.sh using file ci-operator/step-registry/upi/conf/gcp/upi-conf-gcp-commands.sh
- key upi-deprovision-gcp-commands.sh using file ci-operator/step-registry/upi/deprovision/gcp/upi-deprovision-gcp-commands.sh
- key upi-gcp-nested-post-commands.sh using file ci-operator/step-registry/upi/gcp/nested/post/upi-gcp-nested-post-commands.sh
- key upi-gcp-nested-pre-commands.sh using file ci-operator/step-registry/upi/gcp/nested/pre/upi-gcp-nested-pre-commands.sh
- key upi-install-gcp-commands.sh using file ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh
step-registry configmap in namespace ci at cluster app.ci using the following files:
- key upi-conf-gcp-commands.sh using file ci-operator/step-registry/upi/conf/gcp/upi-conf-gcp-commands.sh
- key upi-deprovision-gcp-commands.sh using file ci-operator/step-registry/upi/deprovision/gcp/upi-deprovision-gcp-commands.sh
- key upi-gcp-nested-post-commands.sh using file ci-operator/step-registry/upi/gcp/nested/post/upi-gcp-nested-post-commands.sh
- key upi-gcp-nested-pre-commands.sh using file ci-operator/step-registry/upi/gcp/nested/pre/upi-gcp-nested-pre-commands.sh
- key upi-install-gcp-commands.sh using file ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh

Details

In response to this:

Addressing:

$ sudo podman run --rm --volume "${PWD}:/workdir:ro,z" --workdir /workdir registry.svc.ci.openshift.org/ci/shellcheck sh -c 'find ci-operator/step-registry/upi -name "*.sh" -print0 | xargs -0 -n1 shellcheck'
In ci-operator/step-registry/upi/conf/gcp/upi-conf-gcp-commands.sh line 32:
 HOST_PROJECT=$(jq -r '.hostProject' ${SHARED_DIR}/xpn.json)
                                     ^-----------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean:
 HOST_PROJECT=$(jq -r '.hostProject' "${SHARED_DIR}"/xpn.json)
...
In ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh line 46:
       echo "command failed, retrying in $(( 2 ** $attempt )) seconds"
                                                  ^------^ SC2004: $/${} is unnecessary on arithmetic variables.
...
In ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh line 290:
IMAGE_SOURCE=$(cat /var/lib/openshift-install/rhcos.json | jq -r .gcp.url)
                  ^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.
...
In ci-operator/step-registry/upi/install/gcp/upi-install-gcp-commands.sh line 365:
fgrep -e '-master-0' 05_control_plane.py
^---^ SC2197: fgrep is non-standard and deprecated. Use grep -F instead.
...
For more information:
 https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
 https://www.shellcheck.net/wiki/SC2197 -- fgrep is non-standard and depreca...
 https://www.shellcheck.net/wiki/SC2002 -- Useless cat. Consider 'cmd < file...

More groundwork for #9772.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot requested review from cfergeau and praveenkumar July 8, 2020 15:53

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 8, 2020

openshift-ci-robot assigned jstuever Jul 8, 2020

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 10, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 10, 2020

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 15, 2020

openshift-merge-robot merged commit ac2a12b into openshift:master Jul 15, 2020

wking deleted the upi-shellcheck-cleanup branch July 16, 2020 23:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ci-operator/step-registry/upi: Fix ShellCheck quoting, fgrep, etc. #10113

ci-operator/step-registry/upi: Fix ShellCheck quoting, fgrep, etc. #10113

Uh oh!

wking commented Jul 8, 2020

Uh oh!

wking commented Jul 8, 2020 •

edited

Loading

Uh oh!

wking commented Jul 8, 2020

Uh oh!

jstuever commented Jul 10, 2020 •

edited

Loading

Uh oh!

jstuever commented Jul 10, 2020

Uh oh!

jstuever commented Jul 10, 2020

Uh oh!

openshift-ci-robot commented Jul 10, 2020

Uh oh!

jstuever commented Jul 15, 2020

Uh oh!

openshift-bot commented Jul 15, 2020

Uh oh!

openshift-ci-robot commented Jul 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ci-operator/step-registry/upi: Fix ShellCheck quoting, fgrep, etc. #10113

ci-operator/step-registry/upi: Fix ShellCheck quoting, fgrep, etc. #10113

Uh oh!

Conversation

wking commented Jul 8, 2020

Uh oh!

wking commented Jul 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wking commented Jul 8, 2020

Uh oh!

jstuever commented Jul 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jstuever commented Jul 10, 2020

Uh oh!

jstuever commented Jul 10, 2020

Uh oh!

openshift-ci-robot commented Jul 10, 2020

Uh oh!

jstuever commented Jul 15, 2020

Uh oh!

openshift-bot commented Jul 15, 2020

Uh oh!

openshift-ci-robot commented Jul 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

wking commented Jul 8, 2020 •

edited

Loading

jstuever commented Jul 10, 2020 •

edited

Loading