Skip to content

Conversation

@wking
Copy link
Member

@wking wking commented Jun 18, 2020

Address issues like:

In .//ci-operator/step-registry/baremetalds/devscripts/conf/compact/baremetalds-devscripts-conf-compact-commands.sh line 9:
echo "export NUM_WORKERS=0" >> ${SHARED_DIR}/dev-scripts-additional-config
                               ^-----------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean:
echo "export NUM_WORKERS=0" >> "${SHARED_DIR}"/dev-scripts-additional-config

For more information:
  https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...

and:

In .//ci-operator/step-registry/baremetalds/packet/check/baremetalds-packet-check-commands.sh line 9:
PACKET_PROJECT_ID=$(cat ${CLUSTER_PROFILE_DIR}/.packet-kni-vars|grep packet_project_id|awk '{print $2}')
                        ^--------------------^ SC2086: Double quote to prevent globbing and word splitting.
                        ^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.
...
  https://www.shellcheck.net/wiki/SC2002 -- Useless cat. Consider 'cmd < file...

and:

In .//ci-operator/step-registry/baremetalds/devscripts/gather/baremetalds-devscripts-gather-commands.sh line 26:
SSHOPTS=(-o ConnectTimeout=5 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ServerAliveInterval=90 -i "${CLUSTER_PROFILE_DIR}/.packet-kni-ssh-privatekey")
            ^--------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.
                                ^----------------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.
                                                            ^--------------------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.
                                                                                            ^--------------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.

in preparation for #9772.

@stbenjam
Copy link
Member

stbenjam commented Jun 18, 2020

Why is rehearse testing so much? Most of what it's testing doesn't use these steps. Anyway, if we must do this, it's fine I guess.

IMHO most of the shellcheck recommendation are less than worthless as they hurt readability. The most annoying of which is quoting it all for values that will never have spaces in them.

/shrug
/lgtm

@openshift-ci-robot openshift-ci-robot added the ¯\_(ツ)_/¯ ¯\\\_(ツ)_/¯ label Jun 18, 2020
@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 18, 2020
@stbenjam
Copy link
Member

@andfasano @akiselev1 @derekhiggins @honza Could one of you PTAL? Thanks!

@honza
Copy link
Member

honza commented Jun 18, 2020

/approve

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 18, 2020
@stbenjam
Copy link
Member

/hold

@openshift-ci-robot openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 18, 2020
@stbenjam
Copy link
Member

stbenjam commented Jun 18, 2020

I do want to make sure e2e-metal-ipi runs to the end, hold can be removed when it does

@stbenjam
Copy link
Member

/test pj-rehearse

@stevekuznetsov
Copy link
Contributor

Why is rehearse testing so much? Most of what it's testing doesn't use these steps. Anyway, if we must do this, it's fine I guess.

@stbenjam can you specifically list jobs that should not have been rehearsed?

IMHO most of the shellcheck recommendation are less than worthless as they hurt readability. The most annoying of which is quoting it all for values that will never have spaces in them.

Trusting user input to never have spaces is a good way to have a real bad time ;)

IMO any bash without quotes is immediately suspect

@AlexNPavel
Copy link
Contributor

@stbenjam This PR modifies the file ci-operator/step-registry/openshift/e2e/test/openshift-e2e-test-commands.sh, which is used by every E2E test in the registry. Hence the very large number of rehearsals.

@stbenjam
Copy link
Member

Why is rehearse testing so much? Most of what it's testing doesn't use these steps. Anyway, if we must do this, it's fine I guess.

@stbenjam can you specifically list jobs that should not have been rehearsed?

I missed the change to something under e2e/test, so looks like rehearse did the right thing. That said, given the title of this PR indicates only baremetalds, that should probably have been separate.

@wking
Copy link
Member Author

wking commented Jun 19, 2020

Oops, will drop the accidental file. Not sure why metal was terminated.

…less 'cat'

Address issues like [1,2]:

  In .//ci-operator/step-registry/baremetalds/devscripts/conf/compact/baremetalds-devscripts-conf-compact-commands.sh line 9:
  echo "export NUM_WORKERS=0" >> ${SHARED_DIR}/dev-scripts-additional-config
                                 ^-----------^ SC2086: Double quote to prevent globbing and word splitting.

  Did you mean:
  echo "export NUM_WORKERS=0" >> "${SHARED_DIR}"/dev-scripts-additional-config

  For more information:
    https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...

and [1,2]:

  In .//ci-operator/step-registry/baremetalds/packet/check/baremetalds-packet-check-commands.sh line 9:
  PACKET_PROJECT_ID=$(cat ${CLUSTER_PROFILE_DIR}/.packet-kni-vars|grep packet_project_id|awk '{print $2}')
                          ^--------------------^ SC2086: Double quote to prevent globbing and word splitting.
                          ^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.
  ...
    https://www.shellcheck.net/wiki/SC2002 -- Useless cat. Consider 'cmd < file...

and:

  In .//ci-operator/step-registry/baremetalds/devscripts/gather/baremetalds-devscripts-gather-commands.sh line 26:
  SSHOPTS=(-o ConnectTimeout=5 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ServerAliveInterval=90 -i "${CLUSTER_PROFILE_DIR}/.packet-kni-ssh-privatekey")
              ^--------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.
                                  ^----------------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.
                                                              ^--------------------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.
                                                                                              ^--------------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.

[1]: https://deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gcs/origin-ci-test/pr-logs/pull/openshift_release/9772/pull-ci-openshift-release-master-step-registry-shellcheck/1273670881133989888#1:build-log.txt%3A2
[2]: https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_release/9772/pull-ci-openshift-release-master-step-registry-shellcheck/1273670881133989888/build-log.txt
@wking wking force-pushed the metal-shellcheck-cleanup branch from b23ca51 to cd781ad Compare June 19, 2020 02:51
@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Jun 19, 2020
@wking
Copy link
Member Author

wking commented Jun 19, 2020

Dropped the step-registry/openshift change with b23ca510b7 -> cd781ad. Not sure if the CI tooling will drop the no-longer-touched rehearsals or not...

@wking
Copy link
Member Author

wking commented Jun 19, 2020

Both rehearsals died on install with something like:

level=debug msg="Still waiting for the cluster to initialize: Multiple errors are preventing progress:\n* Could not update oauthclient \"console\" (356 of 594): the server does not recognize this resource, check extension API servers\n* Could not update prometheusrule \"openshift-cluster-samples-operator/samples-operator-alerts\" (308 of 594): the server does not recognize this resource, check extension API servers\n* Could not update role \"openshift-authentication/prometheus-k8s\" (505 of 594): resource may have been deleted\n* Could not update servicemonitor \"openshift-cloud-credential-operator/cloud-credential-operator\" (502 of 594): the server does not recognize this resource, check extension API servers\n* Could not update servicemonitor \"openshift-cluster-machine-approver/cluster-machine-approver\" (518 of 594): the server does not recognize this resource, check extension API servers\n* Could not update servicemonitor \"openshift-cluster-version/cluster-version-operator\" (8 of 594): the server does not recognize this resource, check extension API servers\n* Could not update servicemonitor \"openshift-config-operator/config-operator\" (114 of 594): the server does not recognize this resource, check extension API servers\n* Could not update servicemonitor \"openshift-image-registry/image-registry\" (514 of 594): the server does not recognize this resource, check extension API servers\n* Could not update servicemonitor \"openshift-machine-api/cluster-autoscaler-operator\" (223 of 594): the server does not recognize this resource, check extension API servers\n* deployment openshift-cluster-version/cluster-version-operator is not available MinimumReplicasUnavailable: Deployment does not have minimum availability."
bash: line 42: 11037 Killed                  timeout -s 9 105m make 

Not clear to me yet if that's something I broke or not...

@openshift-ci-robot
Copy link
Contributor

@wking: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/rehearse/openshift/installer/master/e2e-vsphere b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/cluster-network-operator/master/e2e-aws-sdn-multi b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-gcp-upi b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/cluster-network-operator/master/e2e-aws-sdn-single b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/cluster-network-operator/master/e2e-vsphere b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/open-cluster-management/registration-operator/master/e2e b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift-knative/serverless-operator/master/4.3-upgrade-tests-aws-ocp-43 b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/cloud-credential-operator/master/e2e-gcp b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/cloud-credential-operator/master/e2e-azure b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/origin/master/e2e-gcp b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-azure-shared-vpc b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/cluster-etcd-operator/master/e2e-aws b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-aws-shared-vpc b23ca510b7e475b09585086e78927b26558b39c8 link /test pj-rehearse
ci/rehearse/openshift/ovn-kubernetes/master/e2e-metal-ipi-dualstack cd781ad link /test pj-rehearse
ci/rehearse/openshift/baremetal-operator/master/e2e-metal-ipi cd781ad link /test pj-rehearse
ci/prow/pj-rehearse cd781ad link /test pj-rehearse

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@andfasano
Copy link
Contributor

e2e-metal-ipi-dualstack is expected to be not working (development in progress).
Also e2e-metal-ipi on master is currently having some issues, fixes are in progress.

@wking
Copy link
Member Author

wking commented Jun 19, 2020

So are we comfortable with the changes I'm making here, or do I need to change something?

@stbenjam
Copy link
Member

Yes, these changes are fine - #9774 (comment) I only wanted to see it run to the end.

/hold cancel
/lgtm

@openshift-ci-robot openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Jun 21, 2020
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: honza, stbenjam, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 6f20250 into openshift:master Jun 21, 2020
@openshift-ci-robot
Copy link
Contributor

@wking: Updated the following 2 configmaps:

  • step-registry configmap in namespace ci at cluster api.ci using the following files:
    • key baremetalds-devscripts-conf-compact-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/conf/compact/baremetalds-devscripts-conf-compact-commands.sh
    • key baremetalds-devscripts-conf-dualstack-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/conf/dualstack/baremetalds-devscripts-conf-dualstack-commands.sh
    • key baremetalds-devscripts-conf-ipv4-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/conf/ipv4/baremetalds-devscripts-conf-ipv4-commands.sh
    • key baremetalds-devscripts-gather-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/gather/baremetalds-devscripts-gather-commands.sh
    • key baremetalds-devscripts-setup-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/setup/baremetalds-devscripts-setup-commands.sh
    • key baremetalds-e2e-test-commands.sh using file ci-operator/step-registry/baremetalds/e2e/test/baremetalds-e2e-test-commands.sh
    • key baremetalds-packet-check-commands.sh using file ci-operator/step-registry/baremetalds/packet/check/baremetalds-packet-check-commands.sh
    • key baremetalds-packet-setup-commands.sh using file ci-operator/step-registry/baremetalds/packet/setup/baremetalds-packet-setup-commands.sh
    • key baremetalds-packet-teardown-commands.sh using file ci-operator/step-registry/baremetalds/packet/teardown/baremetalds-packet-teardown-commands.sh
  • step-registry configmap in namespace ci at cluster app.ci using the following files:
    • key baremetalds-devscripts-conf-compact-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/conf/compact/baremetalds-devscripts-conf-compact-commands.sh
    • key baremetalds-devscripts-conf-dualstack-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/conf/dualstack/baremetalds-devscripts-conf-dualstack-commands.sh
    • key baremetalds-devscripts-conf-ipv4-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/conf/ipv4/baremetalds-devscripts-conf-ipv4-commands.sh
    • key baremetalds-devscripts-gather-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/gather/baremetalds-devscripts-gather-commands.sh
    • key baremetalds-devscripts-setup-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/setup/baremetalds-devscripts-setup-commands.sh
    • key baremetalds-e2e-test-commands.sh using file ci-operator/step-registry/baremetalds/e2e/test/baremetalds-e2e-test-commands.sh
    • key baremetalds-packet-check-commands.sh using file ci-operator/step-registry/baremetalds/packet/check/baremetalds-packet-check-commands.sh
    • key baremetalds-packet-setup-commands.sh using file ci-operator/step-registry/baremetalds/packet/setup/baremetalds-packet-setup-commands.sh
    • key baremetalds-packet-teardown-commands.sh using file ci-operator/step-registry/baremetalds/packet/teardown/baremetalds-packet-teardown-commands.sh
Details

In response to this:

Address issues like:

In .//ci-operator/step-registry/baremetalds/devscripts/conf/compact/baremetalds-devscripts-conf-compact-commands.sh line 9:
echo "export NUM_WORKERS=0" >> ${SHARED_DIR}/dev-scripts-additional-config
                              ^-----------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean:
echo "export NUM_WORKERS=0" >> "${SHARED_DIR}"/dev-scripts-additional-config

For more information:
 https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...

and:

In .//ci-operator/step-registry/baremetalds/packet/check/baremetalds-packet-check-commands.sh line 9:
PACKET_PROJECT_ID=$(cat ${CLUSTER_PROFILE_DIR}/.packet-kni-vars|grep packet_project_id|awk '{print $2}')
                       ^--------------------^ SC2086: Double quote to prevent globbing and word splitting.
                       ^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.
...
 https://www.shellcheck.net/wiki/SC2002 -- Useless cat. Consider 'cmd < file...

and:

In .//ci-operator/step-registry/baremetalds/devscripts/gather/baremetalds-devscripts-gather-commands.sh line 26:
SSHOPTS=(-o ConnectTimeout=5 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ServerAliveInterval=90 -i "${CLUSTER_PROFILE_DIR}/.packet-kni-ssh-privatekey")
           ^--------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.
                               ^----------------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.
                                                           ^--------------------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.
                                                                                           ^--------------------^ SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.

in preparation for #9772.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@wking wking deleted the metal-shellcheck-cleanup branch June 21, 2020 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. ¯\_(ツ)_/¯ ¯\\\_(ツ)_/¯

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants