flake: templateservicebroker security test should pass security tests

[smarterclayton]

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/16851/test_pull_request_origin_extended_conformance_gce/9755/

[Conformance][templates] templateservicebroker security test should pass security tests [Suite:openshift/conformance/parallel] 1m43s
go run hack/e2e.go -v -test --test_args='--ginkgo.focus=\s\[Conformance\]\[templates\]\stemplateservicebroker\ssecurity\stest\sshould\spass\ssecurity\stests\s\[Suite\:openshift\/conformance\/parallel\]$'
/tmp/openshift/build-rpm-release/tito/rpmbuild-origin97017y/BUILD/origin-3.7.0/_output/local/go/src/github.com/openshift/origin/test/extended/templates/templateservicebroker_security.go:257
Expected error:
    <*client.ServerError | 0xc421d8b3c0>: {
        StatusCode: 403,
        Description: "secrets \"d2a2b8b4-b577-42c8-92b7-0f5b85d03592\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: User \"system:serviceaccount:extended-test-openshift-template-service-broker-j0pr1-hp4dk:apiserver\" cannot update brokertemplateinstances/finalizers.template.openshift.io in project \"extended-test-templates-46xbx-whdrr\", <nil>",
    }
    Forbidden: secrets "d2a2b8b4-b577-42c8-92b7-0f5b85d03592" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: User "system:serviceaccount:extended-test-openshift-template-service-broker-j0pr1-hp4dk:apiserver" cannot update brokertemplateinstances/finalizers.template.openshift.io in project "extended-test-templates-46xbx-whdrr", <nil>
not to have occurred
/tmp/openshift/build-rpm-release/tito/rpmbuild-origin97017y/BUILD/origin-3.7.0/_output/local/go/src/github.com/openshift/origin/test/extended/templates/templateservicebroker_security.go:162

Looks like permission cache not being filled yet?

@openshift/sig-security @openshift/sig-master

[simo5]

another flake due to our clients being "too fast" now ?

[sjenning]

My PR has hit this 7 times in a row.
https://openshift-gce-devel.appspot.com/pr/16851

[joelsmith]

I have 2 stage PRs that have hit it every time. Are PRs against master hitting it too? Both the PR by @sjenning and mine are against stage.
https://openshift-gce-devel.appspot.com/pr/16856
https://openshift-gce-devel.appspot.com/pr/16855

[jim-minter]

I believe this is happening because on GCE, the stage CI testing is using the (more recent) TSB image built from master code.

[jim-minter]

@bparees fyi ^

[bparees]

@jim-minter do you think there's an actual issue w/ the newer TSB image, or that the newer TSB image is exposing an issue in the security cache?

(or three, i guess, the test is flawed and should be waiting for more initialization).

Trying to make sure we have the right assignee.

[enj]

@bparees based on what I gathered from @jim-minter on IRC, the issue is that the new image performs actions that the old image did not (has new get calls). The code in master has been updated to include new RBAC rules to cover that, whereas the code in stage does not have that change. Thus when this new image runs during stage testing, it performs a get operation which it does not have permission to do (which always fails as expected). There is no cache out of sync issue AFAIK.

[bparees]

got it, thanks @enj

so is the plan to backport the rules changes to stage?

[bparees]

or just wait until stage gets refreshed at the end of this sprint?

[jim-minter]

@bparees I think this is purely an artifact of our GCE CI infrastructure not building/using docker images corresponding to the executable version.
I was aware that in GCE CI a given executable might be paired with older images from the same branch, but I was not aware until today that an executable might be paired with newer images from a different branch (executable from stage, images from master).

The choices we saw to deal with this were:

1. disable GCE test in stage
2. cherry-pick both Remove "template.openshift.io/template-instance" label #16808 and Enable asynchronous deprovision in TSB #16815 into stage. But this would also as a side-effect do 1), because that is already contained within those PRs.

So the intention is to do 1): #16887

[bparees]

not sure why #16887 didn't close this, manually closing.