Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PODAUTO-228: Upstream rebase to 2.15.1 #35

Open
wants to merge 84 commits into
base: main
Choose a base branch
from

Conversation

jkyros
Copy link

@jkyros jkyros commented Sep 4, 2024

  1. Create a rebase tree using rebasebot:
rebasebot --source https://github.com/kedacore/keda:release/v2.15 --dest openshift/keda:main \
  --rebase jkyros/keda:rebase-2.15 --tag-policy=strict --update-go-modules --dry-run \
 --github-user-token ~/githubtoken
  1. Check all "numbered carry PRs", rebasebot likes to drop them regardles (looked like they were all legit drops)
  2. Update .ci-operator.yaml to reference rhel-9-release-golang-1.22-openshift-4.18 (vs 4.17, moving to go 1.22 finally)
  3. Squash duplicate carry commits (OWNERS, .ci-operator.yaml, e2e hacks)
  4. Add verify history drop commit to use upstream's release/v2.15 branch as the merge base
  5. Push to own fork, open PR

JorTurFer and others added 30 commits April 25, 2024 13:57
Signed-off-by: Jorge Turrado Ferrero <[email protected]>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* bump deps

Signed-off-by: Jorge Turrado <[email protected]>

* code-generators

Signed-off-by: Jorge Turrado <[email protected]>

* styles

Signed-off-by: Jorge Turrado <[email protected]>

* fix test

Signed-off-by: Jorge Turrado <[email protected]>

* downgrade pending k8s dep

Signed-off-by: Jorge Turrado <[email protected]>

---------

Signed-off-by: Jorge Turrado <[email protected]>
Bumps [helm/kind-action](https://github.com/helm/kind-action) from 1.8.0 to 1.10.0.
- [Release notes](https://github.com/helm/kind-action/releases)
- [Commits](helm/kind-action@v1.8.0...v1.10.0)

---
updated-dependencies:
- dependency-name: helm/kind-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Module `github.com/xdg/scram` was renamed to `github.com/xdg-go/scram`
in October 2018.

Signed-off-by: Mikel Olasagasti Uranga <[email protected]>
This commit fixs issue kedacore#5760. where OP was facing problem with +srv schema

Signed-off-by: Rishikesh Betigeri <[email protected]>
* added custom time horizon in gcp scaler

Signed-off-by: Yaxhveer <[email protected]>

* updated changelog

Signed-off-by: Yaxhveer <[email protected]>

* add custom duration in other gcp stackdriver

Signed-off-by: Yaxhveer <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Jorge Turrado Ferrero <[email protected]>
Signed-off-by: Yashveer <[email protected]>

---------

Signed-off-by: Yaxhveer <[email protected]>
Signed-off-by: Yashveer <[email protected]>
Co-authored-by: Jorge Turrado Ferrero <[email protected]>
…e#5782)

* chore: Remove deprecated AAD-Pod-Identity and AWS-KIAM auths

Signed-off-by: Jorge Turrado <[email protected]>

* fix CI

Signed-off-by: Jorge Turrado <[email protected]>

* fix CI

Signed-off-by: Jorge Turrado <[email protected]>

---------

Signed-off-by: Jorge Turrado <[email protected]>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…core#5746)

Signed-off-by: Simon Kobler <[email protected]>
Signed-off-by: Simon Kobler <[email protected]>
Co-authored-by: Jorge Turrado Ferrero <[email protected]>
Co-authored-by: Simon Kobler <[email protected]>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* added regex pre check before building image

Signed-off-by: Yaxhveer <[email protected]>

* updated changelog

Signed-off-by: Yaxhveer <[email protected]>

* refactored

Signed-off-by: Yaxhveer <[email protected]>

* corrected

Signed-off-by: Yaxhveer <[email protected]>

* corrected changelog

Signed-off-by: Yaxhveer <[email protected]>

* updated the workflow

Signed-off-by: Yaxhveer <[email protected]>

* updated the workflow

Signed-off-by: Yaxhveer <[email protected]>

---------

Signed-off-by: Yaxhveer <[email protected]>
* test: Run e2e tests against Kubernetes v1.28-v1.30

Signed-off-by: Tom Kerkhove <[email protected]>

* test: Run e2e tests against Kubernetes v1.28-v1.30

Signed-off-by: Tom Kerkhove <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Tom Kerkhove <[email protected]>

---------

Signed-off-by: Tom Kerkhove <[email protected]>
…dacore#5851)

Bumps [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action) from 0.19.0 to 0.21.0.
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](aquasecurity/trivy-action@d710430...fd25fed)

---
updated-dependencies:
- dependency-name: aquasecurity/trivy-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
JorTurFer and others added 22 commits July 30, 2024 15:36
* chore: build with keda-tools:1.22.5
to resolve CVE-2024-24790, CVE-2024-24789, and CVE-2024-24791
bump github.com/Azure/azure-sdk-for-go/sdk/azidentity to resolve CVE-2024-35255

Signed-off-by: Paul Yu <[email protected]>

* chore: use go install instead of go get and replacing deprecated tools

Signed-off-by: Paul Yu <[email protected]>

* chore: vendor dependency cleanup

Signed-off-by: Paul Yu <[email protected]>

* Update missing references to 1.21

Signed-off-by: Jorge Turrado <[email protected]>

---------

Signed-off-by: Paul Yu <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>
Co-authored-by: Jorge Turrado Ferrero <[email protected]>
* Add first scaler version

Signed-off-by: cyrilico <[email protected]>

* small refactor for response validation

Signed-off-by: cyrilico <[email protected]>

* Add 'from' property, rename host/token

Signed-off-by: cyrilico <[email protected]>

* Add parsing tests

Signed-off-by: cyrilico <[email protected]>

* update changelog

Signed-off-by: cyrilico <[email protected]>

* Update CHANGELOG.md

Signed-off-by: damas <[email protected]>

* Update values type to float64

Signed-off-by: damas <[email protected]>

* Remove unnecessary conversion

Signed-off-by: damas <[email protected]>

* e2e tests

Signed-off-by: cyrilico <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jorge Turrado Ferrero <[email protected]>
Signed-off-by: cyrilico <[email protected]>

* Update dynatrace_test.go

Signed-off-by: cyrilico <[email protected]>

* Fix bad templating for e2e tests

Signed-off-by: cyrilico <[email protected]>

* Revert unnecessary (?) template variable change

Signed-off-by: cyrilico <[email protected]>

* Apply suggestions from code review

Signed-off-by: Jorge Turrado Ferrero <[email protected]>

* Update tests/scalers/dynatrace/dynatrace_test.go

Signed-off-by: Jorge Turrado Ferrero <[email protected]>

* Do not allow token to be passed in scaledobject trigger

Signed-off-by: cyrilico <[email protected]>

* Remove bad secret, tweak dynakube test config

Signed-off-by: cyrilico <[email protected]>

* Rename property in response parsing

Signed-off-by: cyrilico <[email protected]>

* Update tests/scalers/dynatrace/dynatrace_test.go

Signed-off-by: Jorge Turrado Ferrero <[email protected]>

* use new operator secret, update template variable naming

Signed-off-by: cyrilico <[email protected]>

* forgotten correct variable definition

Signed-off-by: cyrilico <[email protected]>

* try default value in query for e2e tests

Signed-off-by: cyrilico <[email protected]>

* fix missing closing parenthesis, bad indenting

Signed-off-by: cyrilico <[email protected]>

* Update e2e test to use custom metrics

Signed-off-by: Jorge Turrado <[email protected]>

* Close the body to fix static checks

Signed-off-by: Jorge Turrado <[email protected]>

* use declarative scaler config

Signed-off-by: cyrilico <[email protected]>

---------

Signed-off-by: cyrilico <[email protected]>
Signed-off-by: damas <[email protected]>
Signed-off-by: Jorge Turrado Ferrero <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>
Co-authored-by: Jorge Turrado Ferrero <[email protected]>
Co-authored-by: Jorge Turrado <[email protected]>
* Remove deprecated cortexOrgId in prometheus scaler

Signed-off-by: dttung2905 <[email protected]>

* Move to breaking changes

Signed-off-by: dttung2905 <[email protected]>

---------

Signed-off-by: dttung2905 <[email protected]>
* Prepare v2.15

Signed-off-by: Jorge Turrado <[email protected]>

* Update roadmap

Signed-off-by: Jorge Turrado <[email protected]>

---------

Signed-off-by: Jorge Turrado <[email protected]>
* fix: JetSteam scaler detects leader changes correctly (kedacore#6043)

Signed-off-by: Jorge Turrado <[email protected]>

* bump deps & k8s to v0.29.7 (kedacore#6035)

Signed-off-by: Zbynek Roubalik <[email protected]>
Co-authored-by: Jorge Turrado Ferrero <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* fix: Hashicorp Vault PKI doesn't fail with due to KeyPair mismatch (kedacore#6029)

Signed-off-by: Jorge Turrado <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* update changelog

Signed-off-by: Jorge Turrado <[email protected]>

* fix style

Signed-off-by: Jorge Turrado <[email protected]>

---------

Signed-off-by: Jorge Turrado <[email protected]>
Signed-off-by: Zbynek Roubalik <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>
Co-authored-by: Zbynek Roubalik <[email protected]>
This is used by CPaaS where Cachito will make a cached copy of anything
needed during the build so that an offline brew build can succeed.
* hack/cma-verify-history.sh: checks history to make sure to use
  rebasebot-friendly commit messages
* Allow privileged pods in OpenShift test namespaces

The way the e2e test suite is set up, there are several pods that are
to running with more privilege than our "restricted" SCC provides.
Long-term I don't think there is anything in here that *requires* the
privilege, but we'll need to do some testing and find out.

In the mean time, this injects an adjusted pod admission policy into
each test namespace via the centralized namespace creation helper
function so that those privileged pods can run in the test namespaces.

* Specify securityContext for privileged e2e pods

The e2e test suite here is used to running in a "vanilla kube"
environment that does not have OpenShift/OpenShift CI restrictions. This
becomes a problem when one of the test containers attempts to do
something privileged (like bind to a privileged port) and is denied.

This just adds securityContexts to the pods that require privilege so
that they can get assigned a proper SCC and successfully run. The
securityContext addition is limited to only the tests that OpenShift
runs (internal, sequential, cpu/memory/kafka scalers) because we haven't
tested the others.

* Allow e2e test image overrides for OpenShift CI

The e2e test suite references multiple images spread across multiple
public registries (ghcr.io,docker.io,k8s.io) and some of those
registries have pull limits, which will cause our tests to fail.

We also cache some of these upstream images in our CI system, and so it
is beneficial to be able to reference our cached copy rather than have
to pull it from "the internet" every time.

Anyway, the way that the e2e tests are set up, all those images are
hard-coded in each of the manifests, which are just vars that exist in
each test's .go file. They are not templated. There is, however a
central helper function that applies all these resources (using
kubectl).

So, in order for us to be able to override the image list for CI, this
temporarily:
- adds an image rewrite map that specifies replacement images for
  images we might have difficulty pulling
- adds a helper function that will let those replacement images be
  specified by environment variables for use in CI
until we can figure out a more elegant refactor.

* Account for OpenShift CI in Prometheus build test

There is a test in the prometheus sequential suite that checks the git
commit hash of the current code and compares it to the containers
running in the test to make sure that the test version matches the code
version.

This version is injected as GIT_COMMIT during the docker builds
in the Makefile, but it does not get injected when the containers are
build in OpenShift CI. I would like to find a way to inject it via CI,
but until then we are supplying a dummy string "dummy-ci-commit-value"
that is at least "yes you are running against a CI payload we built, and
not one that you pulled from upstream".

Eventually when we figure out how to make all the variables available in
CI and inject them, this can go away because then the commits will
match.
* Pull test container dockerfile out of CI and into keda repo

Previously we were building the test container in CI from a
dockerfile_literal, which was kind of hacky and more difficult to manage
than it being here in the keda repo.

This pulls that dockerfile out of CI and into a Dockerfile.tests which
we now just reference from CI.

* Add Makefile targets to makefile for OpenShift tests

We kind of stuffed those tests into CI quick so we had something, and
when we did we didn't heavily consider ergonomics. Now that we find
ourselves having to enable additional tests for fixes and new features,
it will be much easier in the long run if we can manage the test targets
here in the repo so we don't have to put in a separate PR to the release
repo to see if our changes work.

This adds some e2e-test-openshift* makefile targets that we can point
and whatever we need to, and once CI is updated, it can just call those
targets, whatever they happen to entail.

* Reenable CPU scaler test

Now that we figured out how the CPU test was broken, we can add it back
in to the testing since it's supported.

This adds the cpu test into the e2e-test-openshift Makefile target, so
when CI calls it, it will run with the rest of the scaler tests

* Enable cron E2E tests

Signed-off-by: John Kyros <[email protected]>
This excludes deps and tests from snyk scans to cut down on noise.

This also excludes the tests/ directory as it contains some launcher .go
files that don't end in _test.go, but are part of the testing suite and
are not shipped with the final product.
There are a ton of credentials in our test suite because of how keda
works, but every last one of them is fake/useless. This just excludes
our test files from the scanner so they will stop generating
false-positives.

The preferred method to deal with them is to comment the exact line as
"# notasecret" but that's not feasible given the size of the test suite
and the number of fake credentials.

This should stop the scary false positive ALERT! emails.

Signed-off-by: John Kyros <[email protected]>
…e/v2.15 branch instead of main

Signed-off-by: John Kyros <[email protected]>
@openshift-ci-robot
Copy link

openshift-ci-robot commented Sep 4, 2024

@jkyros: This pull request references PODAUTO-228 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.18.0" version, but no target version was set.

In response to this:

  1. Create a rebase tree using rebasebot:
rebasebot --source https://github.com/kedacore/keda:release/v2.15 --dest openshift/keda:main \
 --rebase jkyros/keda:rebase-2.15 --tag-policy=strict --update-go-modules --dry-run \
--github-user-token ~/githubtoken
  1. Check all "numbered carry PRs", rebasebot likes to drop them regardles (looked like they were all legit drops)
  2. Update .ci-operator.yaml to reference rhel-9-release-golang-1.22-openshift-4.18 (vs 4.17, moving to go 1.22 finally)
  3. Squash duplicate carry commits (OWNERS, .ci-operator.yaml, e2e hacks)
  4. Add verify history drop commit to use upstream's release/v2.15 branch as the merge base
  5. Push to own fork, open PR

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 4, 2024
Copy link

openshift-ci bot commented Sep 4, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jkyros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 4, 2024
So upstream the tests are gated by the launcher, but we aren't using the
launcher, because it doesn't work in our environment. I think we should
probably make our own similar-ish launcher so these hacks don't keep
getting worse over time, but for now we're just going to work around the
azure tests.

Signed-off-by: John Kyros <[email protected]>
@jkyros
Copy link
Author

jkyros commented Sep 5, 2024

Made it past the azure test, but it looks like we failed the persistentLag part of the kafka test (which didn't change, but some tests did get added before it, so maybe they aren't returning the environment to the proper state?)

    kafka_test.go:671: 
        	Error Trace:	/src/tests/scalers/kafka/kafka_test.go:671
        	            				/src/tests/scalers/kafka/kafka_test.go:459
        	Error:      	Should be true
        	Test:       	TestScaler
        	Messages:   	replica count should be 1 after 2 minute
    helper.go:845: No pods with label app=kafka-consumer
    helper.go:626: Applying template: singleDeploymentTemplate
    helper.go:502: Waiting for deployment replicas to hit target. Deployment - kafka-test-deployment, Current  - 1, Target - 1
    helper.go:560: Waiting for some time to ensure deployment replica count doesn't change from 1

Let's see if that's a flake...
/retest-required

Copy link

openshift-ci bot commented Sep 5, 2024

@jkyros: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/keda-e2e-aws-ovn ec83452 link true /test keda-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.
Projects
None yet
Development

Successfully merging this pull request may close these issues.