Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v2] Add Managed Identity support for Azure Monitor scaler to changelog #943

Closed
wants to merge 49 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
5ab1b84
update operator-sdk & deps, k8s v0.17.4
Apr 1, 2020
1dea5dd
introduce version 2.0.0-alpha1
Apr 3, 2020
76a2f02
change apiGroup to keda.sh
Apr 3, 2020
b6cf68a
introduce separate ScaledObject and ScaledJob
Apr 3, 2020
3f35897
ScaledObject scales any resource with /scale subresource
Apr 6, 2020
94cc2f7
instructions on trying v2 alpha
Apr 6, 2020
32e38cf
call handleScaledObject() directly
Apr 7, 2020
6c7d819
[v2] autoscaling v2beta1 -> v2beta2 and minor refactoring (#734)
zroubalik Apr 8, 2020
7e45c31
update operator-sdk cli to v0.16.0
Apr 9, 2020
44baa30
Add a temporary github action for v2 branch (#738)
ahmelsayed Apr 11, 2020
f61370c
Refactor scaleHandler (#735)
ahmelsayed Apr 16, 2020
646753b
[v2] Refactor scaleHandler cont. (#747)
zroubalik Apr 16, 2020
7c91553
[v2] Adding ScaledObject.Status Conditions (#750)
zroubalik Apr 16, 2020
1cadc94
Update PR build action for v2 (#751)
ahmelsayed Apr 17, 2020
121532e
[v2] move GVKR under api package (#756)
zroubalik Apr 17, 2020
09ea3e7
[v2] update to operator-sdk v0.17.0 (#757)
zroubalik Apr 17, 2020
179b283
[v2] minor log messages improvements (#758)
zroubalik Apr 17, 2020
f7f80cd
[v2] use patch to update Status.LastActiveTime (#755)
zroubalik Apr 21, 2020
5e771f2
use operator-sdk v0.17.0 in build-tools
Apr 27, 2020
0ab2221
Introduce KEDA client-go
Apr 28, 2020
ff19224
Check clientset is up to date during PR validation
Apr 28, 2020
1720888
generate CRD shortnames
Apr 29, 2020
75724b9
show related trigger authentication (#794)
zroubalik Apr 29, 2020
efc392c
[v2] triggerauth CRD: show configured parameters (#795)
zroubalik Apr 29, 2020
b67760c
[v2] adding liveness and readiness probes (#792)
zroubalik Apr 29, 2020
c451175
[V2] bump k8s to 1.18 and remove knative.dev deps (#803)
zroubalik May 4, 2020
766d382
fix typo in V2 deploy instructions
zroubalik May 13, 2020
aab3177
suppor HPA configurable scaling behavior on k8s >= 1.18
May 5, 2020
cc77968
resolve merge conflicts after rebase
May 20, 2020
0cdf6b7
[v2] fix e2e - deploy ScaledJob CRD (#846)
zroubalik May 20, 2020
64f6688
[v2] post merge: change apiversion to keda.sh (#843)
zroubalik May 20, 2020
dd0449d
[v2] rbac: allow get on any object (#842)
zroubalik May 20, 2020
fc8feb5
[v2] build-tools: updated operator-sdk to v0.17.1 (#854)
zroubalik May 22, 2020
7070cfc
[v2] during installation ignore ScaledJob CRD validation erros (#859)
zroubalik May 25, 2020
0aad2fb
Add PushScaler interface and impl external-push scaler (#865)
ahmelsayed Jun 16, 2020
e46c535
[v2] removed deprecated brokerList for Kafka scaler (#882)
samuelmacko Jun 17, 2020
0db8e01
fixes and updates post merge from master
Jun 17, 2020
539bcfa
update deps and operator-sdk to v0.18.1
Jun 17, 2020
0744d8d
add v2 features (#898)
zroubalik Jun 25, 2020
d62f6d4
Fix broken links in changelog (#901)
tomkerkhove Jun 26, 2020
be3cb0c
[v2] Adding Standard Resource metrics to KEDA (#874)
ckuduvalli Jun 30, 2020
42e6d40
[v2] change how kafka admin client is created (#907)
zroubalik Jul 2, 2020
c77458b
refactor for v2
Jul 7, 2020
76395e9
[v2] add Standard Resource metrics note to Changelog (#920)
zroubalik Jul 8, 2020
7de7b0e
[v2] set default resource requests and limits (#921)
zroubalik Jul 9, 2020
5d5b04f
[v2] enforce generation of CRDs in v1beta1 version (#922)
zroubalik Jul 9, 2020
9295280
[v2] temporary fix to be able to deploy ScaledJob on k8s 1.18 (#926)
zroubalik Jul 9, 2020
e107f26
[v2]adds consumer offset reset policy option to keda kafka scaler (#925)
grassiale Jul 20, 2020
6a2bab9
Add Managed Identity support for Azure Monitor scaler to changelog
tomkerkhove Jul 22, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/pr-validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ jobs:
name: Validate
runs-on: ubuntu-latest
# build-tools is built from ../../tools/build-tools.Dockerfile
container: kedacore/build-tools:latest
container: kedacore/build-tools:v2
steps:
- name: Check out code
uses: actions/checkout@v1
Expand All @@ -20,6 +20,9 @@ jobs:
restore-keys: |
${{ runner.os }}-go-

- name: Verify Generated clientset is up to date
run: make verify-clientset

- name: Test
run: make test

Expand Down
46 changes: 46 additions & 0 deletions .github/workflows/v2-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: master build
on:
push:
branches:
- v2
jobs:
validate:
name: Validate
runs-on: ubuntu-latest
# build-tools is built from ../../tools/build-tools.Dockerfile
container: kedacore/build-tools:v2
steps:
- name: Check out code
uses: actions/checkout@v1
with:
fetch-depth: 1

- name: Go modules cache
uses: actions/cache@v1
with:
path: /go/pkg
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-

- name: Test
run: make test

- name: Login to Docker Hub
env:
DOCKER_HUB_ACCESS_TOKEN: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
run: echo $DOCKER_HUB_ACCESS_TOKEN | docker login -u $DOCKER_HUB_USERNAME --password-stdin

- name: Publish
run: make publish

- name: Run end to end tests
env:
AZURE_SUBSCRIPTION: ${{ secrets.AZURE_SUBSCRIPTION }}
AZURE_RESOURCE_GROUP: ${{ secrets.AZURE_RESOURCE_GROUP }}
AZURE_SP_ID: ${{ secrets.AZURE_SP_ID }}
AZURE_SP_KEY: ${{ secrets.AZURE_SP_KEY }}
AZURE_SP_TENANT: ${{ secrets.AZURE_SP_TENANT }}
TEST_STORAGE_CONNECTION_STRING: ${{ secrets.TEST_STORAGE_CONNECTION_STRING }}
run: make e2e-test
33 changes: 30 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,45 @@

## Deprecations

- As of v1.3, support for `brokerList` is deprecated for our Kafka topic scaler and will be removed in v2.0 ([#632](https://github.com/kedacore/keda/issues/632))

## History

- [v2.0.0](#v200)
- [v1.4.1](#v141)
- [v1.4.0](#v140)
- [v1.3.0](#v130)
- [v1.2.0](#v120)
- [v1.1.0](#v110)
- [v1.0.0](#v100)

## v2.0.0

### New

- KEDA scales any CustomResource that implements Scale subresource ([#703](https://github.com/kedacore/keda/issues/703))
- Provide KEDA go-client ([#494](https://github.com/kedacore/keda/issues/494))
- Define KEDA readiness and liveness probes ([#788](https://github.com/kedacore/keda/issues/788))
- KEDA Support for configurable scaling behavior in HPA v2beta2 ([#802](https://github.com/kedacore/keda/issues/802))
- Adding Standard Resource metrics to KEDA ([#874](https://github.com/kedacore/keda/pull/874))
- Managed Identity support for Azure Monitor scaler ([#936](https://github.com/kedacore/keda/issues/936))

### Improvements

- HPA: move from autoscaling v2beta1 to v2beta2 ([#721](https://github.com/kedacore/keda/issues/721))
- Introduce shortnames for CRDs ([#774](https://github.com/kedacore/keda/issues/774))
- kubectl get scaledobject should show related trigger authentication ([#777](https://github.com/kedacore/keda/issues/777))
- kubectl get triggerauthentication should show information about configured parameters ([#778](https://github.com/kedacore/keda/issues/778))

### Breaking Changes

- Change apiGroup from keda.k8s.io to keda.sh ([#552](https://github.com/kedacore/keda/issues/552))
- Introduce a separate ScaledObject and ScaledJob([#653](https://github.com/kedacore/keda/issues/653))
- Remove `New()` and `Close()` from the interface of `service ExternalScaler` in `externalscaler.proto`.
- Removed deprecated brokerList for Kafka scaler ([#882](https://github.com/kedacore/keda/pull/882))

### Other
- Update Operator SDK and k8s deps ([#870](https://github.com/kedacore/keda/issues/870))
- Added ScaledObject Status Conditions to display status of scaling ([#750](https://github.com/kedacore/keda/pull/750))

## v1.4.1

### New
Expand Down Expand Up @@ -74,7 +102,6 @@ None.
### Improvements

- Make targetQueryValue configurable in postgreSQL scaler ([#643](https://github.com/kedacore/keda/pull/643))
- Added bootstrapServers to deprecate brokerList ([#621](https://github.com/kedacore/keda/pull/621))
- Removed the need for deploymentName label ([#644](https://github.com/kedacore/keda/pull/644))
- Adding Kubernetes recommended labels to resources ([#596](https://github.com/kedacore/keda/pull/596))

Expand Down
16 changes: 8 additions & 8 deletions CREATE-NEW-SCALER.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ In order to develop a scaler, a developer should do the following:
2. Create the new scaler struct under the `pkg/scalers` folder.
3. Implement the methods defined in the [scaler interface](#scaler-interface) section.
4. Create a constructor according to [this](#constructor).
5. Change the `getScaler` function in `pkg/handler/scale_handler.go` by adding another switch case that matches your scaler.
5. Change the `getScaler` function in `pkg/scaling/scale_handler.go` by adding another switch case that matches your scaler.
6. Run `make build` from the root of KEDA and your scaler is ready.

If you want to deploy locally
If you want to deploy locally
1. Run `export VERSION=local`
2. Open the terminal and go to the root of the source code
3. Run `make build`
5. If you haven't done it yet clone the charts repository: `git clone [email protected]:kedacore/charts.git`
5. If you haven't done it yet clone the charts repository: `git clone [email protected]:kedacore/charts.git`
6. In the terminal, navigate to the `chart/keda` folder (the charts downloaded in step 3), and run the following command (don't forget to replace the placeholder text in the command) `helm install . --set image.keda=kedacore/keda:[tag used in step 1],image.pullPolicy=IfNotPresent`.

The last step assumes that you have `helm` already installed in the cluster. In this step we install the helm chart, and we substitute the image with the image we built in step 1. Notice that we are also overriding the image PullPolice to `IfNotPresent` since this is a local cluster.
Expand All @@ -29,12 +29,12 @@ The scalers in KEDA are implementations of a KEDA `Scaler` Go interface declared
This is the key function of a scaler; it returns a value that represents a current state of an external metric (e.g. length of a queue). The return type is an `ExternalMetricValue` struct which has the following fields:
- `MetricName`: this is the name of the metric that we are returning.
- `Timestamp`: indicates the time at which the metrics were produced.
- `WindowSeconds`: //TODO
- `WindowSeconds`: //TODO
- `Value`: A numerical value that represents the state of the metric. It could be the length of a queue, or it can be the amount of lag in a stream, but it can also be a simple representation of the state.

Kubernetes HPA (Horizontal Pod Autoscaler) will poll `GetMetrics` reulgarly through KEDA's metric server (as long as there is at least one pod), and compare the returned value to a configured value in the ScaledObject configuration. Kubernetes will use the following formula to decide whether to scale the pods up and down:
Kubernetes HPA (Horizontal Pod Autoscaler) will poll `GetMetrics` reulgarly through KEDA's metric server (as long as there is at least one pod), and compare the returned value to a configured value in the ScaledObject configuration. Kubernetes will use the following formula to decide whether to scale the pods up and down:

`desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]`.
`desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]`.

For more details check [Kubernetes HPA documentation](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/).

Expand All @@ -50,15 +50,15 @@ The return type of this function is `MetricSpec`, but in KEDA's case we will mos

### IsActive

For some reason, the scaler might need to declare itself as in-active, and the way it can do this is through implementing the function `IsActive`.
For some reason, the scaler might need to declare itself as in-active, and the way it can do this is through implementing the function `IsActive`.

KEDA polls ScaledObject object according to the `pollingInterval` confiugred in the ScaledObject; it checks the last time it was polled, it checks if the number of replicas is greater than 0, and if the scaler itself is active. So if the scaler returns false for `IsActive`, and if current number of replicas is greater than 0, and there is no configured minimum pods, then KEDA scales down to 0.

### Close
After each poll on the scaler to retrieve the metrics, KEDA calls this function for each scaler to give the scaler the opportunity to close any resources, like http clients for example.

### Constructor
What is missing from the `scaler` interface is a function that constructs the scaler itself. Up until the moment of writing this document, KEDA does not have a dynamic way to load scalers (at least not officially)[***]; instead scalers are part of KEDA's code-base, and they are shipped with KEDA's binary.
What is missing from the `scaler` interface is a function that constructs the scaler itself. Up until the moment of writing this document, KEDA does not have a dynamic way to load scalers (at least not officially)[***]; instead scalers are part of KEDA's code-base, and they are shipped with KEDA's binary.

Thus, each scaler should have a constructing function, KEDA will [explicitly invoke](https://github.com/kedacore/keda/blob/4d0cf5ef09ef348cf3a158634910f00741ae5258/pkg/handler/scale_handler.go#L565) the construction function based on the `trigger` property configured in the ScaledObject.

Expand Down
28 changes: 25 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
##################################################
# Variables #
##################################################
VERSION ?= master
VERSION ?= 2.0.0-alpha1
IMAGE_REGISTRY ?= docker.io
IMAGE_REPO ?= kedacore

Expand Down Expand Up @@ -94,8 +94,12 @@ ifndef GOROOT
@echo "WARNING: GOROOT is not defined"
endif

.PHONY: gofmt
gofmt:
go fmt ./...

.PHONY: build
build: checkenv build-adapter build-controller
build: gofmt checkenv build-adapter build-controller

.PHONY: build-controller
build-controller: generate-api pkg/scalers/liiklus/LiiklusService.pb.go
Expand All @@ -113,10 +117,28 @@ build-adapter: generate-api pkg/scalers/liiklus/LiiklusService.pb.go
.PHONY: generate-api
generate-api:
$(GO_BUILD_VARS) operator-sdk generate k8s
$(GO_BUILD_VARS) operator-sdk generate openapi
$(GO_BUILD_VARS) operator-sdk generate crds --crd-version=v1beta1
# withTriggers and withPods are only used for duck typing so we only need the deepcopy methods
# However operator-sdk generate doesn't appear to have an option for that
# until this issue is fixed: https://github.com/kubernetes-sigs/controller-tools/issues/398
rm deploy/crds/keda.sh_withtriggers_crd.yaml
rm deploy/crds/keda.sh_withpods_crd.yaml

pkg/scalers/liiklus/LiiklusService.pb.go: hack/LiiklusService.proto
protoc -I hack/ hack/LiiklusService.proto --go_out=plugins=grpc:pkg/scalers/liiklus

pkg/scalers/liiklus/mocks/mock_liiklus.go: pkg/scalers/liiklus/LiiklusService.pb.go
mockgen github.com/kedacore/keda/pkg/scalers/liiklus LiiklusServiceClient > pkg/scalers/liiklus/mocks/mock_liiklus.go

##################################################
# Clientset #
##################################################
.PHONY: verify-clientset
verify-clientset:
$(GO_BUILD_VARS) go mod vendor
./hack/verify-codegen.sh

.PHONY: generate-clientset
generate-clientset:
$(GO_BUILD_VARS) go mod vendor
./hack/update-codegen.sh
19 changes: 17 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
<p style="font-size: 30px" align="center"><b>This branch contains unstable KEDA v2.0.0-alpha1, currently under development</b></p>

## How can I try KEDA v2 alpha version?
Make sure to remove previous KEDA (including CRD) from the cluster. Switch to the `v2` branch and deploy yaml files:
```bash
git fetch --all
git checkout v2
kubectl apply -f deploy/crds/keda.sh_scaledobjects_crd.yaml
kubectl apply -f deploy/crds/keda.sh_scaledjobs_crd.yaml
kubectl apply -f deploy/crds/keda.sh_triggerauthentications_crd.yaml
kubectl apply -f deploy/
```


<p align="center"><img src="images/keda-logo-transparent.png" width="300"/></p>
<p style="font-size: 25px" align="center"><b>Kubernetes-based Event Driven Autoscaling</b></p>
<p style="font-size: 25px" align="center">
Expand Down Expand Up @@ -120,8 +134,9 @@ To be KEDA to be fully operational we need to deploy Metrics Server first.

1. Deploy CRDs and KEDA into `keda` namespace
```bash
kubectl apply -f deploy/crds/keda.k8s.io_scaledobjects_crd.yaml
kubectl apply -f deploy/crds/keda.k8s.io_triggerauthentications_crd.yaml
kubectl apply -f deploy/crds/keda.sh_scaledobjects_crd.yaml
kubectl apply -f deploy/crds/keda.sh_scaledjob_crd.yaml
kubectl apply -f deploy/crds/keda.sh_triggerauthentications_crd.yaml
kubectl apply -f deploy/
```
2. Scale down `keda-operator` Deployment
Expand Down
8 changes: 0 additions & 8 deletions build/bin/entrypoint
Original file line number Diff line number Diff line change
@@ -1,11 +1,3 @@
#!/bin/sh -e

# This is documented here:
# https://docs.openshift.com/container-platform/3.11/creating_images/guidelines.html#openshift-specific-guidelines

if ! whoami &>/dev/null; then
if [ -w /etc/passwd ]; then
echo "${USER_NAME:-keda}:x:$(id -u):$(id -g):${USER_NAME:-keda} user:${HOME}:/sbin/nologin" >> /etc/passwd
fi
fi
exec ${OPERATOR} $@
1 change: 1 addition & 0 deletions build/bin/user_setup
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
set -x

# ensure $HOME exists and is accessible by group 0 (we don't know what the runtime UID will be)
echo "${USER_NAME}:x:${USER_UID}:0:${USER_NAME} user:${HOME}:/sbin/nologin" >> /etc/passwd
mkdir -p ${HOME}
chown ${USER_UID}:0 ${HOME}
chmod ug+rwx ${HOME}
Expand Down
13 changes: 6 additions & 7 deletions cmd/adapter/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,17 @@ import (
"os"
"runtime"

"github.com/kedacore/keda/pkg/handler"
kedav1alpha1 "github.com/kedacore/keda/pkg/apis/keda/v1alpha1"
kedaprovider "github.com/kedacore/keda/pkg/provider"
"github.com/kedacore/keda/pkg/scaling"
"github.com/kedacore/keda/version"

"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/klog"
"k8s.io/klog/klogr"

kedav1alpha1 "github.com/kedacore/keda/pkg/apis/keda/v1alpha1"
"github.com/operator-framework/operator-sdk/pkg/k8sutil"
appsv1 "k8s.io/api/apps/v1"
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/client-go/kubernetes/scheme"
"k8s.io/klog"
"k8s.io/klog/klogr"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/config"

Expand Down Expand Up @@ -62,7 +61,7 @@ func (a *Adapter) makeProviderOrDie() provider.MetricsProvider {
os.Exit(1)
}

handler := handler.NewScaleHandler(kubeclient, scheme)
handler := scaling.NewScaleHandler(kubeclient, nil, scheme)

namespace, err := k8sutil.GetWatchNamespace()
if err != nil {
Expand Down
Loading