Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add checkpointStrategy parameter to event-hub #429

Merged
merged 2 commits into from
May 5, 2021
Merged

add checkpointStrategy parameter to event-hub #429

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
43157e2
add checkpointStrategy parameter to event-hub
christle Apr 30, 2021
e4682fc
add legacy checkpointing description to checkpointStrategy parameter …
christle May 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions content/docs/2.3/scalers/azure-event-hub.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ layout = "scaler"
availability = "v1.0+"
maintainer = "Community"
description = "Scale applications based on Azure Event Hubs."
notice = "As of now, the Event Hub scaler only supports reading checkpoints from Blob Storage, as well as scaling only Event Hub applications written in C#, Java, Python or created with Azure Functions."
go_file = "azure_eventhub_scaler"
+++

Expand All @@ -27,9 +26,18 @@ triggers:

tomkerkhove marked this conversation as resolved.
Show resolved Hide resolved
- `connectionFromEnv` - Name of the environment variable your deployment uses to get the connection string appended with `EntityPath=<event_hub_name>`
- `storageConnectionFromEnv` - Name of the environment variable that provides connection string for Azure Storage Account to store checkpoint. As of now the Event Hub scaler only reads from Azure Blob Storage.
- `consumerGroup` - Consumer group of event hub consumer. (default: `$default`)
- `consumerGroup` - Consumer group of Azure Event Hub consumer. (default: `$default`)
- `unprocessedEventThreshold` - : Average target value to trigger scaling actions. (default: 64)
- `blobContainer` - Container name to store checkpoint. This is needed when a using an Event Hub application written in dotnet or java, and not an Azure function
- `blobContainer` - Container name to store checkpoint. This is needed for every `checkpointStrategy` except of `AzureFunction`. With Azure Functions the `blobContainer` is autogenerated and cannot be overridden.
- `checkpointStrategy` - configure the checkpoint behaviour of different Event Hub SDKs. (default: `""`)
- `azureFunction`: Suitable for Azure Functions & Azure WebJobs SDK. This is the default setting, when `blobcontainer` is not specified.
- `blobMetadata`: For all implementations that store checkpoint informations on blob metadata such as current C#, Python, Java and JavaScript Event Hub SDKs.
- `goSdk` For all implementations using the [Golang SDK](https://github.com/Azure/azure-event-hubs-go)'s checkpointing, for example Dapr
- When no checkpoint strategy is specified, the Event Hub scaler will use backwards compatibility and able to scale older implementations of C#, Python or Java Event Hub SDKs. (see "Legacy checkpointing"). If this behaviour should be used, `blobContainer` is also required.

> 💡 **Legacy Checkpointing:** C# applications, which use the `Microsoft.Azure.EventHubs` package, Java applications which use the `azure-eventhubs-eph` package or Python applications which use the `azure-eventhub` below `v5` are supported if no `checkpointStrategy` is specified. These legacy implementations are based on the `EventProcessorHost` client, which stores checkpoint information as blob container content
>
> C# Applications that use the current `Azure.Messaging.EventHubs` package, Java applications which use the current `azure-messaging-eventhubs` package or Python applications since `azure-eventhub v5`, have to set the `checkpointStrategy`to `blobMetadata`. All these implementations based on the `EventProcessorClient` which stores checkpoint information as blob metadata.

### Authentication Parameters

Expand Down