Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -61,4 +61,8 @@ Thumbs.db

# reduced pom files should not be included
dependency-reduced-pom.xml
package-lock.json
package-lock.json

# VS Code
.vscode/
.factorypath
294 changes: 294 additions & 0 deletions sdk/eventhubs/azure-messaging-eventhubs/migration-guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,294 @@
# Migration Guide (Event Hubs v3 to v5)

This document is intended for users that are familiar with v3 of the Java SDK for Event Hubs library
([`azure-eventhubs`][azure-eventhubs] and [`azure-eventhubs-eph`][azure-eventhubs-eph]) and wish to migrate their
application to V5 of the same library.

For users new to the Java SDK for Event Hubs, please see the [README for azure-messaging-eventhubs][README].

## Table of contents

- [Prerequisites](#prerequisites)
- [Updated Maven dependencies](#updated-maven-dependencies)
- [General changes](#general-changes)
- [Converting core classes](#converting-core-classes)
- [Sending events](#sending-events)
- [Receiving events](#receiving-events)
- [Minor renames](#minor-renames)
- [Migration samples](#migration-samples)
- [Migrating code from `PartitionSender` or `EventHubClient` to `EventHubProducerAsyncClient` for sending events](#migrating-code-from-partitionsender-or-eventhubclient-to-eventhubproducerasyncclient-for-sending-events)
- [Migrating code from `PartitionReceiver` to `EventHubConsumerAsyncClient` for receiving events in batches](#migrating-code-from-partitionreceiver-to-eventhubconsumerasyncclient-for-receiving-events-in-batches)
- [Migrating code from `EventProcessorHost` to `EventProcessorClient` for receiving events](#migrating-code-from-eventprocessorhost-to-eventprocessorclient-for-receiving-events)

## Prerequisites
Java Development Kit (JDK) with version 8 or above

## Updated Maven dependencies

Dependencies for Event Hubs has been updated to:
```xml
<dependencies>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-eventhubs</artifactId>
<version>5.0.0-beta.6</version>
</dependency>

<!-- Contains Azure Blob Storage checkpoint store when using EventProcessorClient -->
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-eventhubs-checkpointstore-blob</artifactId>
<version>1.0.0-beta.4</version>
</dependency>
</dependencies>
```

## General changes

In the interest of simplifying the API surface, we've made three clients, each with an asynchronous and synchronous
variant. One client for sending events, `EventHubProducerAsyncClient`, and two for receiving events.
Comment thread
conniey marked this conversation as resolved.
Outdated
`EventProcessorClient` is the production level consumer and `EventHubConsumerAsyncClient` for exploration and
Comment thread
conniey marked this conversation as resolved.
Outdated
lower-level control of `EventData` consumption.

[EventProcessorClient][EventProcessorClient] supports checkpointing and load balancing through a plugin model.
Comment thread
conniey marked this conversation as resolved.
Outdated
Currently, only Azure Blob storage is supported through
Comment thread
conniey marked this conversation as resolved.
Outdated
[azure-messaging-eventhubs-checkpointstore-blob][azure-messaging-eventhubs-checkpointstore-blob], but support for other
durable storage (i.e. Cosmos DB, Redis) can be added in the future.
Comment thread
conniey marked this conversation as resolved.
Outdated

| Operation | Asynchronous client | Synchronous client |
|---|---|---|
| Producing events | [EventHubProducerAsyncClient][EventHubProducerAsyncClient] | [EventHubProducerClient][EventHubProducerClient] |
| Consuming events (supports checkpointing and load balancing) | [EventProcessorClient][EventProcessorClient] | |
| Consuming events | [EventHubConsumerAsyncClient][EventHubConsumerAsyncClient] | [EventHubConsumerClient][EventHubConsumerClient] |

### Converting core classes

Creation of producers or consumers is done through either [EventHubClientBuilder][EventHubClientBuilder] or
[EventProcessorClientBuilder][EventProcessorClientBuilder]. Asynchronous clients are created by invoking
`builder.build*AsyncClient()`. Synchronous clients are created by invoking `builder.build*Client()`.

| In v3 | Equivalent in v5 | Sample |
|---|---|---|
| `EventHubClient.createFromConnectionString()` | `var builder = new EventHubClientBuilder().connectionString();`<br/>then either `builder.buildProducerAsyncClient();` or<br/>`builder.consumerGroup().buildConsumerAsyncClient();` | [Publishing events][PublishEventsWithCustomMetadata], [Consuming events][ConsumeEvents] |
| `EventHubClient.createWithAzureActiveDirectory()` | `var builder = new EventHubClientBuilder().tokenCredential();`<br/>then either `builder.buildProducerAsyncClient();` or<br/>`builder.consumerGroup().buildConsumerAsyncClient();` | [Publishing events with Azure AD][PublishEventsWithAzureIdentity] |
| `EventProcessorHost.EventProcessorHostBuilder.newBuilder()` | `new EventProcessorClientBuilder().buildEventProcessorClient()` | [EventProcessorClient with Blob storage][EventProcessorClientInstantiation] |

### Sending events

`EventHubProducerAsyncClient` and `EventHubProducerClient` can publish events to a single partition or allow the service
to load balance events between all the partitions. The behaviour is determined when using
[`CreateBatchOptions`][CreateBatchOptions] in `producer.createBatch(CreateBatchOptions)`.

| In v3 | Equivalent in v5 | Sample |
|---|---|---|
| `PartitionSender.send(...)` | `EventHubProducerAsyncClient.send()` | [Publishing events to a specific partition][PublishEventsToSpecificPartition] |
| `EventHubClient.send(...)` | `EventHubProducerAsyncClient.send()` | [Publishing events][PublishEventsWithCustomMetadata] |

### Receiving events

| In v3 | Equivalent in v5 | Sample |
|---|---|---|
| `PartitionReceiver.receive()` | `EventHubConsumerAsyncClient.receiveFromPartition()` | [Consuming events][ConsumeEvents] |
| `PartitionReceiver.setReceiveHandler()` | `EventHubConsumerAsyncClient.receiveFromPartition()` | [Consuming events][ConsumeEvents] |

### Minor renames

| In v3 | Equivalent in v5 |
|---|---|
| `EventPosition.fromStartOfStream()` | `EventPosition.earliest()` |
| `EventPosition.fromEndOfStream()` | `EventPosition.latest()` |

## Migration samples

* [Sending events](#migrating-from-partitionsender-or-eventhubclient-to-eventhubproducerasyncclient-for-sending-events)
* [Receiving events](#migrating-code-from-partitionreceiver-to-eventhubconsumerclient-for-receiving-events)
* [Receiving events with checkpointing](#migrating-code-from-eventprocessorhost-to-eventhubconsumerclient-for-receiving-events)

### Migrating code from `PartitionSender` or `EventHubClient` to `EventHubProducerAsyncClient` for sending events
In v3, there were multiple options on how to publish events to an Event Hub.

In v5, this has been consolidated into a more efficient `send(EventDataBatch)` method. Batching merges information from
multiple events into a single sent message, reducing the amount of network communication needed vs sending events one at
a time.

The code below assumes all events fit into a single batch. For a more complete example, see sample: [Publishing events
to specific partition][PublishEventsToSpecificPartition].

So in v3:
```java
EventHubClient client = EventHubClient.createFromConnectionStringSync("connection-string-for-an-event-hub",
Executors.newScheduledThreadPool(4));

List<EventData> events = Arrays.asList(EventData.create("foo".getBytes()), EventData.create("bar".getBytes()));

CompletableFuture<Void> sendFuture = client.createPartitionSender("my-partition-id")
.thenCompose(producer -> {
EventDataBatch batch = producer.createBatch();
for (EventData event : events) {
try {
// Assuming all events fit into a single batch. This returns false if it does not.
// If that is the case, we'll send the full batch then create another one to continue adding events to.
batch.tryAdd(event);
} catch (PayloadSizeExceededException e) {
System.err.println("Event is too large for batch. Exception: " + e);
}
}

return producer.send(batch);
});

sendFuture.get();
```

In v5:
```java
List<EventData> events = Arrays.asList(EventData.create("foo".getBytes()), EventData.create("bar".getBytes()));

EventHubProducerAsyncClient producer = new EventHubClientBuilder()
.connectionString("connection-string-for-an-event-hub")
.buildAsyncProducerClient();

CreateBatchOptions options = new CreateBatchOptions()
.setPartitionId("my-partition-id");

Mono<Void> sendOperation = producer.createBatch(options).flatMap(batch -> {
for (EventData event : data) {
// Assuming all events fit into a single batch. This returns false if it does not.
// If that is the case, we'll send the full batch then create another one to continue adding events to.
batch.tryAdd(event);
Comment thread
conniey marked this conversation as resolved.
}
return producer.send(batch);
});

sendOperation.block();
```

### Migrating code from `PartitionReceiver` to `EventHubConsumerAsyncClient` for receiving events in batches

In v3, events were received by creating a `PartitionReceiver` and invoking `receive(int)` multiple times to receive
events up to a certain number.

In v5, [project Reactor][project-reactor] is used, so events can be streamed as they come in without having to use a
batched receive approach.

This code which receives from a partition in v3:

```java
EventHubClient client = EventHubClient.createFromConnectionStringSync("connection-string-for-an-event-hub",
Executors.newScheduledThreadPool(5));
PartitionReceiver consumer = client.createReceiverSync("my-consumer-group", "my-partition-id",
EventPosition.fromStartOfStream());

// Gets 100 events or until the receive timeout elapses.
consumer.receive(100).thenAccept(events -> {
for (EventData event : events) {
System.out.println("Sequence number: " + event.getSystemProperties().getSequenceNumber());
System.out.println("Contents: " + new String(event.getBytes(), StandardCharsets.UTF_8));
}
}).get();

// Gets the next 50 events or until the receive timeout elapses.
consumer.receive(50).thenAccept(events -> {
for (EventData event : events) {
System.out.println("Sequence number: " + event.getSystemProperties().getSequenceNumber());
System.out.println("Contents: " + new String(event.getBytes(), StandardCharsets.UTF_8));
}
}).get();
```

Becomes this in v5:
```java
EventHubConsumerAsyncClient consumer = new EventHubClientBuilder()
.connectionString("connection-string-for-an-event-hub")
.consumerGroup("my-consumer-group")
.buildAsyncConsumerClient();

// This is a non-blocking call. It'll subscribe and return a Disposable. This will stream events as they come
// in, starting from the beginning of the partition.
Disposable subscription = consumer.receiveFromPartition("my-partition-id", EventPosition.earliest())
.subscribe(partitionEvent -> {
EventData event = partitionEvent.getData();
System.out.println("Sequence number: " + event.getSequenceNumber());
System.out.println("Contents: " + new String(event.getBody(), StandardCharsets.UTF_8));
});

// Keep fetching events
// When you are finished, dispose of the subscription.
subscription.dispose();
```

See [`ConsumeEvents.java`][ConsumeEvents] for a sample program demonstrating this.

### Migrating code from `EventProcessorHost` to `EventProcessorClient` for receiving events

In v3, `EventProcessorHost` allowed you to balance the load between multiple instances of your program and checkpoint
events when receiving.

In v5, `EventProcessorClient` allows you to do the same and includes a plugin model, so other durable stores can be used
if desired.

The following code in v3:
```java
private static void main(String[] args) throws Exception {
EventProcessorHost processor = EventProcessorHost.EventProcessorHostBuilder.newBuilder("a-processor-name", "my-consumer-group")
.useAzureStorageCheckpointLeaseManager("storage-connection-string", "storage-container-name", "prefix")
.useEventHubConnectionString("connection-string-for-an-event-hub")
.build();

processor.registerEventProcessor(MyEventProcessor.class).get();

// When you are finished processing events.
processor.unregisterEventProcessor();
}

class MyEventProcessor implements IEventProcessor {
@Override
public void onOpen(PartitionContext context) {
System.out.println("Started receiving on partition: " + context.getPartitionId());
}

@Override
public void onClose(PartitionContext context, CloseReason reason) {
System.out.printf("Stopped receiving on partition: %s. Reason: %s%n", context.getPartitionId(), reason);
}

@Override
public void onEvents(PartitionContext context, Iterable<EventData> events) {
System.out.println("Received events from partition: %s." + context.getPartitionId());
for (EventData event : events) {
System.out.println("Sequence number: " + event.getSystemProperties().getSequenceNumber());
System.out.println("Contents: " + new String(event.getBytes(), StandardCharsets.UTF_8));
}
}

@Override
public void onError(PartitionContext context, Throwable error) {
System.err.printf("Error occurred on partition: %s. Error: %s%n", context.getPartitionId(), error);
}
}
```

And in v5:
```java
// TODO
```

<!-- Links -->
[azure-eventhubs-eph]: https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-eph
[azure-eventhubs]: https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs
[azure-messaging-eventhubs-checkpointstore-blob]: https://search.maven.org/artifact/com.azure/azure-messaging-eventhubs-checkpointstore-blob
[ConsumeEvents]: src/samples/java/com/azure/messaging/eventhubs/ConsumeEvents.java
[CreateBatchOptions]: src/main/java/com/azure/messaging/eventhubs/models/CreateBatchOptions.java
[EventHubClientBuilder]: src/main/java/com/azure/messaging/eventhubs/EventHubClientBuilder.java
[EventHubConsumerAsyncClient]: src/main/java/com/azure/messaging/eventhubs/EventHubConsumerAsyncClient.java
[EventHubConsumerClient]: src/main/java/com/azure/messaging/eventhubs/EventHubConsumerClient.java
[EventHubProducerAsyncClient]: src/main/java/com/azure/messaging/eventhubs/EventHubProducerAsyncClient.java
[EventHubProducerClient]: src/main/java/com/azure/messaging/eventhubs/EventHubProducerClient.java
[EventProcessorClient]: src/main/java/com/azure/messaging/eventhubs/EventProcessorClient.java
[EventProcessorClientBuilder]: src/main/java/com/azure/messaging/eventhubs/EventProcessorClientBuilder.java
[EventProcessorClientInstantiation]: ../azure-messaging-eventhubs-checkpointstore-blob/src/samples/java/com/azure/messaging/eventhubs/checkpointstore/blob/BlobCheckpointStoreSample.java
[PublishEventsToSpecificPartition]: src/samples/java/com/azure/messaging/eventhubs/PublishEventsToSpecificPartition.java
[PublishEventsWithAzureIdentity]: src/samples/java/com/azure/messaging/eventhubs/PublishEventsWithAzureIdentity.java
[PublishEventsWithCustomMetadata]: src/samples/java/com/azure/messaging/eventhubs/PublishEventsWithCustomMetadata.java
[README]: README.md
[project-reactor]: https://projectreactor.io/
2 changes: 1 addition & 1 deletion sdk/storage/azure-storage-blob/migrationGuides/V8_V12.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ For more info of the motivation behind this major change, please refer to [this
Java Development Kit (JDK) with version 8 or above

## Converting Core Classes
Our core synchronous classes have been replaced, as well as new a~~~~synchronous counterparts added.
Our core synchronous classes have been replaced, as well as new asynchronous counterparts added.
Comment thread
conniey marked this conversation as resolved.
Outdated

| Core V8 classes | Equivalent V12 Class | NEW Asynchronous clients |
|---:|---:|---:|
Expand Down