Merge latest changes from Azure/azure-event-hubs#444 (#3511)

* Update Apache Proton-J dependency (0.29.0 --> 0.31.0) (#407)

* PartitionReceiver - add a method that provides an EventPosition which corresponds to an EventData returned last by the receiver (#408)

* Support IsPartitionEmpty property for PartitionRuntimeInformation (#399)

* Move setPrefetchCount API to the ReceiverOptions class from the PartitionReceiver and update the settings of Default & Max Prefetch count (#410)

This pull request includes two major changes related to Prefetch API.

1) Move setPrefetchCount API to the ReceiverOptions class so that prefetch value specified by a user can be used instead of using default value when communicating to the service during link open and initializing a receiver. This change also addresses the receiver stuck issue caused by setPrefetchAPI in a race condition.

2) Change the default value and set the upper bound of the prefetch count. Note that prefetch count should be greater than or equal to maxEventCount which can be set when either a) calling receive() API or b) implementing the getMaxEventCount API of the SessionReceiverHandler interface.

* Fixes several issues in the reactor related components (#411)

This pull request contains the following changes.

1) Finish pending tasks when recreating the reactor and make sure pending calls scheduled on the old reactor get complete.
2) Fix the session open timeout issue which can result in NPE in proton-J engine.
3) Make session open timeout configurable and use the value of OperationTimeout.
4) Update the message of exceptions and include an entity name in the exception message.
5) API change - use ScheduledExecutorService.
6) Improve tracing.

* Implement comparable on EventData (#395)

* Update receive/send link creation logic and improve tracing (#414)

* Prep for releasing client 2.0.0 and EPH 2.2.0 (#415)

* Ensure that links are closed when transport error occurrs (#417)

* ensure links are recreated on transport/connection failure
* update API document for EventProcessorOptions class
* add traces for link create/close case

* Prep for releasing client 2.1.0 and EPH 2.3.0 (#418)

* Update prefetch sendflow logic and increment version for new release (#420)

* Fix args for proxy auth call to Authenticator (#421)

* Prepare EPH 2.3.4 release (#423)

* Prepare EPH 2.4.0 release (#423) (#424)

* Handle proton:io errors with meaningful error msg (#427)

* Handle proton:io errors with meaningful error msg

* Use Proton-supplied message if present

* Minor changes to lease scanner (#428)

* Add logging if the scanner threw an exception.
* Change logging level to warn when scanner shuts down for any reason.
* Scanner can call EventProcessorOptions.notifyOfException, which calls user code. Change notifyOfException to defensively catch any exceptions coming out of user code.

* Make EventData.SystemProperties completely public (#435)

Porting testability changes from .NET Core to Java: provide full access to EventData's SystemProperties so that a complete EventData can be fabricated in tests.

* Digest Support: init first connection with null headers (#431)

Related to Azure/qpid-proton-j-extensions#10

* Fix lease scanner issues when Storage unreachable (#434)

This fix is for issue #432. There are two parts:

AzureStorageCheckpointLeaseManager performs certain Storage actions within a forEach. If those actions fail, the StorageException gets wrapped in a NoSuchElementException. Catch those and strip off the NoSuchElementException, then handle the StorageException in the existing way.

The unexpected NoSuchElementExceptions were not being caught anywhere and the scanner thread was dying without rescheduling itself. Added code in PartitionMananger.scan to catch any exceptions that leak out of PartitionScanner and reschedule the scanner unless the host instance is shutting down.

* message receiver - fix null pointer error and ensure that receive link is recreated upon a failure (#439)

* message receiver/sender - fix null pointer error and ensure that receive/send link is recreated on a failure.

* Update version numbers for release (#440)

* Update prefetch count for a receiver (#441)

* Fix an issue of creating multiple sessions for $management & $cbs channel for a single connection and improve logging (#443)

* Fix an issue of creating multiple sessions for $management & $cbs for a connection and improve logging

* Update version numbers for new release (#444)

* Update spotbugs.xml report versions

Author: conniey

eng/spotbugs-aggregate-report/pom.xml

@@ -16,8 +16,8 @@
 
     <properties>
         <azure-batch.version>5.0.1</azure-batch.version>
-        <azure-eventhubs.version>2.3.0</azure-eventhubs.version>
-        <azure-eventhubs-eph.version>2.5.0</azure-eventhubs-eph.version>
+        <azure-eventhubs.version>2.3.1</azure-eventhubs.version>
+        <azure-eventhubs-eph.version>2.5.1</azure-eventhubs-eph.version>
         <azure-keyvault.version>1.2.0</azure-keyvault.version>
         <azure-servicebus.version>2.0.0</azure-servicebus.version>
         <azure-storage-blob.version>10.5.0</azure-storage-blob.version>

eventhubs/data-plane/ConsumingEvents.md

@@ -1,33 +1,33 @@
-# Publishing Events with the Java client for Azure Event Hubs 
+# Publishing Events with the Java client for Azure Event Hubs
 
-The vast majority of Event Hub applications using this and the other client libraries are and will be event publishers. 
+The vast majority of Event Hub applications using this and the other client libraries are and will be event publishers.
 And for most of these publishers, publishing events is extremely simple and handled with just a few API gestures.
 
 ## Getting Started
 
 This library is available for use in Maven projects from the Maven Central Repository, and can be referenced using the
-following dependency declaration inside of your Maven project file:    
+following dependency declaration inside of your Maven project file:
 
 ```XML
     <dependency>
-        <groupId>com.microsoft.azure</groupId> 
+        <groupId>com.microsoft.azure</groupId>
         <artifactId>azure-eventhubs</artifactId>
-        <version>2.3.0</version>
+        <version>2.3.1</version>
     </dependency>
  ```
- 
- For different types of build environments, the latest released JAR files can also be [explicitly obtained from the 
- Maven Central Repository](https://search.maven.org/#search%7Cga%7C1%7Ca%3A%22azure-eventhubs%22) or from [the Release distribution point on GitHub](https://github.com/Azure/azure-event-hubs/releases).  
+
+ For different types of build environments, the latest released JAR files can also be [explicitly obtained from the
+ Maven Central Repository](https://search.maven.org/#search%7Cga%7C1%7Ca%3A%22azure-eventhubs%22) or from [the Release distribution point on GitHub](https://github.com/Azure/azure-event-hubs/releases).
+
+
+For a simple event publisher, you'll need to import the *com.microsoft.azure.eventhubs* package for the Event Hub client classes.
 
 
-For a simple event publisher, you'll need to import the *com.microsoft.azure.eventhubs* package for the Event Hub client classes. 
- 
- 
 ```Java
     import com.microsoft.azure.eventhubs.*;
 ```
 
-Event Hubs client library uses qpid proton reactor framework which exposes AMQP connection and message delivery related 
+Event Hubs client library uses qpid proton reactor framework which exposes AMQP connection and message delivery related
 state transitions as reactive events. In the process,
 the library will need to run many asynchronous tasks while sending and receiving messages to Event Hubs.
 So, `EventHubClient` requires an instance of `Executor`, where all these tasks are run.
@@ -37,63 +37,63 @@ So, `EventHubClient` requires an instance of `Executor`, where all these tasks a
     ScheduledExecutorService executor = Executors.newScheduledThreadPool(8)
 ```
 
-Using an Event Hub connection string, which holds all required connection information including an authorization key or token 
-(see [Connection Strings](#connection-strings)), you then create an *EventHubClient* instance.   
-   
+Using an Event Hub connection string, which holds all required connection information including an authorization key or token
+(see [Connection Strings](#connection-strings)), you then create an *EventHubClient* instance.
+
 ```Java
     ConnectionStringBuilder connStr = new ConnectionStringBuilder()
                 .setNamespaceName("----ServiceBusNamespaceName-----")
                 .setEventHubName("----EventHubName-----")
                 .setSasKeyName("-----SharedAccessSignatureKeyName-----")
-                .setSasKey("---SharedAccessSignatureKey----");	
-	
+                .setSasKey("---SharedAccessSignatureKey----");
+
     EventHubClient ehClient = EventHubClient.createSync(connStr.toString(), executor);
 ```
 
-Once you have the client in hands, you can package any arbitrary payload as a plain array of bytes and send it. The samples 
-we use to illustrate the functionality send a UTF-8 encoded JSON data, but you can transfer any format you wish. 
+Once you have the client in hands, you can package any arbitrary payload as a plain array of bytes and send it. The samples
+we use to illustrate the functionality send a UTF-8 encoded JSON data, but you can transfer any format you wish.
 
 ```Java
     EventData sendEvent = EventData.create(payloadBytes);
     ehClient.sendSync(sendEvent);
 ```
-         
-The entire client API is built for Java 8's concurrent task model, generally returning 
+
+The entire client API is built for Java 8's concurrent task model, generally returning
 [*CompletableFuture<T>*](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html), so the library has these methods suffixed with *Sync* as their Synchronous counterparts/varaints.
 
 ## AMQP 1.0
 Azure Event Hubs allows for publishing events using the HTTPS and AMQP 1.0 protocols. The Azure Event Hub endpoints
-also support AMQP over the WebSocket protocol, allowing event traffic to leverage the same outbound TCP port as 
-HTTPS. 
+also support AMQP over the WebSocket protocol, allowing event traffic to leverage the same outbound TCP port as
+HTTPS.
 
-This client library is built on top of the [Apache Qpid Proton-J]() libraries and supports AMQP, which is significantly 
-more efficient at publishing event streams than HTTPS. AMQP 1.0 is an international standard published as ISO/IEC 19464:2014.  
+This client library is built on top of the [Apache Qpid Proton-J]() libraries and supports AMQP, which is significantly
+more efficient at publishing event streams than HTTPS. AMQP 1.0 is an international standard published as ISO/IEC 19464:2014.
 
-AMQP is session-oriented and sets up the required addressing information and authorization information just once for each 
-send link, while HTTPS requires doing so with each sent message. AMQP also has a compact binary format to express common 
-event properties, while HTTPS requires passing message metadata in a verbose text format. AMQP can also keep a significant 
-number of events "in flight" with asynchronous and robust acknowledgement flow, while HTTPS enforces a strict request-reply 
+AMQP is session-oriented and sets up the required addressing information and authorization information just once for each
+send link, while HTTPS requires doing so with each sent message. AMQP also has a compact binary format to express common
+event properties, while HTTPS requires passing message metadata in a verbose text format. AMQP can also keep a significant
+number of events "in flight" with asynchronous and robust acknowledgement flow, while HTTPS enforces a strict request-reply
 pattern.
 
-AMQP 1.0 is a TCP based protocol. For Azure Event Hubs, all traffic *must* be protected using TLS (SSL) and is using 
-TCP port 5671.  
+AMQP 1.0 is a TCP based protocol. For Azure Event Hubs, all traffic *must* be protected using TLS (SSL) and is using
+TCP port 5671.
 
 This library will provide HTTPS support via WebSockets when Proton-J supports HTTPS.
 
 ## Connection Strings
 
 Azure Event Hubs and Azure Service Bus share a common format for connection strings. A connection string holds all required
-information to set up a connection with an Event Hub. The format is a simple property/value list of the form 
-{property}={value} with pairs separated by ampersands (&). 
+information to set up a connection with an Event Hub. The format is a simple property/value list of the form
+{property}={value} with pairs separated by ampersands (&).
 
 | Property              |  Description                                               |
-|-----------------------|------------------------------------------------------------| 
+|-----------------------|------------------------------------------------------------|
 | Endpoint              | URI for the Event Hubs namespace. Typically has the form *sb://{namespace}.servicebus.windows.net/*   |
-| EntityPath            | Relative path of the Event Hub in the namespace. Commonly this is just the Event Hub name                   |  
+| EntityPath            | Relative path of the Event Hub in the namespace. Commonly this is just the Event Hub name                   |
 | SharedAccessKeyName   | Name of a Shared Access Signature rule configured for the Event Hub or the Event Hub name. For publishers, the rule must include "Send" permissions. |
 | SharedAccessKey       | Base64-encoded value of the Shared Access Key for the rule |
 | SharedAccessSignature | A previously issued Shared Access Signature token  (not yet supported; will be soon)        |
- 
+
 A connection string will therefore have the following form:
 
 ```
@@ -102,42 +102,42 @@ A connection string will therefore have the following form:
 
 ## Advanced Operations
 
-The publisher example shown in the overview above sends an event into the Event Hub without further qualification. This is 
-the preferred and most flexible and reliable option. For specific needs, Event Hubs offers two extra options to 
-qualify send operations: Publisher policies and partion addressing.     
+The publisher example shown in the overview above sends an event into the Event Hub without further qualification. This is
+the preferred and most flexible and reliable option. For specific needs, Event Hubs offers two extra options to
+qualify send operations: Publisher policies and partion addressing.
 
 ### Partition Addressing
 
-Any Event Hub's event store is split up into at least 4 partitions, each maintaining a separate event log. You can think 
-of partitions like lanes on a highway. The more events the Event Hub needs to handle, the more lanes (partitions) you have 
-to add. Each partition can handle at most the equivalent of 1 "throughput unit", equivalent to at most 1000 events per 
+Any Event Hub's event store is split up into at least 4 partitions, each maintaining a separate event log. You can think
+of partitions like lanes on a highway. The more events the Event Hub needs to handle, the more lanes (partitions) you have
+to add. Each partition can handle at most the equivalent of 1 "throughput unit", equivalent to at most 1000 events per
 second and at most 1 Megabyte per second.
 
 In some cases, publisher applications need to address partitions directly in order to pre-categorize events for consumption.
-A partition is directly addressed either by using the partition's identifier or by using some string (partition key) that gets 
+A partition is directly addressed either by using the partition's identifier or by using some string (partition key) that gets
 consistently hashed to a particular partition.
 
-This capability, paired with a large number of partitions, may appear attractive for implementing a fine grained, per publisher 
+This capability, paired with a large number of partitions, may appear attractive for implementing a fine grained, per publisher
 subscription scheme similar to what Topics offer in Service Bus Messaging - but it's not at all how the capability should be used
-and it's likely not going to yield satisfying results. 
- 
-Partition addressing is designed as a routing capability that consistently assigns events from the same sources to the same partition allowing 
-downstream consumer systems to be optimized, but under the assumption of very many of such sources (hundreds, thousands) share 
-the same partition. If you need fine-grained content-based routing, Service Bus Topics might be the better option. 
+and it's likely not going to yield satisfying results.
+
+Partition addressing is designed as a routing capability that consistently assigns events from the same sources to the same partition allowing
+downstream consumer systems to be optimized, but under the assumption of very many of such sources (hundreds, thousands) share
+the same partition. If you need fine-grained content-based routing, Service Bus Topics might be the better option.
 
 #### Using Partition Keys
 
 Of the two addressing options, the preferable one is to let the hash algorithm map the event to the appropriate partition.
-The gesture is a straightforward extra override to the send operation supplying the partition key: 
+The gesture is a straightforward extra override to the send operation supplying the partition key:
 
 ```Java
     EventData sendEvent = EventData.create(payloadBytes);
 >   ehClient.sendSync(sendEvent, partitionKey);
 ```
-     
+
 #### Using Partition Ids
 
-If you indeed need to target a specific partition, for instance because you must use a particular distribution strategy, 
+If you indeed need to target a specific partition, for instance because you must use a particular distribution strategy,
 you can send directly to the partition, but doing so requires an extra gesture so that you don't accidentally choose this
 option. To send to a partition you explicitly need to create a client object that is tied to the partition as shown below:
 
@@ -151,24 +151,24 @@ option. To send to a partition you explicitly need to create a client object tha
 #### Publisher Policies
 
 Event Hub Publisher Policies are not yet supported by this client and will be supported in a future release.
- 
+
 #### Special considerations for partitions and publisher policies
 
-Using partitions or publisher policies (which are effectively a special kind of partition key) may impact throughput 
-and availability of your Event Hub solution. 
+Using partitions or publisher policies (which are effectively a special kind of partition key) may impact throughput
+and availability of your Event Hub solution.
 
-When you do a regular send operation that does not prescribe a particular partition, the Event Hub will choose a 
-partition at random, ensuring about equal distribution of events across partitions. Sticking with the above analogy, 
-all highway lanes get the same traffic. 
+When you do a regular send operation that does not prescribe a particular partition, the Event Hub will choose a
+partition at random, ensuring about equal distribution of events across partitions. Sticking with the above analogy,
+all highway lanes get the same traffic.
 
-If you explicitly choose the partition key or partition-id, it's up to you to take care that traffic is evenly 
-distributed, otherwise you may end up with a traffic jam (in the form of throttling) on one partition while there's 
-little or no traffic on another partition. 
+If you explicitly choose the partition key or partition-id, it's up to you to take care that traffic is evenly
+distributed, otherwise you may end up with a traffic jam (in the form of throttling) on one partition while there's
+little or no traffic on another partition.
 
-Also, like every other aspect of distributed systems, the log storage backing any partition may rarely and briefly slow 
+Also, like every other aspect of distributed systems, the log storage backing any partition may rarely and briefly slow
 down or experience congestion. If you leave choosing the target partition for an event to Event Hubs, it can flexibly
-react to such availability blips for publishers.        
+react to such availability blips for publishers.
 
-Generally, you should *not* use partitioning as a traffic prioritization scheme, and you should *not* use it 
-for fine grained assignment of particular kinds of events to a particular partitions. *Partitions are a load 
+Generally, you should *not* use partitioning as a traffic prioritization scheme, and you should *not* use it
+for fine grained assignment of particular kinds of events to a particular partitions. *Partitions are a load
 distribution mechanism, not a filtering model*.

eventhubs/data-plane/PublishingEvents.md

@@ -7,14 +7,14 @@
   <parent>
     <groupId>com.microsoft.azure</groupId>
     <artifactId>azure-eventhubs-clients</artifactId>
-    <version>2.3.0</version>
+    <version>2.3.1</version>
     <relativePath>../pom.xml</relativePath>
   </parent>
 
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.microsoft.azure</groupId>
   <artifactId>azure-eventhubs-eph</artifactId>
-  <version>2.5.0</version>
+  <version>2.5.1</version>
 
   <name>Microsoft Azure SDK for Event Hubs Event Processor Host(EPH)</name>
   <description>EPH is built on top of the Azure Event Hubs Client and provides a number of features not present in that lower layer</description>

eventhubs/data-plane/azure-eventhubs-eph/pom.xml

@@ -7,7 +7,7 @@
   <parent>
     <groupId>com.microsoft.azure</groupId>
     <artifactId>azure-eventhubs-clients</artifactId>
-    <version>2.3.0</version>
+    <version>2.3.1</version>
     <relativePath>../pom.xml</relativePath>
   </parent>
 

eventhubs/data-plane/azure-eventhubs-extensions/pom.xml

@@ -7,7 +7,7 @@
   <parent>
     <groupId>com.microsoft.azure</groupId>
     <artifactId>azure-eventhubs-clients</artifactId>
-    <version>2.3.0</version>
+    <version>2.3.1</version>
     <relativePath>../pom.xml</relativePath>
   </parent>
 

eventhubs/data-plane/azure-eventhubs/pom.xml

@@ -406,7 +406,7 @@ private void parseConnectionString(final String connectionString) {
                     this.transportType = TransportType.fromString(values[valueIndex]);
                 } catch (IllegalArgumentException exception) {
                     throw new IllegalConnectionStringFormatException(
-                            String.format("Invalid value specified for property '%s' in the ConnectionString.", TRANSPORT_TYPE_CONFIG_NAME),
+                            String.format(Locale.US, "Invalid value specified for property '%s' in the ConnectionString.", TRANSPORT_TYPE_CONFIG_NAME),
                             exception);
                 }
             } else {

eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/ConnectionStringBuilder.java

@@ -37,7 +37,7 @@ public final class ClientConstants {
     public static final String NO_RETRY = "NoRetry";
     public static final String DEFAULT_RETRY = "Default";
     public static final String PRODUCT_NAME = "MSJavaClient";
-    public static final String CURRENT_JAVACLIENT_VERSION = "2.3.0";
+    public static final String CURRENT_JAVACLIENT_VERSION = "2.3.1";
     public static final String PLATFORM_INFO = getPlatformInfo();
     public static final String FRAMEWORK_INFO = getFrameworkInfo();
     public static final String CBS_ADDRESS = "$cbs";

eventhubs/data-plane/azure-eventhubs/src/main/java/com/microsoft/azure/eventhubs/impl/ClientConstants.java

@@ -15,7 +15,7 @@
   <groupId>com.microsoft.azure</groupId>
   <artifactId>azure-eventhubs-clients</artifactId>
   <packaging>pom</packaging>
-  <version>2.3.0</version>
+  <version>2.3.1</version>
 
   <name>Microsoft Azure Event Hubs SDK Parent</name>
   <description>Java libraries for talking to Windows Azure Event Hubs</description>