Skip to content

v1.6.1

Compare
Choose a tag to compare
@edenhill edenhill released this 24 Feb 13:39
· 573 commits to master since this release

librdkafka v1.6.1

librdkafka v1.6.1 is a maintenance release.

Upgrade considerations

  • Fatal idempotent producer errors are now also fatal to the transactional
    producer. This is a necessary step to maintain data integrity prior to
    librdkafka supporting KIP-360. Applications should check any transactional
    API errors for the is_fatal flag and decommission the transactional producer
    if the flag is set.
  • The consumer error raised by auto.offset.reset=error now has error-code
    set to ERR__AUTO_OFFSET_RESET to allow an application to differentiate
    between auto offset resets and other consumer errors.

Fixes

General fixes

  • Admin API and transactional send_offsets_to_transaction() coordinator
    requests, such as TxnOffsetCommitRequest, could in rare cases be sent
    multiple times which could cause a crash.
  • ssl.ca.location=probe is now enabled by default on Mac OSX since the
    librdkafka-bundled OpenSSL might not have the same default CA search paths
    as the system or brew installed OpenSSL. Probing scans all known locations.

Transactional Producer fixes

  • Fatal idempotent producer errors are now also fatal to the transactional
    producer.
  • The transactional producer could crash if the transaction failed while
    send_offsets_to_transaction() was called.
  • Group coordinator requests for transactional
    send_offsets_to_transaction() calls would leak memory if the
    underlying request was attempted to be sent after the transaction had
    failed.
  • When gradually producing to multiple partitions (resulting in multiple
    underlying AddPartitionsToTxnRequests) sub-sequent partitions could get
    stuck in pending state under certain conditions. These pending partitions
    would not send queued messages to the broker and eventually trigger
    message timeouts, failing the current transaction. This is now fixed.
  • Committing an empty transaction (no messages were produced and no
    offsets were sent) would previously raise a fatal error due to invalid state
    on the transaction coordinator. We now allow empty/no-op transactions to
    be committed.

Consumer fixes

  • The consumer will now retry indefinitely (or until the assignment is changed)
    to retrieve committed offsets. This fixes the issue where only two retries
    were attempted when outstanding transactions were blocking OffsetFetch
    requests with ERR_UNSTABLE_OFFSET_COMMIT. #3265