KAFKA-5494: enable idempotence with max.in.flight.requests.per.connection > 1#3743

Closed

apurvam wants to merge 40 commits into

apache:trunkfrom

apurvam:KAFKA-5494-increase-max-in-flight-for-idempotent-producer

Commits on Sep 14, 2017

Initial commit of client side changes with some tests
Apurva Mehta
committed
Implemented broker side changes to cache extra metadata.
Apurva Mehta
committed
Change the client side code so that the sequence numbers are assigned
Apurva Mehta
committed
WIP
Apurva Mehta
committed
Implemented log cleaning functionality with tests
Apurva Mehta
committed
Fix merge issues aftre rebasing onto trunk
Apurva Mehta
committed
Remove some unneeded variables and function parameters
Apurva Mehta
committed
More minor cleanups
Apurva Mehta
committed
WIP
Apurva Mehta
committed
Handled remaining corner cases:
Apurva Mehta
committed
Rebased onto trunk and fixed some merge conflicts
Apurva Mehta
committed
Added unit tests for out of order responses and expiring batches
Apurva Mehta
committed
Address PR comments on client side code
Apurva Mehta
committed
WIP
Apurva Mehta
committed
Revert changes where we stored the metadata for the last 5 batches. We don't need this information except in extremely rare cases, and it doesn't seem worth the cost of supporting two on disk versi…
Apurva Mehta
committed
Addressed a few more PR comments
Apurva Mehta
committed
Address review comments for the broker side chanegs. Added tests for the new duplicate detection logic.
Apurva Mehta
committed
Handle duplicates during wraparound
Apurva Mehta
committed
Two changes:
Apurva Mehta
committed
Addressed more PR comments
Apurva Mehta
committed
Address a few more minor PR comments
Apurva Mehta
committed
Revert changes to log4j.properties
Apurva Mehta
committed
Delete unnecessary trailing whitespace
Apurva Mehta
committed
Let the producerIdEntry have a concurrent structure for batch metadata. This makes accesses from the cleaner threads (log cleaner, producer id expiration, etc.) safe
Apurva Mehta
committed
Bunch of changes:
Apurva Mehta
committed
Rename ProducerIdEntry.lastOffset to ProducerIdEntry.lastDataOffset everywhere
Apurva Mehta
committed
Address more PR comments
Apurva Mehta
committed
Fix previous commit which introduced a bug in RecordAccmulator.drain.
Apurva Mehta
committed
PR comment: return currentLastSequence instead of doing another map lookup
Apurva Mehta
committed
Remove the concurrent queue from ProducerIdEntry and instead return just the lastSeq per producer to the LogCleaner, hence simpliying the concurrency model
Apurva Mehta
committed
Address other minor comments
Apurva Mehta
committed
Correctly transition to a fatal error state when batches in retry are expired during a transaction
Apurva Mehta
committed
Added assertions on the send futures for some of the newly added test cases in SenderTest
Apurva Mehta
committed
check the response offsets in all the tests
Apurva Mehta
committed
Fix checkstyle failures
Apurva Mehta
committed
Address further PR comments
Apurva Mehta
committed
Address PR comments.
Apurva Mehta
committed
Fix up build failures after merge
Apurva Mehta
committed
Address more PR comments:
Apurva Mehta
committed
Address final PR comment
Apurva Mehta
committed