[HUDI-5023] Cleaning up `QueueBasedExecutor` impls#7238

Merged

codope merged 29 commits intoapache:masterfrom

onehouseinc:ak/exeq-fix

Dec 7, 2022

Commits on Dec 3, 2022

Fixed incorrect use of `CountDownLatch`
Alexey Kudinkin
committed
Removed unnecessary casting
Alexey Kudinkin
committed
Adding logs
Alexey Kudinkin
committed
Unified `getResult` and `finish` in one API
Alexey Kudinkin
committed
Simplifying `DisruptorExecutor`
Alexey Kudinkin
committed
`BoundedInMemoryQueueIterable` > `BoundedInMemoryQueue`
Alexey Kudinkin
committed
Lifted all queue lifecycle management to `HoodieExecutorBase`
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Rebased direct users of the BIMQ to rely on `QueueBasedExecutorFactory` instead
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Close the queue after processing is done
Alexey Kudinkin
committed
Cleaning up superfluous APIs
Alexey Kudinkin
committed
Adding missing deps to bundles
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Moved actual queue iteration into `BoundedInMemoryQueueExecutor`
Alexey Kudinkin
committed
Cleaned up `IteratorBasedQueueConsumer`
Alexey Kudinkin
committed
Revisited `BoundedInMemoryExecutor` bootstrapping w/o consumer;
Alexey Kudinkin
committed
Streamline and unify producing handling w/in Queue-based executors
Alexey Kudinkin
committed
Abstracted common producing seq
Alexey Kudinkin
committed
Delineated sealing/closing of the queue (addressing the issue of shuttind down the queue prematurely)
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Lifted up all concurrency management to `BaseHoodieQueueBasedExuecutor`;
Alexey Kudinkin
committed
Decouple consuming seq from returning the result, to address the issue of DQ returning prematurely
Alexey Kudinkin
committed
Relocated Disruptor queue setup under `doConsume`
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Replace `join` w/ `get` to be able to handle `InterruptedException`
Alexey Kudinkin
committed
Simplify the test
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Fixed tests
Alexey Kudinkin
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HUDI-5023] Cleaning up `QueueBasedExecutor` impls#7238

[HUDI-5023] Cleaning up `QueueBasedExecutor` impls#7238
codope merged 29 commits intoapache:masterfrom
onehouseinc:ak/exeq-fix

Commits on Dec 3, 2022

Fixed incorrect use of `CountDownLatch`

Removed unnecessary casting

Adding logs

Unified `getResult` and `finish` in one API

Simplifying `DisruptorExecutor`

`BoundedInMemoryQueueIterable` > `BoundedInMemoryQueue`

Lifted all queue lifecycle management to `HoodieExecutorBase`

Tidying up

Rebased direct users of the BIMQ to rely on `QueueBasedExecutorFactory` instead

Tidying up

Close the queue after processing is done

Cleaning up superfluous APIs

Adding missing deps to bundles

Tidying up

Moved actual queue iteration into `BoundedInMemoryQueueExecutor`

Cleaned up `IteratorBasedQueueConsumer`

Revisited `BoundedInMemoryExecutor` bootstrapping w/o consumer;

Streamline and unify producing handling w/in Queue-based executors

Abstracted common producing seq

Delineated sealing/closing of the queue (addressing the issue of shuttind down the queue prematurely)

Tidying up

Lifted up all concurrency management to `BaseHoodieQueueBasedExuecutor`;

Decouple consuming seq from returning the result, to address the issue of DQ returning prematurely

Relocated Disruptor queue setup under `doConsume`

Tidying up

Replace `join` w/ `get` to be able to handle `InterruptedException`

Simplify the test

Tidying up

Fixed tests