Flink: Incrementally rewrite data files in streaming#37

Merged

wking1986 merged 20 commits intomaster-bkbase-0.13.1from

May 11, 2022

Commits on May 11, 2022

Flink: init streaming rewrite.

Reo-LEI
authored and
vanliu-tx
committed
refactor rewrite files committer.

Reo-LEI
authored and
vanliu-tx
committed
Add unittest for stream rewriter.

Reo-LEI
authored and
vanliu-tx
committed
Add unittest for rewrite rile committer.

Reo-LEI
authored and
vanliu-tx
committed
Add unittest for file committer.

Reo-LEI
authored and
vanliu-tx
committed
Fix unittest.

Reo-LEI
authored and
vanliu-tx
committed
Fix unittest.

Reo-LEI
authored and
vanliu-tx
committed
Fix unittest.

Reo-LEI
authored and
vanliu-tx
committed
Fix rewrite commit error when no rewrite result.

Reo-LEI
authored and
vanliu-tx
committed
Move to flink v1.13.

Reo-LEI
authored and
vanliu-tx
committed
Wrap PartitionData by StructLikeWrapper.

Reo-LEI
authored and
vanliu-tx
committed
support to emit all needed files for restore and concurrent writer.

Reo-LEI
authored and
vanliu-tx
committed
Fix unittest.

Reo-LEI
authored and
vanliu-tx
committed
fix class not found.

Reo-LEI
authored and
vanliu-tx
committed
not swallow exception when rewrite fail.

Reo-LEI
authored and
vanliu-tx
committed
port to flink 0.14

Reo-LEI
authored and
vanliu-tx
committed
Flink: fix streaming rewrite kryo serialization error for partition by StructLikeWrapper.

Reo-LEI
authored and
vanliu-tx
committed
Flink: refactor streaming rewrite.

Reo-LEI
authored and
vanliu-tx
committed
minor: fix build
vanliu-tx
committed
minor: fix build
vanliu-tx
committed