Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
246 commits
Select commit Hold shift + click to select a range
897d784
Create release branch for version 0.13.0.
yihua Jan 26, 2023
9dc6d79
[MINOR] Add `hudi-platform-service` and `hudi-metaserver-server-bundl…
yihua Jan 28, 2023
46cd7f6
[HUDI-5635] Fix release scripts (#7775)
yihua Jan 28, 2023
acda6d0
[MINOR] Add missing Apache License in source files (#7779)
yihua Jan 28, 2023
ca72c96
[MINOR] Fix validate_staged_release.sh (#7780)
yihua Jan 28, 2023
7f67e6a
[HUDI-5592] Fixing some of the flaky tests in CI (#7720)
nsivabalan Jan 27, 2023
9ac8765
[HUDI-5534] Optimizing Bloom Index lookup when using Bloom Filters fr…
alexeykudinkin Jan 27, 2023
c570e6b
[HUDI-5627] Improve the usability of Hudi CLI bundle (#7762)
yihua Jan 27, 2023
df05628
[HUDI-5623] Increase default time to wait between retries by lock pro…
yihua Jan 27, 2023
ee7764b
[HUDI-5023] Switching default Write Executor type to `SIMPLE` (#7476)
alexeykudinkin Jan 28, 2023
01acf99
[HUDI-5630] Fixing flaky parquet projection tests (#7768)
nsivabalan Jan 28, 2023
281b29e
[HUDI-5629] Clean CDC log files for enable/disable scenario (#7767)
YannByron Jan 28, 2023
cf52dc2
[HUDI-5626] Rename CDC logging mode options (#7760)
xushiyan Jan 28, 2023
44cd68b
[MINOR] Skip docs generation for table service manager (#7773)
xushiyan Jan 28, 2023
1b592c2
Revert "[MINOR] Add `hudi-platform-service` and `hudi-metaserver-serv…
xushiyan Jan 28, 2023
9495918
[HUDI-5631] Improve defaults of early conflict detection configs (#7770)
yihua Jan 28, 2023
01d56a6
[HUDI-5624] Fix HoodieAvroRecordMerger to use new precombine API (#7759)
bschell Jan 28, 2023
0d7d4c4
[HUDI-5628] Fixing log record reader scan V2 config name (#7764)
nsivabalan Jan 29, 2023
be1aa61
[HUDI-5638] Fix metaserver test setup (#7785)
xushiyan Jan 29, 2023
53c1d5d
[HUDI-5640] Add missing profiles in `deploy_staging_jars.sh` (#7784)
yihua Jan 29, 2023
8ccb80c
[HUDI-5639] Fixing stream identifier for single writer with spark str…
nsivabalan Jan 29, 2023
2ce0869
[MINOR] Fix `HoodieCDCRDD` setting flag `usesVirtualKeys` (#7777)
xushiyan Jan 29, 2023
48977a6
[HUDI-5503] Optimize flink table factory option check (#7608)
hbgstc123 Jan 29, 2023
5765e1b
[MINOR] Cleaning up recently introduced configs (#7772)
alexeykudinkin Jan 29, 2023
399549d
[HUDI-5629] Clean CDC log files for enable/disable scenario (#7786)
Zouxxyy Jan 30, 2023
cbff2bc
[HUDI-5637] Add Kryo for hive sync bundle (#7781)
codope Jan 30, 2023
b81f49e
[HUDI-5634] Rename CDC related classes (#7410)
YannByron Jan 30, 2023
8ffc7ce
[HUDI-5632] Fix failure launching Spark jobs from hudi-cli-bundle (#7…
yihua Jan 30, 2023
fe75c9a
[MINOR] Make `data_before_after` the default cdc logging mode (#7797)
xushiyan Jan 31, 2023
4768408
[HUDI-5563] Check table exist before drop table (#7679)
Zouxxyy Jan 31, 2023
2ced537
[HUDI-5568] Fix the BucketStreamWriteFunction to rebase the local fil…
loukey-lj Jan 31, 2023
571c0f2
[HUDI-5655] Closing write client for spark ds writer in all cases (in…
nsivabalan Jan 31, 2023
138af1a
[HUDI-5654] Fixing read of an empty rollback completed meta files fro…
nsivabalan Jan 31, 2023
bba8098
[HUDI-5487] Reduce duplicate logs in ExternalSpillableMap (#7579)
cxzl25 Jan 31, 2023
fb612be
[MINOR] Standardise schema concepts on Flink Engine (#7761)
voonhous Jan 31, 2023
9c21118
[HUDI-5567] Make the bootstrapping exception message more clear (#7684)
weimingdiit Jan 31, 2023
f0152ac
[HUDI-5553] Prevent partition(s) from being dropped if there are pend…
voonhous Jan 31, 2023
e6d44c0
[HUDI-5633] Fixing performance regression in `HoodieSparkRecord` (#7769)
alexeykudinkin Jan 31, 2023
22b634b
[HUDI-5585][flink] Fix flink creates and writes the table, the spark …
waywtdcc Feb 1, 2023
1ccf37f
[HUDI-5540] Close write client after usage of DeleteMarker/RollbackTo…
stream2000 Feb 1, 2023
9d3a8d5
[HUDI-5317] Fix insert overwrite table for partitioned table (#7793)
Zouxxyy Feb 1, 2023
b3fa996
[HUDI-5646] Guard dropping columns by a config, do not allow by defau…
codope Feb 1, 2023
6ef2135
[MINOR] Restoring existing behavior for `DeltaStreamer` Incremental S…
alexeykudinkin Feb 1, 2023
4a61e82
[HUDI-5681] Fixing Kryo being instantiated w/ invalid `SparkConf` (#7…
alexeykudinkin Feb 2, 2023
c487ee4
[HUDI-5676] Fix BigQuerySyncTool standalone mode (#7816)
xushiyan Feb 2, 2023
40d534e
[HUDI-5647] Automate savepoint and restore tests (#7796)
danny0405 Feb 2, 2023
91a9c82
[HUDI-5684] Fix CTAS and Insert Into to avoid combine-on-insert by de…
alexeykudinkin Feb 2, 2023
43373a7
[HUDI-5678] Fix `deduceShuffleParallelism` in row-writing Bulk Insert…
jonvex Feb 2, 2023
3bcbdd0
[0.13.0 Only] Disable default Avro schema validation (#7802)
alexeykudinkin Feb 2, 2023
3fc2c8e
[HUDI-5691] Fixing `HoodiePruneFileSourcePartitions` to properly hand…
alexeykudinkin Feb 3, 2023
4e5699b
[HUDI-5671] BucketIndexPartitioner partition algorithm skew (#7815)
loukey-lj Feb 3, 2023
46aa15b
[HUDI-5551] support seconds unit on event_time metrics (#7664)
scxwhite Feb 3, 2023
e0ef594
[HUDI-5682] Bucket index does not work correctly for multi-writer sce…
danny0405 Feb 3, 2023
9af629c
[HUDI-5653] Reset TestDataSource for TestHoodieDeltaStreamerWithMulti…
jonvex Feb 3, 2023
78a2c64
[MINOR] Fixing CTAS configuration not propagated properly (#7832)
alexeykudinkin Feb 3, 2023
4286d79
[HUDI-5496] Avoid unnecessary file system parsing to initialize metad…
nsivabalan Feb 4, 2023
003d6e7
[HUDI-5689] Make CDC config working for DeltaStreamer path (#7830)
xushiyan Feb 4, 2023
989d540
[MINOR] Validate configs for OCC early conflict detection (#7848)
xushiyan Feb 4, 2023
2d2d751
Make most of the Spark SQL DML operations configs overridable (#7850)
alexeykudinkin Feb 4, 2023
eb6bcee
[HUDI-5700] Annotate config classes for docs generation (#7853)
yihua Feb 4, 2023
f5cfec4
[HUDI-5701] Remove meta fields from cdc new record in CDCLogger (#7852)
YannByron Feb 5, 2023
2b17dd5
[HUDI-5702] Avoid to write useless change-data when compaction (#7849)
YannByron Feb 5, 2023
c6c38f9
[HUDI-5703] Improve `ConfigProperty` APIs for docs generation (#7854)
yihua Feb 5, 2023
21e6e89
[MINOR] Improve configuration configs (#7855)
yihua Feb 5, 2023
f8772de
[HUDI-5704] De-coupling column drop flag and schema validation flag (…
nsivabalan Feb 6, 2023
4acb016
[HUDI-5638] Activate platform service profile when deploy (#7858)
xushiyan Feb 6, 2023
721b913
Bumping release candidate number 2
yihua Feb 6, 2023
b711e39
[MINOR] Added safety-net check to catch any potential issue to deduce…
alexeykudinkin Feb 7, 2023
820006e
[HUDI-5718] Unsupported Operation Exception for compaction (#7874)
danny0405 Feb 7, 2023
847e7a9
[HUDI-5758] Restoring state of `HoodieKey` to make sure it's binary c…
alexeykudinkin Feb 10, 2023
7ccf6e6
[HUDI-5768] Fix Spark Datasource read of metadata table (#7924)
yihua Feb 12, 2023
d4106f3
[HUDI-5764] Rollback delta commits from `HoodieIndexer` lazily in met…
yihua Feb 12, 2023
4254fc9
[HUDI-5771] Improve deploy script of release artifacts (#7927)
yihua Feb 13, 2023
fe66488
Bumping release candidate number 3
yihua Feb 13, 2023
a3f0615
[MINOR] Update release version to reflect published version 0.13.0
yihua Feb 17, 2023
d729558
[HUDI-5329] Spark reads hudi table error when flink creates the table…
wuwenchi Feb 7, 2023
7e6c9ac
[HUDI-5720] Improve the validation script of the staged source releas…
yihua Feb 7, 2023
cab788f
[HUDI-5720] Improve the validation script of the staged source releas…
yihua Feb 7, 2023
f35ef9c
[MINOR] Fix indexer typos (#7863)
a49a Feb 8, 2023
593296a
[MINOR] fixed docs for WRITE_EXECUTOR_TYPE (#7880)
jonvex Feb 8, 2023
dbf8d7e
[HUDI-5721] Add Github actions on source validation (#7883)
yihua Feb 8, 2023
2459ec6
[HUDI-5270] Duplicate key error when insert_overwrite the same partit…
weimingdiit Feb 8, 2023
9fce16b
[HUDI-4968] Update misleading read.streaming.skip_compaction/skip_clu…
voonhous Feb 8, 2023
3106650
[MINOR] Description update for Flink Clustering and Compaction config…
voonhous Feb 9, 2023
f7a4933
[HUDI-5734] Fix flink batch read skip clustering data lost (#7903)
hbgstc123 Feb 10, 2023
0cf2aa3
[HUDI-5772] Align Flink clustering configuration with HoodieClusterin…
SteNicholas Feb 14, 2023
ec165b1
[HUDI-4406] Support Flink compaction/clustering write error resolveme…
chenshzh Feb 14, 2023
570805c
[HUDI-5792] Disabling flaky tests: TestDisruptorExecutionInSpark (#7946)
nsivabalan Feb 14, 2023
80002bc
[MINOR] Enable Azure CI to publish test results (#7943)
jonvex Feb 15, 2023
d8ebe99
[HUDI-5787] HMSDDLExecutor should set table type to EXTERNAL_TABLE wh…
SteNicholas Feb 15, 2023
4135ff9
[MINOR] Fix format name and remove redundant line in examples (#7952)
yihua Feb 16, 2023
3c79122
[MINOR] Improving logging and tracking of failures for metadata valid…
harsh1231 Feb 16, 2023
b371f83
[HUDI-5814] Disable flaky tests to unblock CI (#7986)
yihua Feb 17, 2023
e241b2f
[HUDI-5811] Specify the artifact of protoc to solve the problem that …
huangxiaopingRD Feb 17, 2023
15e6292
[HUDI-5800] Fix test failure in TestHoodieMergeOnReadTable (#7959)
lokeshj1703 Feb 17, 2023
8488f53
[HUDI-5799] Fix Spark partition validation in TestBulkInsertInternalP…
lokeshj1703 Feb 17, 2023
c6efff2
[HUDI-5821] Improve the resource release for HoodieTableFileSystemVie…
danny0405 Feb 21, 2023
ca9dc97
[HUDI-5792] fix CI test:TestDisruptorExecutionInSpark timeout problem…
gaoshihang Feb 21, 2023
4b24d6e
Handle empty payloads for AbstractDebeziumAvroPayload (#7944)
jonvex Feb 21, 2023
69df09f
[MINOR] Close InflaterInputStream in finally when calling decompressB…
TengHuo Feb 22, 2023
8d1a69b
update hudi-flink exception string (#8003)
jhkkhtd Feb 22, 2023
0218eae
[HUDI-5817] Fix async indexer metadata writer to avoid eager rollback…
yihua Feb 22, 2023
3fcbd43
[HUDI-5825] disable Spark UI in tests if SPARK_EVLOG_DIR not set (#8005)
kazdy Feb 24, 2023
8a2a3c7
[HUDI-5455] Add commons-configuration2 in hudi cli bundle (#7536)
rahil-c Feb 26, 2023
7c70f98
[MINOR] Refactor tests - spark read only once to df, then assert (#8045)
kazdy Feb 27, 2023
889187d
[HUDI-5838] Mask sensitive info while printing hudi properties in Del…
lokeshj1703 Feb 27, 2023
61eba6f
[HUDI-5855] Release resource actively for Flink hive meta sync (#8050)
XuQianJin-Stars Feb 27, 2023
7f0176b
[MINOR] fix the compilation failure for master branch. (#8059)
watermelon12138 Feb 27, 2023
1b6b539
[MINOR] schema evolution relax decimal type conversion conditions (#…
watermelon12138 Feb 28, 2023
d385b45
[HUDI-5852] Release the HoodieSyncTool actively in time where feasibl…
danny0405 Mar 1, 2023
d8ed0de
[HUDI-5869] Fix Some Typos in Hudi-Common (#8083)
slfan1989 Mar 2, 2023
7b18532
[HUDI-5728] HoodieTimelineArchiver archives the latest instant before…
SteNicholas Mar 3, 2023
d6ec59a
[HUDI-5665] Adding support to re-use table configs (#7901)
nsivabalan Mar 3, 2023
c780147
[HUDI-5736] Common de-coupling column drop flag and schema validation…
trushev Mar 4, 2023
8c99acc
fixing build failure
nsivabalan Mar 18, 2023
b42eecc
[HUDI-5796] Adding auto inferring partition from incoming df (#7951)
nsivabalan Mar 4, 2023
9ceebf6
[HUDI-5863] Fix HoodieMetadataFileSystemView serving stale view at th…
yihua Mar 4, 2023
988a8c5
Fix some comments on schema change (#8084)
mapleFU Mar 6, 2023
e4a85c1
[HUDI-5883] Avoid throwing error if data table does not exist in Meta…
yihua Mar 7, 2023
d446cb3
[HUDI-5882] Close table metadata and file system view instances in Me…
yihua Mar 7, 2023
a6a2b43
[MINOR] Fix testHoodieFlinkClusteringScheduleAfterArchive (#8110)
voonhous Mar 8, 2023
87ae8ae
[HUDI-5333] Ignore file system type of basePath when using RocksDbBas…
1032851561 Mar 10, 2023
c0f0e50
[HUDI-5857] Insert overwrite into bucket table would generate new fil…
beyond1920 Mar 10, 2023
36f0af2
[HUDI-5913] Table can not read correctly when computed column is in t…
1032851561 Mar 11, 2023
eb0e0fd
[HUDI-5919] Fix the validation of partition listing in metadata table…
yihua Mar 11, 2023
5e3262c
[HUDI-5915] Fixed load ckpMeatadata error when using minio (#8149)
linfey90 Mar 14, 2023
f61e41c
[MINOR] Ignoring warn msg for timeline server for metadata table (#8168)
nsivabalan Mar 14, 2023
70bef3f
[HUDI-5926] Improve cleaner parallelism (#8171)
yihua Mar 14, 2023
57aac40
[HUDI-5925] Improve bootstrap parallelism (#8170)
yihua Mar 14, 2023
8c8f472
[HUDI-5927] Improve parallelism of deleting invalid files (#8172)
yihua Mar 14, 2023
f1944b3
[MINOR] bot.yml ignore more filetype. (#7890)
jackwener Mar 15, 2023
a2f1297
[HUDI-5931] Improve the description of operation in HoodieDeltaStream…
huangxiaopingRD Mar 16, 2023
f39c3f6
[HUDI-5920] Improve documentation of parallelism configs (#8157)
yihua Mar 17, 2023
da7308d
[HUDI-5865] Fix table service client to instantiate with timeline ser…
yihua Mar 18, 2023
6ba197a
[HUDI-5785] Enhance Spark Datasource tests (#7938)
yihua Mar 18, 2023
b63603a
[HUDI-5058] HoodieCatalog#getTable sets primary key with hoodie.datas…
SteNicholas Feb 20, 2023
d166e16
[HUDI-5729] Fix RowDataKeyGen method getRecordKey (#7894)
sandyfog Feb 20, 2023
29f3e9a
[HUDI-5850] Fix timestamp(6) field long overflow (#8052)
sandyfog Feb 27, 2023
78f9b12
[HUDI-5917] Fix HoodieRetryWrapperFileSystem getDefaultReplication (#…
sandyfog Mar 11, 2023
f8148b0
fixing build failures
nsivabalan Mar 22, 2023
dc9dd5c
[HUDI-5950] Fixing pending instant deduction to trigger compaction in…
nsivabalan Mar 21, 2023
173298e
[HUDI-5822] Fix bucket stream writer fileId not found exception (#8263)
voonhous Mar 22, 2023
814715f
[HUDI-5962] Adding timeline server support to integ test suite (#8248)
nsivabalan Mar 22, 2023
dc10c44
[HUDI-5835] After performing the update operation, the hoodie table c…
xiarixiaoyao Feb 28, 2023
bf9537b
[HUDI-5878] Optimize the txn metadata for BaseCommitActionExecutor (#…
danny0405 Mar 6, 2023
de465bc
[HUDI-5887] Distinguish the single writer enabling metadata table and…
danny0405 Mar 8, 2023
580acf7
[HUDI-5941] Fix for RowData class cast exception (#8145)
kkrugler Mar 10, 2023
5766f14
[HUDI-5938] No need to init the properties for each data flushing (#8…
danny0405 Mar 16, 2023
f594750
[HUDI-5862] Strength MARKER based rollback for log files with a singl…
voonhous Mar 1, 2023
a52a5a3
[HUDI-5957] Fix table props not being properly propagated for HoodieC…
Zouxxyy Mar 23, 2023
c6b34bc
[HUDI-5289] Avoiding repeated trigger of clustering dag (#8275)
nsivabalan Mar 24, 2023
8a8ba79
[HUDI-5979] Add dependencies to hudi-trino-bundle needed for Trino co…
codope Mar 25, 2023
039dcbf
[HUDI-5985] Fix orc version for spark 3.3 (#8294)
xushiyan Mar 26, 2023
e9ea6ea
[HUDI-5891] Fix clustering on bootstrapped tables (#8206)
codope Mar 27, 2023
fe6d52f
[HUDI-5984] Enable FT for spark3.x versions in CI (#8293)
xushiyan Mar 27, 2023
a33984f
[HUDI-5952] Fix NPE when use kafka callback (#8227)
wuzhenhua01 Mar 29, 2023
2cae5bf
[HUDI-5928] Fixing shutting down deltastreamer properly when post wri…
nsivabalan Mar 29, 2023
2f2c0f9
[HUDI-5993] Connection leak for lock provider (#8304)
xicm Mar 30, 2023
dced583
[HUDI-5992] Fix (de)serialization for avro versions > 1.10.0 (#8307)
voonhous Mar 30, 2023
eeb3342
[HUDI-6003] Add schema evolution nullability test case (#8320)
xushiyan Mar 30, 2023
55838f0
[HUDI-5967] Add partition ordering for full table scans (#8269)
alexguo8 Mar 24, 2023
97bfb10
[HUDI-915][HUDI-5656] Rebased `HoodieBootstrapRelation` onto `HoodieB…
alexeykudinkin Feb 24, 2023
c57f167
[HUDI-5960] Allow bootstrap procedure to throw an exception when exec…
huangxiaopingRD Mar 22, 2023
93ddf99
[HUDI-5999] Fix bootstrap relation for nonpartitioned bootstrap table…
jonvex Mar 31, 2023
1041e44
Fix bootstrap test failure
codope Apr 3, 2023
d2f1dee
[HUDI-5921] Partition path should be considered in BucketIndexConcurr…
xicm Apr 1, 2023
51a0229
[HUDI-6030] Cleans the ckp meta while the JM restarts (#8374)
danny0405 Apr 5, 2023
8e8592d
[HUDI-6045] Support append mode by default for MOR table with INSERT …
zhuanshenbsj1 Apr 7, 2023
67a39b7
[HUDI-6038] Fix async compact/clustering serdes conflicts caused by W…
chenshzh Apr 7, 2023
2ee120d
[HUDI-6032] Fix read meta field '_hoodie_commit_time' multiple times …
LinMingQiang Apr 7, 2023
3503c9a
[HUDI-5406] Fix NoClassDefFoundError: org/apache/log4j/helpers/OnlyOn…
lokeshj1703 Apr 7, 2023
ba2f4fc
[HUDI-5955] Fix incremental clean not work caused by archiving (#8373)
hbgstc123 Apr 8, 2023
006d15c
[HUDI-5989] Fix date conversion issue when performing partition pruni…
voonhous Apr 10, 2023
89429e9
[MINOR] Avoid synchronized block if key is present in cache (#8433)
the-other-tim-brown Apr 13, 2023
1c2ce16
[HUDI-6053] Fix integration test run for M1 mac (#8419)
lokeshj1703 Apr 13, 2023
c6d3373
[HUDI-5990] Avoid missing data during incremental queries (#8299)
rfyu Apr 13, 2023
67f98e4
[HUDI-6048] Check if partition exists before list partition by path p…
Zouxxyy Apr 14, 2023
eec70bc
[HUDI-6074] Check inlineClusteringEnabled in isAsyncClusteringEnabled…
Zouxxyy Apr 14, 2023
099c132
[MINOR] Fix hard-coded storage level for indexing (#7993)
xushiyan Apr 14, 2023
4bb04c3
[HUDI-6073] Table create schema should not include metadata fields (#…
danny0405 Apr 14, 2023
0152cb9
[HUDI-6063] Modify logging errors In JDBCExecutor (#8434)
slfan1989 Apr 15, 2023
17b8dee
[HUDI-5991] Fix RDDCustomColumnsSortPartitioner's RepartitionRecords…
wuwenchi Apr 16, 2023
5c1f41a
[HUDI-6050] Fix write helper deduplicate records lost origin data ope…
onlywangyh Apr 16, 2023
a7e9612
[HUDI-6080] Use hoodie.properties to determine if the table exists (#…
xicm Apr 17, 2023
249be86
[HUDI-6052] Standardise TIMESTAMP(6) format when writing to Parquet f…
voonhous Apr 17, 2023
7c4a04a
[HUDI-6076] Fix the unit of clustering.plan.strategy.target.file.max.…
Zouxxyy Apr 18, 2023
27d92af
[HUDI-6071] Flink Hive Catalog throws exception through SQL Client wh…
zhiqiang-dev Apr 18, 2023
fcefbe7
[HUDI-6009] Let the jetty server in TimelineService create daemon thr…
cxzl25 Apr 19, 2023
e517039
[HUDI-6033] Fix rounding exception when to decimal casting (#8380)
voonhous Apr 19, 2023
b885a2f
[HUDI-6015] Refresh the table after executing rollback to instantTime…
huangxiaopingRD Apr 19, 2023
774a3e2
[HUDI-6087] Fix flink streaming source savepoint instant (#8476)
voonhous Apr 19, 2023
16600f7
[HUDI-5987] Fix clustering on bootstrap table with row writer disable…
codope Apr 20, 2023
26c10ae
[HUDI-5509] Check if dfs support atomic creation when using filesyste…
fengjian428 Apr 20, 2023
93e922c
[HUDI-6084] Added FailOnFirstErrorWriteStatus for MDT to ensure that …
prashantwason Apr 20, 2023
da9f346
[HUDI-6101] Use UnBoundedCompactionStrategy for MDT compactions (#8496)
prashantwason Apr 20, 2023
54a6ab4
[HUDI-6108] Alter table should not include metadata fields in the dat…
danny0405 Apr 21, 2023
bef5ba0
[HUDI-6100] Fixed overflow in setting log block size by making it lon…
prashantwason Apr 21, 2023
ca0a619
[HUDI-6109] Fix the scala compile ambiguity of Properties#putAll (#8511)
danny0405 Apr 21, 2023
16cb9c4
[HUDI-6039] Fixing FS based listing for full cleaning in clean Planne…
nsivabalan Apr 21, 2023
0157e4b
[HUDI-6091] Add Java 11 and 17 to bundle validation (#8481)
yihua Apr 21, 2023
7dab4d8
[HUDI-6092] Reuse schema objects while deserializing log blocks (#8484)
prashantwason Apr 22, 2023
a95e7c7
[HUDI-4920] fix PartialUpdatePayload cannot return deleted record in …
fengjian428 Apr 23, 2023
84ccaf1
[HUDI-6123] Auto adjust lock configs for single writer (#8542)
danny0405 Apr 23, 2023
6d0490a
[HUDI-6018] Set owner of hive table to ShortUserName in Kerberos (#8361)
lvyanquan Apr 24, 2023
323f3ca
[MINOR] Updating DOAP file for 0.12.3 (#8553)
nsivabalan Apr 24, 2023
4278b77
[HUDI-5957] Fix table not exist when using 'db.table' in createHoodie…
Zouxxyy Apr 25, 2023
1a22328
[HUDI-6135] FlinkClusteringConfig adds --sort-memory option to suppor…
SteNicholas Apr 25, 2023
6683f94
[HUDI-6090] Optimise payload size for list of FileGroupDTO (#8480)
lokeshj1703 Apr 25, 2023
97afbd5
[MINOR] fix misleading configuration value (#8534)
voonhous Apr 25, 2023
81c95ad
[HUDI-6127] Flink Hudi Write support commit on an empty batch (#8550)
stream2000 Apr 27, 2023
f1b88d4
[HUDI-6114] Fixed rollback of blocks in scanInternalV1 (#8523)
prashantwason Apr 28, 2023
dce677b
[HUDI-6072] Fix NPE when upsert merger and null map or array (#8432)
parisni Apr 28, 2023
e02e273
[HUDI-6145] Fix the flink table create schema to be compatible with S…
danny0405 Apr 28, 2023
d602513
[MINOR] Improve TestStreamWriteOperatorCoordinator#testCommitOnEmptyB…
stream2000 Apr 28, 2023
d921ff9
[MINOR] Fix the hudi-cli export command (#8608)
prashantwason Apr 30, 2023
3fabaff
[MINOR] Match the directory-filter-regex to the relative directory na…
prashantwason Apr 30, 2023
ac886e4
Revert "[MINOR] Enable Azure CI to publish test results (#7943)" (#8612)
xushiyan Apr 30, 2023
35ed607
[HUDI-6132] Fixing checkpoint management for multiple streaming write…
nsivabalan May 2, 2023
e6aefc6
[MINOR] Log schema if it changes in Hive sync (#8615)
codope May 2, 2023
b754efe
[HUDI-6068] Improve logic of getOldestInstantToRetainForClustering wh…
hbgstc123 May 2, 2023
fc3c136
[MINOR] Fix hadoop configuration not being applied by FileIndex (#8595)
Paddy0523 May 4, 2023
6d99025
[MINOR] Improve documentation of org.apache.hudi.common.table.view.Ab…
Paddy0523 May 4, 2023
940e773
[HUDI-6184] Improve the test on incremental queries (#8648)
yihua May 6, 2023
4d49ef7
[HUDI-6027] Remove unnecessary scala-maven-plugin (#8336)
CTTY May 6, 2023
81360ee
[HUDI-6185] Too many logs in the ExternalSpillableMap (#8649)
danny0405 May 6, 2023
8e7b491
[HUDI-6174] Fixing flaky tests in HoodieDeltastreamer (#8646)
nsivabalan May 7, 2023
98e2fb1
[HUDI-6186] Fix lock identity in InProcessLockProvider (#8658)
yihua May 8, 2023
3a953cc
[HUDI-5816] List all partitions as the fallback mechanism in Hive and…
yihua May 8, 2023
f68c6d6
[MINOR] Use try with resource to close stream (#8596)
c-f-cooper May 8, 2023
eb47987
[MINOR] Add script to build bundle validation image with Spark 3.3.2 …
yihua May 8, 2023
24d8e82
[MINOR][DOC][hudi-metaserver] Fix typos in README.md (#8536)
littleeleventhwolf May 9, 2023
5e7ac2d
[HUDI-6047] Clustering operation on consistent hashing index resultin…
rohan-uptycs May 10, 2023
947adeb
[HUDI-6196] Keep compatibility for old version archival instants with…
prashantwason May 10, 2023
b169f78
[HUDI-6170] Use correct zone id while calculating earliestTimeToRetai…
lokeshj1703 May 10, 2023
30d497a
[HUDI-5308] Hive3 query returns null when the where clause has a part…
xicm May 10, 2023
e72a6de
[HUDI-5868] Make hudi-spark compatible against Spark 3.3.2 (#8082)
rahil-c May 11, 2023
842e901
[MINOR] Added docs on gotchas when using PartialUpdateAvroPayload (#8…
voonhous May 11, 2023
3a1414c
[MINOR] Migrate azure-pipelines.yml with notes (#8694)
xushiyan May 12, 2023
f58c73f
[HUDI-6204] Add bundle validation on Spark 3.3.2 (#8692)
yihua May 13, 2023
eaedc7c
Do not combine records if write operation is Upsert and COMBINE_BEFOR…
kazdy May 15, 2023
1d780e2
[HUDI-6199] Fix deletes with custom payload implementation (#8690)
yihua May 15, 2023
d800630
[HUDI-6134] Prevent clean run concurrently in flink (#8568)
hbgstc123 May 16, 2023
ecf3f36
[HUDI-6222] ParquetSchemaConverter shoud always convert the Map key t…
danny0405 May 17, 2023
2e9b88a
Create release branch for version 0.13.1
zhangyue19921010 May 18, 2023
7a65439
Create release branch for version 0.13.1
zhangyue19921010 May 23, 2023
056020d
Update MergeOnReadInputFormat.java
hanleiit Mar 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
74 changes: 68 additions & 6 deletions .github/workflows/bot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,32 +15,61 @@ on:
- '**.pdf'
- '**.png'
- '**.svg'
- '**.yaml'
- '**.yml'
- '.gitignore'
branches:
- master
- 'release-*'
env:
MVN_ARGS: -ntp -B -V -Pwarn-log -Dorg.slf4j.simpleLogger.log.org.apache.maven.plugins.shade=warn -Dorg.slf4j.simpleLogger.log.org.apache.maven.plugins.dependency=warn
MVN_ARGS: -e -ntp -B -V -Pwarn-log -Dorg.slf4j.simpleLogger.log.org.apache.maven.plugins.shade=warn -Dorg.slf4j.simpleLogger.log.org.apache.maven.plugins.dependency=warn
SPARK_COMMON_MODULES: hudi-spark-datasource/hudi-spark,hudi-spark-datasource/hudi-spark-common

jobs:
validate-source:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK 8
uses: actions/setup-java@v2
with:
java-version: '8'
distribution: 'adopt'
architecture: x64
- name: Check Binary Files
run: ./scripts/release/validate_source_binary_files.sh
- name: Check Copyright
run: |
./scripts/release/create_source_directory.sh hudi-tmp-repo
cd hudi-tmp-repo
./scripts/release/validate_source_copyright.sh
- name: RAT check
run: ./scripts/release/validate_source_rat.sh

test-spark:
runs-on: ubuntu-latest
strategy:
matrix:
include:
- scalaProfile: "scala-2.11"
sparkProfile: "spark2.4"
sparkModules: "hudi-spark-datasource/hudi-spark2"

- scalaProfile: "scala-2.12"
sparkProfile: "spark2.4"
sparkModules: "hudi-spark-datasource/hudi-spark2"

- scalaProfile: "scala-2.12"
sparkProfile: "spark3.1"
sparkModules: "hudi-spark-datasource/hudi-spark3.1.x"

- scalaProfile: "scala-2.12"
sparkProfile: "spark3.2"
sparkModules: "hudi-spark-datasource/hudi-spark3.2.x"

- scalaProfile: "scala-2.12"
sparkProfile: "spark3.3"
sparkModules: "hudi-spark-datasource/hudi-spark3.3.x"

steps:
- uses: actions/checkout@v2
Expand All @@ -66,9 +95,18 @@ jobs:
env:
SCALA_PROFILE: ${{ matrix.scalaProfile }}
SPARK_PROFILE: ${{ matrix.sparkProfile }}
SPARK_MODULES: ${{ matrix.sparkModules }}
if: ${{ !endsWith(env.SPARK_PROFILE, '2.4') }} # skip test spark 2.4 as it's covered by Azure CI
run:
mvn test -Punit-tests -D"$SCALA_PROFILE" -D"$SPARK_PROFILE" -pl hudi-common,hudi-spark-datasource/hudi-spark $MVN_ARGS
mvn test -Punit-tests -D"$SCALA_PROFILE" -D"$SPARK_PROFILE" -pl "hudi-common,$SPARK_COMMON_MODULES,$SPARK_MODULES" $MVN_ARGS
- name: FT - Spark
env:
SCALA_PROFILE: ${{ matrix.scalaProfile }}
SPARK_PROFILE: ${{ matrix.sparkProfile }}
SPARK_MODULES: ${{ matrix.sparkModules }}
if: ${{ !endsWith(env.SPARK_PROFILE, '2.4') }} # skip test spark 2.4 as it's covered by Azure CI
run:
mvn test -Pfunctional-tests -D"$SCALA_PROFILE" -D"$SPARK_PROFILE" -pl "$SPARK_COMMON_MODULES,$SPARK_MODULES" $MVN_ARGS

test-flink:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -107,12 +145,16 @@ jobs:
include:
- flinkProfile: 'flink1.16'
sparkProfile: 'spark3.3'
sparkRuntime: 'spark3.3.2'
- flinkProfile: 'flink1.15'
sparkProfile: 'spark3.3'
sparkRuntime: 'spark3.3.1'
- flinkProfile: 'flink1.14'
sparkProfile: 'spark3.2'
sparkRuntime: 'spark3.2.3'
- flinkProfile: 'flink1.13'
sparkProfile: 'spark3.1'
sparkRuntime: 'spark3.1.3'
steps:
- uses: actions/checkout@v2
- name: Set up JDK 8
Expand All @@ -127,13 +169,33 @@ jobs:
SPARK_PROFILE: ${{ matrix.sparkProfile }}
SCALA_PROFILE: 'scala-2.12'
run: |
mvn clean package -D"$SCALA_PROFILE" -D"$SPARK_PROFILE" -Pintegration-tests -DskipTests=true -Phudi-platform-service $MVN_ARGS
mvn clean package -D"$SCALA_PROFILE" -D"$FLINK_PROFILE" -pl packaging/hudi-flink-bundle -am -Davro.version=1.10.0 -DskipTests=true $MVN_ARGS
- name: IT - Bundle Validation
mvn clean package -D"$SCALA_PROFILE" -D"$SPARK_PROFILE" -DdeployArtifacts=true -DskipTests=true $MVN_ARGS
# TODO remove the sudo below. It's a needed workaround as detailed in HUDI-5708.
sudo chown -R "$USER:$(id -g -n)" hudi-platform-service/hudi-metaserver/target/generated-sources
mvn clean package -D"$SCALA_PROFILE" -D"$FLINK_PROFILE" -DdeployArtifacts=true -DskipTests=true $MVN_ARGS -pl packaging/hudi-flink-bundle -am -Davro.version=1.10.0
- name: IT - Bundle Validation - OpenJDK 8
env:
FLINK_PROFILE: ${{ matrix.flinkProfile }}
SPARK_RUNTIME: ${{ matrix.sparkRuntime }}
SCALA_PROFILE: 'scala-2.12'
run: |
HUDI_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
./packaging/bundle-validation/ci_run.sh $HUDI_VERSION openjdk8
- name: IT - Bundle Validation - OpenJDK 11
env:
FLINK_PROFILE: ${{ matrix.flinkProfile }}
SPARK_RUNTIME: ${{ matrix.sparkRuntime }}
SCALA_PROFILE: 'scala-2.12'
run: |
HUDI_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
./packaging/bundle-validation/ci_run.sh $HUDI_VERSION openjdk11
- name: IT - Bundle Validation - OpenJDK 17
env:
FLINK_PROFILE: ${{ matrix.flinkProfile }}
SPARK_PROFILE: ${{ matrix.sparkProfile }}
SPARK_RUNTIME: ${{ matrix.sparkRuntime }}
SCALA_PROFILE: 'scala-2.12'
if: ${{ endsWith(env.SPARK_PROFILE, '3.3') }} # Only Spark 3.3 supports Java 17 as of now
run: |
HUDI_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
./packaging/bundle-validation/ci_run.sh $HUDI_VERSION
./packaging/bundle-validation/ci_run.sh $HUDI_VERSION openjdk17
6 changes: 5 additions & 1 deletion azure-pipelines.yml → azure-pipelines-20230430.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# NOTE:
# This config file defines how Azure CI runs tests with Spark 2.4 and Flink 1.17 profiles.
# PRs will need to keep in sync with master's version to trigger the CI runs.

trigger:
branches:
include:
Expand Down Expand Up @@ -85,7 +89,7 @@ parameters:
variables:
BUILD_PROFILES: '-Dscala-2.11 -Dspark2.4 -Dflink1.16'
PLUGIN_OPTS: '-Dcheckstyle.skip=true -Drat.skip=true -Djacoco.skip=true -ntp -B -V -Pwarn-log -Dorg.slf4j.simpleLogger.log.org.apache.maven.plugins.shade=warn -Dorg.slf4j.simpleLogger.log.org.apache.maven.plugins.dependency=warn'
MVN_OPTS_INSTALL: '-DskipTests $(BUILD_PROFILES) $(PLUGIN_OPTS)'
MVN_OPTS_INSTALL: '-Phudi-platform-service -DskipTests $(BUILD_PROFILES) $(PLUGIN_OPTS)'
MVN_OPTS_TEST: '-fae -Pwarn-log $(BUILD_PROFILES) $(PLUGIN_OPTS)'
SPARK_VERSION: '2.4.4'
HADOOP_VERSION: '2.7'
Expand Down
10 changes: 10 additions & 0 deletions doap_HUDI.rdf
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,16 @@
<created>2022-12-28</created>
<revision>0.12.2</revision>
</Version>
<Version>
<name>Apache Hudi 0.13.0</name>
<created>2023-02-25</created>
<revision>0.13.0</revision>
</Version>
<Version>
<name>Apache Hudi 0.12.3</name>
<created>2023-04-23</created>
<revision>0.12.3</revision>
</Version>
</release>
<repository>
<GitRepository>
Expand Down
2 changes: 1 addition & 1 deletion docker/demo/config/test-suite/compact-test.properties
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ hoodie.bulkinsert.shuffle.parallelism=100
hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
hoodie.deltastreamer.source.test.max_unique_records=100000000
hoodie.embed.timeline.server=false

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
Expand Down
2 changes: 1 addition & 1 deletion docker/demo/config/test-suite/multi-writer-1.properties
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ hoodie.metadata.enable=false
hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
hoodie.deltastreamer.source.test.max_unique_records=100000000
hoodie.embed.timeline.server=false

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
Expand Down
2 changes: 1 addition & 1 deletion docker/demo/config/test-suite/multi-writer-2.properties
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ hoodie.metadata.enable=false
hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
hoodie.deltastreamer.source.test.max_unique_records=100000000
hoodie.embed.timeline.server=false

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ hoodie.metadata.enable=false
hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
hoodie.deltastreamer.source.test.max_unique_records=100000000
hoodie.embed.timeline.server=false

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ hoodie.metadata.enable=false
hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
hoodie.deltastreamer.source.test.max_unique_records=100000000
hoodie.embed.timeline.server=false

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ hoodie.metadata.enable=false
hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
hoodie.deltastreamer.source.test.max_unique_records=100000000
hoodie.embed.timeline.server=false

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ hoodie.metadata.enable=false
hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
hoodie.deltastreamer.source.test.max_unique_records=100000000
hoodie.embed.timeline.server=false

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ hoodie.bulkinsert.shuffle.parallelism=100
hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
hoodie.deltastreamer.source.test.max_unique_records=100000000
hoodie.embed.timeline.server=false

hoodie.deltastreamer.source.input.selector=org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector

hoodie.datasource.hive_sync.skip_ro_suffix=true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ hoodie.keep.min.commits=12
hoodie.keep.max.commits=14

hoodie.compact.inline=true
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ hoodie.delete.shuffle.parallelism=25
hoodie.cleaner.commits.retained=8
hoodie.keep.min.commits=12
hoodie.keep.max.commits=14
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ hoodie.bulkinsert.shuffle.parallelism=25
hoodie.delete.shuffle.parallelism=25

hoodie.compact.inline=true
hoodie.embed.timeline.server=false


hoodie.cleaner.commits.retained=8
hoodie.keep.min.commits=12
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ hoodie.delete.shuffle.parallelism=25
hoodie.cleaner.commits.retained=8
hoodie.keep.min.commits=12
hoodie.keep.max.commits=14
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ hoodie.upsert.shuffle.parallelism=25
hoodie.bulkinsert.shuffle.parallelism=25
hoodie.delete.shuffle.parallelism=25

hoodie.embed.timeline.server=false


hoodie.compact.inline=true
hoodie.deltastreamer.source.test.num_partitions=100
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ hoodie.keep.max.commits=14

hoodie.compact.inline=true
hoodie.metadata.enable=true
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ hoodie.cleaner.commits.retained=8
hoodie.keep.min.commits=12
hoodie.keep.max.commits=14

hoodie.embed.timeline.server=false

hoodie.metadata.enable=true

hoodie.deltastreamer.source.test.num_partitions=100
Expand Down
2 changes: 1 addition & 1 deletion docker/demo/config/test-suite/test-clustering.properties
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ hoodie.upsert.shuffle.parallelism=25
hoodie.bulkinsert.shuffle.parallelism=25
hoodie.delete.shuffle.parallelism=25

hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ hoodie.delete.shuffle.parallelism=25

hoodie.metadata.enable=false
hoodie.compact.inline=true
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ hoodie.cleaner.commits.retained=8
hoodie.keep.min.commits=12
hoodie.keep.max.commits=14

hoodie.embed.timeline.server=false

hoodie.metadata.enable=true
hoodie.compact.inline=true

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ hoodie.keep.min.commits=12
hoodie.keep.max.commits=14

hoodie.metadata.enable=true
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ hoodie.delete.shuffle.parallelism=25

hoodie.metadata.enable=true
hoodie.compact.inline=true
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
2 changes: 1 addition & 1 deletion docker/demo/config/test-suite/test-metadata.properties
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ hoodie.bulkinsert.shuffle.parallelism=25
hoodie.delete.shuffle.parallelism=25

hoodie.metadata.enable=true
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ hoodie.delete.shuffle.parallelism=25

hoodie.metadata.enable=false
hoodie.compact.inline=true
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ hoodie.delete.shuffle.parallelism=25

hoodie.metadata.enable=true
hoodie.compact.inline=true
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ hoodie.bulkinsert.shuffle.parallelism=25
hoodie.delete.shuffle.parallelism=25

hoodie.metadata.enable=true
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ hoodie.bulkinsert.shuffle.parallelism=25
hoodie.delete.shuffle.parallelism=25

hoodie.metadata.enable=false
hoodie.embed.timeline.server=false


hoodie.deltastreamer.source.test.num_partitions=100
hoodie.deltastreamer.source.test.datagen.use_rocksdb_for_storing_existing_keys=false
Expand Down
Loading