Skip to content

Conversation

@bowenliang123
Copy link
Contributor

Why are the changes needed?

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before make a pull request

@pan3793
Copy link
Member

pan3793 commented Mar 21, 2023

Upgrading Hudi should re-evaluate all transitive deps

@bowenliang123
Copy link
Contributor Author

Upgrading Hudi should re-evaluate all transitive deps

Sure, it's a draft to see the impact on ITs.

@bowenliang123
Copy link
Contributor Author

Closing this PR. As potential breaking changes for requiring primaryKey prop in 0.13.0, the failed test logs:
cc @cxzl25

SparkHudiOperationSuite:
- get columns type *** FAILED ***
  org.apache.kyuubi.jdbc.hive.KyuubiSQLException: org.apache.kyuubi.KyuubiSQLException: Error operating ExecuteStatement: java.lang.IllegalArgumentException: No `primaryKey` is specified.
	at org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:40)
	at org.apache.spark.sql.hudi.HoodieOptionConfig$.validateTable(HoodieOptionConfig.scala:199)
	at org.apache.spark.sql.catalyst.catalog.HoodieCatalogTable.parseSchemaAndConfigs(HoodieCatalogTable.scala:260)
	at org.apache.spark.sql.catalyst.catalog.HoodieCatalogTable.initHoodieTable(HoodieCatalogTable.scala:173)

@bowenliang123 bowenliang123 deleted the hudi-0.13.0 branch March 21, 2023 14:30
@cxzl25
Copy link
Contributor

cxzl25 commented Mar 22, 2023

Upgrading Hudi should re-evaluate all transitive deps

Let's continue to split the hudi module test.

#4260

@cxzl25
Copy link
Contributor

cxzl25 commented Mar 22, 2023

java.lang.IllegalArgumentException: No primaryKey is specified.

 options (
   primaryKey = 'XXX',

Mandatory Record Key Field
The configuration for setting the record key field, hoodie.datasource.write.recordkey.field, is now required to be set and has no default value. Previously, the default value is uuid.

https://hudi.apache.org/releases/release-0.13.0/#mandatory-record-key-field

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants