Skip to content

Conversation

@ajantha-bhat
Copy link
Member

@ajantha-bhat ajantha-bhat commented Apr 11, 2023

#7024 didn't really fix the issue.

After analyzing the dependency tree found that com.palantir.baseline-checkstyle plugin will always use the checkstyle version from com.palantir.baseline:gradle-baseline-java:4.42.0 (which is 9.1) even though we override it with some other version (which is 9.3) using classpath.

So, the solution is to override using the toolchain version of the CheckstyleExtension as supported from com.palantir.baseline-checkstyle instead of classpath.
https://github.com/palantir/gradle-baseline/blob/267dd19dad6dbcc9d9772437340b4abc72deda55/gradle-baseline-java/src/main/groovy/com/palantir/baseline/plugins/BaselineCheckstyle.java#L43-L45

  • Executed the dependency tree gradle iceberg-api:dependencies and confirmed it is using checkstyle 9.3 which has the fix of OOM issue which is causing build flakiness.
  • Fixed the check style errors reported from 9.3 version (which was not present in 9.1 version)

fixes #7292

@ajantha-bhat
Copy link
Member Author

cc: @nastra, @Fokko, @stevenzwu, @jackye1995 , @XN137

@ajantha-bhat
Copy link
Member Author

ajantha-bhat commented Apr 11, 2023

Oops. This was perfectly compiled locally. But failed in the CI build.
Let me continue the analysis tomorrow.

@ajantha-bhat ajantha-bhat marked this pull request as draft April 11, 2023 16:45
@ajantha-bhat ajantha-bhat changed the title Build: Fix flaky checkstyle failure Build: Fix flaky checkstyle issue Apr 11, 2023
baseline.gradle Outdated
apply plugin: 'com.palantir.baseline-checkstyle'
// com.palantir.baseline:gradle-baseline-java:4.42.0 (the last version supporting Java 8) pulls
// in an old version of the checkstyle(9.1), which has this OutOfMemory bug https://github.com/checkstyle/checkstyle/issues/10934
// So, replace com.palantir.baseline-checkstyle plugin usage with gradle checkstyle plugin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, replace "com.palantir.baseline-checkstyle" with "checkstyle" 9.3 (the latest java 8 supported version) which contains a fix

@ajantha-bhat ajantha-bhat marked this pull request as ready for review April 12, 2023 09:07
org.apache.avro.Schema.create(org.apache.avro.Schema.Type.INT));
field = new org.apache.avro.Schema.Field("time_field", fieldSchema);
updatedField = new org.apache.avro.Schema.Field("time_field", fieldSchema);
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe init updatedField = field at the beginning rather than having an else

@ajantha-bhat
Copy link
Member Author

PR is ready for review.

@nastra nastra requested a review from Fokko April 12, 2023 10:18
@szehon-ho
Copy link
Member

Merged, let's see if this fixes it, thanks @ajantha-bhat for investigation, and @nastra for review

manisin pushed a commit to Snowflake-Labs/iceberg that referenced this pull request May 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Intermittent CheckStyle Failure on SparkParquetReaders

3 participants