Skip to content

[HUDI-4780] Improve test setup #6704

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
loukey-lj wants to merge 11 commits into from
Closed

[HUDI-4780] Improve test setup #6704

Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
ee9ae14
Merge pull request #1 from apache/master
loukey-lj Jan 26, 2021
3fb10f7
Merge branch 'apache:master' into master
loukey-lj May 30, 2021
0fa6348
Merge branch 'apache:master' into master
loukey-lj Feb 16, 2022
9427b43
Merge branch 'apache:master' into master
loukey-lj Feb 17, 2022
99bf8f2
Merge branch 'apache:master' into master
loukey-lj Feb 17, 2022
83f0414
Merge branch 'apache:master' into master
loukey-lj May 17, 2022
fb01262
Merge branch 'apache:master' into master
loukey-lj May 22, 2022
e1dfa9d
Merge branch 'apache:master' into master
loukey-lj Jun 15, 2022
c35a685
Merge branch 'apache:master' into master
loukey-lj Aug 25, 2022
05caed5
Merge branch 'apache:master' into master
loukey-lj Sep 5, 2022
d95e07d
Add test for HUDI-4780
Sep 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions hudi-common/src/test/java/org/apache/hudi/common/functional/TestHoodieLogFormat.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import org.apache.hudi.common.model.HoodieLogFile;
import org.apache.hudi.common.model.HoodieRecord;
import org.apache.hudi.common.model.HoodieTableType;
import org.apache.hudi.common.table.HoodieTableMetaClient;
import org.apache.hudi.common.table.log.AppendResult;
import org.apache.hudi.common.table.log.HoodieLogFileReader;
import org.apache.hudi.common.table.log.HoodieLogFormat;
Expand Down Expand Up @@ -124,12 +125,13 @@ public static void tearDownClass() {

@BeforeEach
public void setUp() throws IOException, InterruptedException {
this.fs = MiniClusterUtil.fileSystem;
FileSystem fileSystemOriginal = MiniClusterUtil.fileSystem;

assertTrue(fs.mkdirs(new Path(tempDir.toAbsolutePath().toString())));
assertTrue(fileSystemOriginal.mkdirs(new Path(tempDir.toAbsolutePath().toString())));
this.partitionPath = new Path(tempDir.toAbsolutePath().toString());
this.basePath = tempDir.getParent().toString();
HoodieTestUtils.init(MiniClusterUtil.configuration, basePath, HoodieTableType.MERGE_ON_READ);
HoodieTableMetaClient init = HoodieTestUtils.init(MiniClusterUtil.configuration, basePath, HoodieTableType.MERGE_ON_READ);
this.fs = init.getFs();
Comment on lines +128 to +134
Copy link
Contributor

@yihua yihua Sep 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@loukey-lj Could you clarify how the changes improve the tests for #6602 you put up?

Copy link
Contributor Author

@loukey-lj loukey-lj Sep 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the actual running process fs comes from TableMateClinet.So I initialized a TableMateClinetto get fs instead of fileSystemOriginal.

Copy link
Contributor

@yihua yihua Sep 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Could you point out which set of tests fail without your previous fix (#6602) after this PR? We should add some comments on these tests mentioning that they test the logic around creating new log files based the size configured. If there are no such test, we should add one. Let me know if this makes sense. We can sync up through Hudi Slack.

}

@AfterEach
Expand Down