Skip to content

[HUDI-5333] Ignore file system type of basePath when using RocksDbBasedFileSystemView #7387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
bvaradar merged 8 commits into from
Mar 10, 2023
Merged

[HUDI-5333] Ignore file system type of basePath when using RocksDbBasedFileSystemView #7387

bvaradar merged 8 commits into from
Mar 10, 2023

Conversation

@1032851561
Copy link
Contributor

@1032851561 1032851561 commented Dec 6, 2022

Change Logs

In RocksDbBasedFileSystemView, the rocksDBBasePath is base on table basePath , because of special characters ":" , it will fail to run under Windows system.

  • exception:
Caused by: java.io.IOException: Unable to create :\tmp\hoodie_timeline_rocksdb\file:_D:_data_warehouse_tb_test\f6b39e9e-f00c-400a-897e-ca2f72c244ba
	at org.apache.hudi.common.util.FileIOUtils.mkdir(FileIOUtils.java:70)
	at org.apache.hudi.common.util.collection.RocksDBDAO.init(RocksDBDAO.java:107)
  • table options:
'hoodie.filesystem.view.type' = 'EMBEDDED_KV_STORE',
'path' = 'file:///D:/data/warehouse/tb_test'

Impact

None.

Risk level (write none, low medium or high below)

None.

Documentation Update

None.

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@1032851561 1032851561 changed the title [BUG] Ignore file system type of basePath when using RocksDbBasedFileSystemView [HUDI-5333] Ignore file system type of basePath when using RocksDbBasedFileSystemView Dec 6, 2022
@hudi-bot
Copy link
Collaborator

hudi-bot commented Dec 6, 2022

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@bvaradar bvaradar self-assigned this Feb 18, 2023
bvaradar
bvaradar approved these changes Feb 28, 2023
View reviewed changes
Copy link
Contributor

@bvaradar bvaradar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@bvaradar bvaradar merged commit 8129a85 into apache:master Mar 10, 2023
nsivabalan pushed a commit to nsivabalan/hudi that referenced this pull request Mar 18, 2023
@1032851561 @nsivabalan
[HUDI-5333] Ignore file system type of basePath when using RocksDbBas…
87ae8ae 
…edFileSystemView (apache#7387)

Co-authored-by: chenzhiming <[email protected]>
nsivabalan pushed a commit to nsivabalan/hudi that referenced this pull request Mar 22, 2023
@1032851561 @nsivabalan
[HUDI-5333] Ignore file system type of basePath when using RocksDbBas…
168892b 
…edFileSystemView (apache#7387)

Co-authored-by: chenzhiming <[email protected]>
fengjian428 pushed a commit to fengjian428/hudi that referenced this pull request Apr 5, 2023
@1032851561
[HUDI-5333] Ignore file system type of basePath when using RocksDbBas…
fdb85e0 
…edFileSystemView (apache#7387)

Co-authored-by: chenzhiming <[email protected]>
stayrascal pushed a commit to stayrascal/hudi that referenced this pull request Apr 20, 2023
@1032851561 @stayrascal
[HUDI-5333] Ignore file system type of basePath when using RocksDbBas…
bbb122b 
…edFileSystemView (apache#7387)

Co-authored-by: chenzhiming <[email protected]>
KnightChess pushed a commit to KnightChess/hudi that referenced this pull request Jan 2, 2024
@1032851561 @KnightChess
[HUDI-5333] Ignore file system type of basePath when using RocksDbBas…
dff0f45 
…edFileSystemView (apache#7387)

Co-authored-by: chenzhiming <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bvaradar bvaradar bvaradar approved these changes

Assignees

@bvaradar bvaradar

Labels

None yet

Projects

Hudi PR Support
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@1032851561 @hudi-bot @bvaradar