Skip to content

Comments

[GLUTEN-9382][VL]Support bucket write with non partition table#9575

Merged
JkSelf merged 1 commit intoapache:mainfrom
JkSelf:bucket-non-partition
Jun 5, 2025
Merged

[GLUTEN-9382][VL]Support bucket write with non partition table#9575
JkSelf merged 1 commit intoapache:mainfrom
JkSelf:bucket-non-partition

Conversation

@JkSelf
Copy link

@JkSelf JkSelf commented May 9, 2025

What changes were proposed in this pull request?

Fix #9382

How was this patch tested?

Existing fallback unit tests

@github-actions
Copy link

github-actions bot commented May 9, 2025

#9382

@FelixYBW
Copy link
Contributor

Depends on velox PR facebookincubator/velox#13283

@zhouyuan
Copy link
Member

@JkSelf
it seems the failed UTs are related:
https://github.com/apache/incubator-gluten/actions/runs/14920730616/job/41916923311?pr=9575

2025-05-09T04:36:40.6499130Z 'Gluten - determining the number of reducers: plan already partitioned' offload to gluten
2025-05-09T04:36:40.6507895Z - Gluten - determining the number of reducers: plan already partitioned *** FAILED ***

@JkSelf
Copy link
Author

JkSelf commented May 14, 2025

@JkSelf it seems the failed UTs are related: https://github.com/apache/incubator-gluten/actions/runs/14920730616/job/41916923311?pr=9575

2025-05-09T04:36:40.6499130Z 'Gluten - determining the number of reducers: plan already partitioned' offload to gluten
2025-05-09T04:36:40.6507895Z - Gluten - determining the number of reducers: plan already partitioned *** FAILED ***

@zhouyuan Yes, I am currently working on resolving the failed unit tests. It appears there are some discrepancies in how Bucket reads are handled between Spark and Velox.

@JkSelf JkSelf force-pushed the bucket-non-partition branch from b8fffc7 to db1da82 Compare May 20, 2025 01:49
@github-actions github-actions bot added CORE works for Gluten Core INFRA labels May 20, 2025
@github-actions
Copy link

Run Gluten Clickhouse CI on x86

@JkSelf JkSelf force-pushed the bucket-non-partition branch from db1da82 to 6497d57 Compare May 20, 2025 03:04
@github-actions github-actions bot removed the INFRA label May 20, 2025
@github-actions
Copy link

Run Gluten Clickhouse CI on x86

@JkSelf JkSelf force-pushed the bucket-non-partition branch from 6497d57 to 7a8e213 Compare May 20, 2025 05:04
@github-actions github-actions bot removed the CORE works for Gluten Core label May 20, 2025
@JkSelf
Copy link
Author

JkSelf commented May 20, 2025

@wangyum Can you help to verify whether #9382 can be fixed in your env? Thanks.

Copy link
Member

@PHILO-HE PHILO-HE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@JkSelf JkSelf merged commit c5febd8 into apache:main Jun 5, 2025
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[VL] Unsupported native write: Non-compatible hive bucket write is not supported

4 participants