Skip to content

ARROW-18237: [Java] Extend Table code #14573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
lidavidm merged 22 commits into from
Nov 29, 2022
Merged

ARROW-18237: [Java] Extend Table code #14573

lidavidm merged 22 commits into from
Nov 29, 2022

Conversation

@lwhite1
Copy link
Contributor

@lwhite1 lwhite1 commented Nov 2, 2022

Initial merge of MutableTable / MutableRow plus tests.

@github-actions
Copy link

github-actions bot commented Nov 2, 2022

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW

Opening JIRAs ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename pull request title in the following format?

ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

@lwhite1 lwhite1 changed the title Merge and extend MutableTable code ARROW-18237: [Java] Merge and extend MutableTable code Nov 3, 2022
@lwhite1 lwhite1 marked this pull request as ready for review November 8, 2022 17:14
@lwhite1
Copy link
Contributor Author

lwhite1 commented Nov 8, 2022 •
edited
Loading

@davisusanibar Would you please review? This is a partial implementation but it's already quite large. The code is mostly very simple and well tested. I think before adding anything more it would be best to review and merge what's here

davisusanibar
davisusanibar approved these changes Nov 14, 2022
View reviewed changes
Copy link
Contributor

@davisusanibar davisusanibar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for all the documentation added on the methods/classes (+TODO).

@lwhite1 lwhite1 marked this pull request as draft November 29, 2022 13:08
@lwhite1 lwhite1 changed the title ARROW-18237: [Java] Merge and extend MutableTable code ARROW-18237: [Java] Extend Table code Nov 29, 2022
@github-actions
Copy link

github-actions bot commented Nov 29, 2022

https://issues.apache.org/jira/browse/ARROW-18237

@github-actions
Copy link

github-actions bot commented Nov 29, 2022

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@lwhite1 lwhite1 marked this pull request as ready for review November 29, 2022 17:58
@lwhite1
Copy link
Contributor Author

lwhite1 commented Nov 29, 2022

@lidavidm Could you please give this a final check and merge? It was originally intended to include new code for mutable table support but has been simplified to include adding a small number of necessary methods to the existing classes.

lidavidm
lidavidm approved these changes Nov 29, 2022
View reviewed changes
java/vector/src/main/java/org/apache/arrow/vector/table/Row.java Outdated
import org.apache.arrow.vector.holders.NullableUInt8Holder;

/**
* TODO: Modify the getters for Duration and others so that they return something better than ArrowBuf when possible
Copy link
Member

@lidavidm lidavidm Nov 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to leave this TODO in here? (It should perhaps be linked to an issue?)

Copy link
Contributor Author

@lwhite1 lwhite1 Nov 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed obsolete TODOs

@lidavidm lidavidm merged commit 2e9611a into apache:master Nov 29, 2022
@ursabot
Copy link

ursabot commented Nov 30, 2022

Benchmark runs are scheduled for baseline = 3b0e135 and contender = 2e9611a. 2e9611a is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️25.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.1% ⬆️0.0%] test-mac-arm
[Finished ⬇️0.27% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.17% ⬆️0.03%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 2e9611a8 ec2-t3-xlarge-us-east-2
[Failed] 2e9611a8 test-mac-arm
[Finished] 2e9611a8 ursa-i9-9960x
[Finished] 2e9611a8 ursa-thinkcentre-m75q
[Finished] 3b0e1357 ec2-t3-xlarge-us-east-2
[Failed] 3b0e1357 test-mac-arm
[Finished] 3b0e1357 ursa-i9-9960x
[Finished] 3b0e1357 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@asfimport asfimport mentioned this pull request Nov 30, 2022
Closed
@Yicong-Huang Yicong-Huang mentioned this pull request May 1, 2023
Merged
Yicong-Huang added a commit to apache/texera that referenced this pull request May 5, 2023
@Yicong-Huang
Bump Apache Arrow to 11.0.0 (#1900)
e26c96e 
This PR bumps Apache Arrow version from 10.0.0 to 11.0.0.

Main changes related to PyAmber:

## Java/Scala side:

- Distribute Apple M1 compatible JNI libraries via mavencentral
([#14472](apache/arrow#14472)).
- Improve performance by short-circuiting null checks when comparing non
null field types ([#15106](apache/arrow#15106)).
- Extend Table copy functionality, and support returning copies of
individual vectors
([#14389](apache/arrow#14389)).
- Several enhancements to dictionary encoding
([#14891](apache/arrow#14891),
([#14902](apache/arrow#14902),
([#14874](apache/arrow#14874)).
- Extend Table to support additional vector types
([#14573](apache/arrow#14573)).
- Enhance and simplify handling of allocation management by integrating
C Data into allocator hierarchy
([#14506](apache/arrow#14506)).

## Python side:
- PyArrow now requires pandas >= 1.0
([ARROW-18173](https://issues.apache.org/jira/browse/ARROW-18173)).
- Added support for the [DataFrame Interchange
Protocol](https://data-apis.org/dataframe-protocol/latest/purpose_and_scope.html)
for pyarrow.Table
([GH-33346](apache/arrow#33346)).
- Support for custom metadata of record batches in the IPC read and
write APIs
([ARROW-16430](https://issues.apache.org/jira/browse/ARROW-16430)).
- The Time32Scalar, Time64Scalar, Date32Scalar and Date64Scalar classes
got a .value attribute to access the underlying integer value, similar
to the other date-time related scalars
([ARROW-18264](https://issues.apache.org/jira/browse/ARROW-18264)).
- Casting to string is now supported for duration
([ARROW-15822](https://issues.apache.org/jira/browse/ARROW-15822)) and
decimal
([ARROW-17458](https://issues.apache.org/jira/browse/ARROW-17458))
types, which also means those can now be written to CSV.

## Issues fixed:
- Now Do_action (from Python server back to Java Client) is returning a
stream of results properly, and it alerts when the results are not fully
consumed by the client. Such results will be used to send the flow
control credits back from the Python side. We limit the results to be
exact 1 for now, although it can be a stream.
- Fix a bug in the Python proxy server, when unregistered action is
invoked, it should not parse and return the results.
pribor pushed a commit to GlobalWebIndex/arrow that referenced this pull request Oct 24, 2025
@lwhite1 @kou
ARROW-18237: [Java] Extend Table code (apache#14573)
2310270 
Initial merge of MutableTable / MutableRow plus tests. 

Authored-by: Larry White <[email protected]>
Signed-off-by: David Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lidavidm lidavidm lidavidm approved these changes

+1 more reviewer

@davisusanibar davisusanibar davisusanibar approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Component: Java

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@lwhite1 @ursabot @lidavidm @davisusanibar