ARROW-15281: [C++] Implement ability to retrieve fragment filename #12560
Conversation
python/pyarrow/_dataset.pyx
Outdated
There was a problem hiding this comment.
C++ may return an error, but Python will raise an exception, right?
There was a problem hiding this comment.
Yes I think so, I checked that again.
If we see here
https://github.com/apache/arrow/blob/master/python/pyarrow/_dataset.pyx#L2014
then this goes to here,
https://github.com/apache/arrow/blob/master/python/pyarrow/error.pxi#L99
which raises an ArrowInvalid exception. So, it might be better to mention that it raises an exception here.
cc: @westonpace
There was a problem hiding this comment.
Yikes, yes this was totally my fault. My original suggestion was bad. Thank you for switching it back.
…gment, used ToString()
sanjibansg
force-pushed
the
filename_retrieval
branch
from
56fa37f to
55b6aea
Compare
lidavidm
left a comment
lidavidm
left a comment
There was a problem hiding this comment.
Thanks, just a minor suggestion
|
Failure in hash-join-node-test is unrelated, but interesting. I've added it to ARROW-15221. |
|
Benchmark runs are scheduled for baseline = 600288e and contender = 3eb1bac. 3eb1bac is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
4 participants
This PR adds the feature to include the filename of a fragment if it is a FileFragment as a scalar value and includes the augmented fields in the dataset schema