AWS: Support RangeReadable in Analytics Accelerator Stream #13361
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jun 20, 2025
Merged
SanjayMarreddi
added a commit
to awslabs/analytics-accelerator-s3
that referenced
this pull request
Jun 25, 2025
## Description of change We want to support `readFully` as a part of our ongoing effort to integrate with Iceberg S3FileIO by default. [RangeReadable in Iceberg](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/io/RangeReadable.java#L47) **Note:** Here's the comparison with existing read method: | Aspect | `read(byte[] buffer, int offset, int length)` | `readFully(long position, byte[] buffer, int offset, int length)` | |--------|----------------------------------------------|-------------------------------------------------------------------| | Method Signature | Reads from current stream position | Reads from specified position | | Position Behavior | Advances the stream position by the number of bytes actually read | Does not modify the stream position (position-independent read) | | Return & Error Handling | Returns `int` indicating bytes read; returns -1 at end of stream; may return fewer bytes than requested | Returns `void`; throws `IOException` if unable to read exact number of requested bytes | #### Relevant issues Once this feature is released as `1.2.0`, we will update this [Iceberg PR](apache/iceberg#13361) #### Does this contribution introduce any breaking changes to the existing APIs or behaviors? No #### Does this contribution introduce any new public APIs or behaviors? Yes #### How was the contribution tested? Ran existing and new tests locally #### Does this contribution need a changelog entry? - [x] I have updated the CHANGELOG or README if appropriate --- By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the [Developer Certificate of Origin (DCO)](https://developercertificate.org/).
ozkoca
pushed a commit
to awslabs/analytics-accelerator-s3
that referenced
this pull request
Jun 26, 2025
## Description of change We want to support `readFully` as a part of our ongoing effort to integrate with Iceberg S3FileIO by default. [RangeReadable in Iceberg](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/io/RangeReadable.java#L47) **Note:** Here's the comparison with existing read method: | Aspect | `read(byte[] buffer, int offset, int length)` | `readFully(long position, byte[] buffer, int offset, int length)` | |--------|----------------------------------------------|-------------------------------------------------------------------| | Method Signature | Reads from current stream position | Reads from specified position | | Position Behavior | Advances the stream position by the number of bytes actually read | Does not modify the stream position (position-independent read) | | Return & Error Handling | Returns `int` indicating bytes read; returns -1 at end of stream; may return fewer bytes than requested | Returns `void`; throws `IOException` if unable to read exact number of requested bytes | #### Relevant issues Once this feature is released as `1.2.0`, we will update this [Iceberg PR](apache/iceberg#13361) #### Does this contribution introduce any breaking changes to the existing APIs or behaviors? No #### Does this contribution introduce any new public APIs or behaviors? Yes #### How was the contribution tested? Ran existing and new tests locally #### Does this contribution need a changelog entry? - [x] I have updated the CHANGELOG or README if appropriate --- By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the [Developer Certificate of Origin (DCO)](https://developercertificate.org/).
SanjayMarreddi
force-pushed
the
support_range_reads_in_AAL_stream
branch
from
371918f to
9ae820a
Compare
…erator is enabled
… is either enabled/disabled
SanjayMarreddi
force-pushed
the
support_range_reads_in_AAL_stream
branch
from
9ae820a to
a85ef81
Compare
stubz151
approved these changes
Jul 8, 2025
stubz151
approved these changes
Jul 8, 2025
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is a follow-up to the existing PRs #13347 and #13348 in the series addressing feature gaps in the stream powered by analytics-accelerator-s3. It's part of our ongoing effort towards default-on integration. This PR specifically adds support for range-based reads.
Testing
When
s3.analytics-accelerator.enabledis set totrue, previously skipped integration testtestRangeReadworks now.