TEZ-4450: Shuffle data fetch fails when shuffle data is transferred via CompositeRoutedDataMovementEvent #243
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
thanks for the patch @ganeshashree, could you please open a jira ticket for this? |
ganeshashree
changed the title
Contributor
Author
|
@abstractdog Jira ticket is https://issues.apache.org/jira/browse/TEZ-4450. Please review. |
ganeshashree
changed the title
ganeshashree
force-pushed
the
TEZ-4450
branch
from
32fb372 to
e248c03
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
…ia CompositeRoutedDataMovementEvent
ganeshashree
force-pushed
the
TEZ-4450
branch
from
e248c03 to
c934afb
Compare
|
🎊 +1 overall
This message was automatically generated. |
abstractdog
requested changes
Oct 23, 2022
Contributor
abstractdog
left a comment
abstractdog
left a comment
There was a problem hiding this comment.
looks good to me, only a minor question
| @@ -0,0 +1,13 @@ | |||
| # Licensed under the Apache License, Version 2.0 (the "License"); | |||
Contributor
There was a problem hiding this comment.
does this extra file have any advantages compared to adding this below as a dependency:
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
<version>xxx</version>
<scope>test</scope>
</dependency>
Contributor
Author
There was a problem hiding this comment.
@abstractdog Both options work, the difference is just the extra file versus extra dependency. As per this doc, mockito-inline artifact likely to be discontinued once mocking of final classes and methods gets integrated into the default mock maker. Please let me know if adding mockito-inline dependency looks better.
Contributor
There was a problem hiding this comment.
thanks for clarifying, I'm fine with this file then
prabhjyotsingh
pushed a commit
to acceldata-io/tez
that referenced
this pull request
Nov 11, 2024
…ia CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (cherry picked from commit 8ebc4b0)
prabhjyotsingh
pushed a commit
to acceldata-io/tez
that referenced
this pull request
Nov 12, 2024
…ia CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (cherry picked from commit 8ebc4b0)
prabhjyotsingh
pushed a commit
to acceldata-io/tez
that referenced
this pull request
Nov 12, 2024
…ia CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (cherry picked from commit 8ebc4b0) (cherry picked from commit 94da1f2)
prabhjyotsingh
pushed a commit
to acceldata-io/tez
that referenced
this pull request
Nov 12, 2024
…ia CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (cherry picked from commit 8ebc4b0) (cherry picked from commit 94da1f2)
prabhjyotsingh
pushed a commit
to acceldata-io/tez
that referenced
this pull request
Nov 12, 2024
…ia CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (cherry picked from commit 8ebc4b0) (cherry picked from commit 94da1f2)
prabhjyotsingh
pushed a commit
to acceldata-io/tez
that referenced
this pull request
Nov 12, 2024
…ia CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (cherry picked from commit 8ebc4b0) (cherry picked from commit 94da1f2)
prabhjyotsingh
pushed a commit
to acceldata-io/tez
that referenced
this pull request
Nov 12, 2024
…ia CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (cherry picked from commit 8ebc4b0) (cherry picked from commit 94da1f2)
prabhjyotsingh
pushed a commit
to acceldata-io/tez
that referenced
this pull request
Nov 20, 2024
…ia CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (cherry picked from commit 8ebc4b0) (cherry picked from commit 94da1f2) (cherry picked from commit 238df23)
prabhjyotsingh
added a commit
to acceldata-io/tez
that referenced
this pull request
Nov 20, 2024
…nsferred via CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (#20) (cherry picked from commit 8ebc4b0) (cherry picked from commit 94da1f2) (cherry picked from commit 238df23) Co-authored-by: Ganesha Shreedhara <[email protected]>
shubhluck
pushed a commit
to acceldata-io/tez
that referenced
this pull request
Nov 21, 2024
…nsferred via CompositeRoutedDataMovementEvent (apache#243) (Ganesha Shreedhara reviewed by Laszlo Bodor) (#20) (cherry picked from commit 8ebc4b0) (cherry picked from commit 94da1f2) (cherry picked from commit 238df23) Co-authored-by: Ganesha Shreedhara <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixed failure in fetching shuffle data when it's transferred via CompositeRoutedDataMovementEvent.
Added unit test to test the failure scenario.
Jira ticket: https://issues.apache.org/jira/browse/TEZ-4450