[WIP]: Spark 51272 51016 combined: For testing of HA Test#50263

Closed

ahshahid wants to merge 66 commits intoapache:masterfrom

ahshahid:SPARK-51272--51016-combined

Commits on Feb 20, 2025

SPARK-51016. Fixing the code of isInDeterminate boolean for a Stage
ashahid
committed
Merge branch 'upstream-master' into SPARK-51016
ashahid
committed
SPARK-51016. updating comment
ashahid
committed

Commits on Feb 21, 2025

SPARK-51272. Fix for the race condition in Scheduler causing failure in retrying all partitions in case of indeterministic shuffle keys
ashahid
committed

Commits on Feb 22, 2025

SPARK-51272. added a reliable bug test demonstrating the race condition which will fail if all partitions are not retried for indeterministic shuffle stage
ashahid
committed

Commits on Feb 23, 2025

SPARK-51272. fixed comment
ashahid
committed

Commits on Feb 27, 2025

Merge branch 'upstream-master' into SPARK-51016
ashahid
committed
Merge branch 'upstream-master' into SPARK-51272
ashahid
committed

Commits on Feb 28, 2025

Commits on Mar 1, 2025

Merge branch 'upstream-master' into SPARK-51016
ashahid
committed

Commits on Mar 3, 2025

SPARK-51016. refcatored the tests to use inner joins instead of Left Outer to replicate the bug.
ashahid
committed

Commits on Mar 4, 2025

Commits on Mar 6, 2025

initial changes
ashahid
committed
SPARK-51272. incorporating review feedback
ashahid
committed
SPARK-51272. incorporating review feedback. removing explict invocation of beforeEach and afterEach in test
ashahid
committed
initial changes
ashahid
committed
SPARK-51272. rectified a comment and renamed the test
ashahid
committed

Commits on Mar 7, 2025

initial changes
ashahid
committed
SPARK-51272. incorporated review feedback to refactor test
ashahid
committed
SPARK-51272. incorporated review feedback to refactor test
ashahid
committed
initial changes
ashahid
committed

Commits on Mar 10, 2025

initial changes
ashahid
committed
initial changes
ashahid
committed
SPARK-51272. implemented review feedback. refactored code. no longer passing attempt id as parameter
ashahid
committed

Commits on Mar 11, 2025

SPARK-51272. Fixed the situation where ResultStage is inDeterminate but because the ShuffleMap stage which has failed is deterministic, it misses the situation where possibly the other shuffle stag…
ashahid
committed

Commits on Mar 12, 2025

Commits on Mar 13, 2025

SPARK-51272. formatting change
ashahid
committed
Merge branch 'SPARK-51016' into SPARK-51272--51016-combined
ashahid
committed
Merge branch 'SPARK-51272' into SPARK-51272--51016-combined
ashahid
committed
SPARK-51272. SPARK-51016. combined PRs with HA Test enabled
ashahid
committed
SPARK-51272. Increased the number of iterations. Using two positions ( stage submitted and stage completed to kill the executors)
ashahid
committed
Merge branch 'SPARK-51272' into SPARK-51272--51016-combined
ashahid
committed
Merge branch 'upstream-master' into SPARK-51016
ashahid
committed
SPARK-51272. fixed scalastyle issue
ashahid
committed
Merge branch 'SPARK-51272' into SPARK-51272--51016-combined
ashahid
committed

Commits on Mar 14, 2025

Commits on Mar 15, 2025

Commits on Mar 17, 2025

Commits on Mar 19, 2025

SPARK-51016. Implemented review feedback. Thank you @squito
ashahid
committed

Commits on Mar 20, 2025

Commits on Apr 3, 2025