Skip to content

Commit 83d0967

Browse files
Ngone51jiangxb1987
authored andcommitted
[SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
### What changes were proposed in this pull request? Change from `messages.toList.iterator` to `Iterator.single(messages.toList)`. ### Why are the changes needed? In this test, the expected result of `rdd2.collect().head` should actually be `List("0", "1", "2", "3")` but is `"0"` now. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Updated test. Thanks WeichenXu123 reported this problem. Closes #28596 from Ngone51/fix_allgather_test. Authored-by: yi.wu <[email protected]> Signed-off-by: Xingbo Jiang <[email protected]>
1 parent 60118a2 commit 83d0967

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

core/src/test/scala/org/apache/spark/scheduler/BarrierTaskContextSuite.scala

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -69,12 +69,12 @@ class BarrierTaskContextSuite extends SparkFunSuite with LocalSparkContext with
6969
// Pass partitionId message in
7070
val message: String = context.partitionId().toString
7171
val messages: Array[String] = context.allGather(message)
72-
messages.toList.iterator
72+
Iterator.single(messages.toList)
7373
}
74-
// Take a sorted list of all the partitionId messages
75-
val messages = rdd2.collect().head
76-
// All the task partitionIds are shared
77-
for((x, i) <- messages.view.zipWithIndex) assert(x.toString == i.toString)
74+
val messages = rdd2.collect()
75+
// All the task partitionIds are shared across all tasks
76+
assert(messages.length === 4)
77+
assert(messages.forall(_ == List("0", "1", "2", "3")))
7878
}
7979

8080
test("throw exception if we attempt to synchronize with different blocking calls") {

0 commit comments

Comments
 (0)