Skip to content

Commit 87a7c4f

Browse files
advancedxypwendell
authored andcommitted
[SPARK-1511] use Files.move instead of renameTo in TestUtils.scala
JIRA issue:[SPARK-1511](https://issues.apache.org/jira/browse/SPARK-1511) TestUtils.createCompiledClass method use renameTo() to move files which fails when the src and dest files are in different disks or partitions. This pr uses Files.move() instead. The move method will try to use renameTo() and then fall back to copy() and delete(). I think this should handle this issue. I didn't found a test suite for this file, so I add file existence detection after file moving. Author: Ye Xianjin <[email protected]> Closes #427 from advancedxy/SPARK-1511 and squashes the following commits: a2b97c7 [Ye Xianjin] Based on @srowen's comment, assert file existence. 6f95550 [Ye Xianjin] use Files.move instead of renameTo to handle the src and dest files are in different disks or partitions. (cherry picked from commit 10b1c59) Signed-off-by: Patrick Wendell <[email protected]>
1 parent d8fc4a4 commit 87a7c4f

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed

core/src/main/scala/org/apache/spark/TestUtils.scala

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -100,9 +100,14 @@ private[spark] object TestUtils {
100100

101101
val fileName = className + ".class"
102102
val result = new File(fileName)
103-
if (!result.exists()) throw new Exception("Compiled file not found: " + fileName)
103+
assert(result.exists(), "Compiled file not found: " + result.getAbsolutePath())
104104
val out = new File(destDir, fileName)
105-
result.renameTo(out)
105+
106+
// renameTo cannot handle in and out files in different filesystems
107+
// use google's Files.move instead
108+
Files.move(result, out)
109+
110+
assert(out.exists(), "Destination file not moved: " + out.getAbsolutePath())
106111
out
107112
}
108113
}

0 commit comments

Comments
 (0)