Commit 52b6cd2
committed
[SPARK-3889] Attempt to avoid SIGBUS by not mmapping files in ConnectionManager
In general, individual shuffle blocks are frequently small, so mmapping them
often creates a lot of waste. It may not be bad to mmap the larger ones, but
it is pretty inconvenient to get configuration into ManagedBuffer, and besides
it is unlikely to help all that much.
Note that user of ManagedBuffer#nioByteBuffer() seems generally bad practice,
and would ideally never be used for data that may be large. Users of such data
would ideally stream the data instead.1 parent 4e9b551 commit 52b6cd2
File tree
1 file changed
+4
-1
lines changed- core/src/main/scala/org/apache/spark/network
1 file changed
+4
-1
lines changedLines changed: 4 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
75 | | - | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
76 | 79 | | |
77 | 80 | | |
78 | 81 | | |
| |||
0 commit comments