Skip to content

Commit a63cdc7

Browse files
rxinAndrew Or
authored andcommitted
[SPARK-10612] [SQL] Add prepare to LocalNode.
The idea is that we should separate the function call that does memory reservation (i.e. prepare) from the function call that consumes the input (e.g. open()), so all operators can be a chance to reserve memory before they are all consumed. Author: Reynold Xin <[email protected]> Closes #8761 from rxin/SPARK-10612.
1 parent b6e9986 commit a63cdc7

File tree

1 file changed

+8
-0
lines changed
  • sql/core/src/main/scala/org/apache/spark/sql/execution/local

1 file changed

+8
-0
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/local/LocalNode.scala

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,14 @@ abstract class LocalNode(conf: SQLConf) extends TreeNode[LocalNode] with Logging
4545

4646
def output: Seq[Attribute]
4747

48+
/**
49+
* Called before open(). Prepare can be used to reserve memory needed. It must NOT consume
50+
* any input data.
51+
*
52+
* Implementations of this must also call the `prepare()` function of its children.
53+
*/
54+
def prepare(): Unit = children.foreach(_.prepare())
55+
4856
/**
4957
* Initializes the iterator state. Must be called before calling `next()`.
5058
*

0 commit comments

Comments
 (0)