Skip to content

[SPARK-48358][SQL] Support for REPEAT statement #47756

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dusantism-db wants to merge 12 commits into from
Closed

[SPARK-48358][SQL] Support for REPEAT statement #47756

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
c67d6ca
add initial repeat statement
dusantism-db Aug 13, 2024
e6e27ec
scripting interpreter suite tests for repeat
dusantism-db Aug 13, 2024
391ae32
scripting execution node suite tests for repeat
dusantism-db Aug 13, 2024
8719f75
scripting parser suite tests for repeat
dusantism-db Aug 13, 2024
934e952
add REPEAT and UNTIL keywords
dusantism-db Aug 13, 2024
75ac771
remove unnecessary assert
dusantism-db Aug 14, 2024
a6ca945
refactor execution tests for while and repeat to use shared logic
dusantism-db Sep 2, 2024
8e8e690
add leave and iterate support for repeat statement + tests
dusantism-db Sep 6, 2024
c21d782
finish tests for iterate and leave repeat statement
dusantism-db Sep 6, 2024
e99566a
fix formatting
dusantism-db Sep 9, 2024
51080c2
add negative tests
dusantism-db Sep 9, 2024
27ee356
address comments
dusantism-db Sep 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/sql-ref-ansi-compliance.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -646,6 +646,7 @@ Below is a list of all the keywords in Spark SQL.
|REGEXP|non-reserved|non-reserved|not a keyword|
|RENAME|non-reserved|non-reserved|non-reserved|
|REPAIR|non-reserved|non-reserved|non-reserved|
|REPEAT|non-reserved|non-reserved|non-reserved|
|REPEATABLE|non-reserved|non-reserved|non-reserved|
|REPLACE|non-reserved|non-reserved|non-reserved|
|RESET|non-reserved|non-reserved|non-reserved|
Expand Down Expand Up @@ -734,6 +735,7 @@ Below is a list of all the keywords in Spark SQL.
|UNLOCK|non-reserved|non-reserved|non-reserved|
|UNPIVOT|non-reserved|non-reserved|non-reserved|
|UNSET|non-reserved|non-reserved|non-reserved|
|UNTIL|non-reserved|non-reserved|non-reserved|
|UPDATE|non-reserved|non-reserved|reserved|
|USE|non-reserved|non-reserved|non-reserved|
|USER|reserved|non-reserved|reserved|
Expand Down
2 changes: 2 additions & 0 deletions sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseLexer.g4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,7 @@ REFERENCES: 'REFERENCES';
REFRESH: 'REFRESH';
RENAME: 'RENAME';
REPAIR: 'REPAIR';
REPEAT: 'REPEAT';
REPEATABLE: 'REPEATABLE';
REPLACE: 'REPLACE';
RESET: 'RESET';
Expand Down Expand Up @@ -453,6 +454,7 @@ UNKNOWN: 'UNKNOWN';
UNLOCK: 'UNLOCK';
UNPIVOT: 'UNPIVOT';
UNSET: 'UNSET';
UNTIL: 'UNTIL';
UPDATE: 'UPDATE';
USE: 'USE';
USER: 'USER';
Expand Down
9 changes: 9 additions & 0 deletions sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ compoundStatement
| beginEndCompoundBlock
| ifElseStatement
| whileStatement
| repeatStatement
| leaveStatement
| iterateStatement
;
Expand All @@ -85,6 +86,10 @@ ifElseStatement
(ELSE elseBody=compoundBody)? END IF
;

repeatStatement
: beginLabel? REPEAT compoundBody UNTIL booleanExpression END REPEAT endLabel?
;

leaveStatement
: LEAVE multipartIdentifier
;
Expand Down Expand Up @@ -1660,6 +1665,7 @@ ansiNonReserved
| REFRESH
| RENAME
| REPAIR
| REPEAT
| REPEATABLE
| REPLACE
| RESET
Expand Down Expand Up @@ -1735,6 +1741,7 @@ ansiNonReserved
| UNLOCK
| UNPIVOT
| UNSET
| UNTIL
| UPDATE
| USE
| VALUES
Expand Down Expand Up @@ -2023,6 +2030,7 @@ nonReserved
| REFRESH
| RENAME
| REPAIR
| REPEAT
| REPEATABLE
| REPLACE
| RESET
Expand Down Expand Up @@ -2107,6 +2115,7 @@ nonReserved
| UNLOCK
| UNPIVOT
| UNSET
| UNTIL
| UPDATE
| USE
| USER
Expand Down
18 changes: 18 additions & 0 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,20 @@ class AstBuilder extends DataTypeAstBuilder
WhileStatement(condition, body, Some(labelText))
}

override def visitRepeatStatement(ctx: RepeatStatementContext): RepeatStatement = {
val labelText = generateLabelText(Option(ctx.beginLabel()), Option(ctx.endLabel()))
val boolExpr = ctx.booleanExpression()

val condition = withOrigin(boolExpr) {
SingleStatement(
Project(
Seq(Alias(expression(boolExpr), "condition")()),
OneRowRelation()))}
val body = visitCompoundBody(ctx.compoundBody())

RepeatStatement(condition, body, Some(labelText))
}

private def leaveOrIterateContextHasLabel(
ctx: RuleContext, label: String, isIterate: Boolean): Boolean = {
ctx match {
Expand All @@ -275,6 +289,10 @@ class AstBuilder extends DataTypeAstBuilder
if Option(c.beginLabel()).isDefined &&
c.beginLabel().multipartIdentifier().getText.toLowerCase(Locale.ROOT).equals(label)
=> true
case c: RepeatStatementContext
if Option(c.beginLabel()).isDefined &&
c.beginLabel().multipartIdentifier().getText.toLowerCase(Locale.ROOT).equals(label)
=> true
Comment on lines +292 to +295
Copy link
Contributor

@miland-db miland-db Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we merge this with the above case for WhileStatementContext? You have multiple options for this, using | operator or Either type.

Copy link
Contributor Author

@dusantism-db dusantism-db Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wouldn't be trivial, as there is no common supertype between RepeatStatementContext and WhileStatementContext with the beginLabel() method defined. Maybe we could have a labeledStatement grammar rule, or something similar, to abstract some of this label logic from all statements with labels (ifElse, while, repeat..)

case _ => false
}
}
Expand Down
17 changes: 17 additions & 0 deletions ...st/src/main/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingLogicalOperators.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,23 @@ case class WhileStatement(
body: CompoundBody,
label: Option[String]) extends CompoundPlanStatement

/**
* Logical operator for REPEAT statement.
* @param condition Any expression evaluating to a Boolean.
* Body is executed as long as the condition evaluates to false
* @param body Compound body is a collection of statements that are executed once no matter what,
* and then as long as condition is false.
* @param label An optional label for the loop which is unique amongst all labels for statements
* within which the LOOP statement is contained.
* If an end label is specified it must match the beginning label.
* The label can be used to LEAVE or ITERATE the loop.
*/
case class RepeatStatement(
condition: SingleStatement,
body: CompoundBody,
label: Option[String]) extends CompoundPlanStatement


/**
* Logical operator for LEAVE statement.
* The statement can be used both for compounds or any kind of loops.
Expand Down
Loading