Skip to content

Conversation

@beliefer
Copy link
Contributor

What changes were proposed in this pull request?

#32229 support ANSI SQL intervals by the aggregate function avg.
But have not treat that the input zero rows. so this will lead to:

Caused by: java.lang.ArithmeticException: / by zero
	at com.google.common.math.LongMath.divide(LongMath.java:367)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source)
	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
	at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:759)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
	at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1864)
	at org.apache.spark.rdd.RDD.$anonfun$count$1(RDD.scala:1253)
	at org.apache.spark.rdd.RDD.$anonfun$count$1$adapted(RDD.scala:1253)
	at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2248)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:131)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:498)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1437)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:501)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Why are the changes needed?

Fix a bug.

Does this PR introduce any user-facing change?

No. Just new feature.

How was this patch tested?

new tests.

@github-actions github-actions bot added the SQL label Apr 27, 2021
Copy link
Member

@MaxGekk MaxGekk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about to make PR title more precise:
[SPARK-34837][SQL][FOLLOWUP] Fix division by zero in the avg function over ANSI intervals

@beliefer beliefer changed the title [SPARK-34837][SQL][FOLLOWUP] Support ANSI SQL intervals by the aggregate function avg [SPARK-34837][SQL][FOLLOWUP] Fix division by zero in the avg function over ANSI intervals Apr 27, 2021
@SparkQA
Copy link

SparkQA commented Apr 27, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42503/

@SparkQA
Copy link

SparkQA commented Apr 27, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42503/

@MaxGekk
Copy link
Member

MaxGekk commented Apr 27, 2021

+1, LGTM. Merging to master.
Thank you, @beliefer .

@MaxGekk MaxGekk closed this in 55dea2d Apr 27, 2021
@SparkQA
Copy link

SparkQA commented Apr 27, 2021

Test build #137983 has finished for PR 32358 at commit 02d8282.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@beliefer
Copy link
Contributor Author

@MaxGekk Thanks for you review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants