Skip to content

[SPARK-38126][SQL][TESTS] Check the whole message of error classes #35416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
MaxGekk wants to merge 4 commits into from
Closed

[SPARK-38126][SQL][TESTS] Check the whole message of error classes #35416

MaxGekk wants to merge 4 commits into from

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Feb 7, 2022 •
edited
Loading

What changes were proposed in this pull request?

In the PR, I propose to check the whole error messages of error classes in the test suites Query.*ErrorsSuite.

Why are the changes needed?

  1. To catch changes in error classes.
  2. To improve test coverage.
  3. Set user expectations from Spark's errors.
  4. The checks can be considered as documentation of error messages in tests.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

By running the modified test suites:

$ build/sbt "test:testOnly *QueryParsingErrorsSuite"
$ build/sbt "test:testOnly *QueryCompilationErrorsSuite"
$ build/sbt "test:testOnly *QueryCompilationErrorsDSv2Suite"
$ build/sbt "test:testOnly *QueryExecutionErrorsSuite"

@github-actions github-actions bot added the SQL label Feb 7, 2022
@MaxGekk MaxGekk changed the title [WIP][SPARK-38126][SQL] Check the whole message of error classes [SPARK-38126][SQL] Check the whole message of error classes Feb 7, 2022
@MaxGekk MaxGekk marked this pull request as ready for review February 7, 2022 09:40
@MaxGekk
Copy link
Member Author

MaxGekk commented Feb 7, 2022

@cloud-fan @imback82 @ivoson @SaurabhChawla100 @yutoacts @leesf Could you review this PR, please.

@MaxGekk MaxGekk changed the title [SPARK-38126][SQL] Check the whole message of error classes [SPARK-38126][SQL][TESTS] Check the whole message of error classes Feb 7, 2022
cloud-fan
cloud-fan reviewed Feb 7, 2022
View reviewed changes
sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala
Dataset.ofRows(spark, plan)
}.message
assert(msg.contains("The feature is not supported: " +
assert(msg.matches("The feature is not supported: " +
Copy link
Contributor

@cloud-fan cloud-fan Feb 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a user-facing error? The test constructs logical plan directly which is not user-facing.

Copy link
Member Author

@MaxGekk MaxGekk Feb 7, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exception was introduced by #28572. @Ngone51 Can it be triggered by user code?

Probably, it would be nice to revise all existing error classes/exceptions, and check their visibility to users.

Copy link
Member Author

@MaxGekk MaxGekk Feb 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened SPARK-38131 for the task.

@MaxGekk
Copy link
Member Author

MaxGekk commented Feb 7, 2022

Merging to master. Thank you, @cloud-fan for review.

@MaxGekk MaxGekk closed this in 65c0bdf Feb 7, 2022
HyukjinKwon
HyukjinKwon reviewed Feb 8, 2022
View reviewed changes
Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM2

@MaxGekk MaxGekk mentioned this pull request Mar 7, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HyukjinKwon HyukjinKwon HyukjinKwon left review comments

@cloud-fan cloud-fan cloud-fan approved these changes

@gengliangwang gengliangwang Awaiting requested review from gengliangwang

Assignees

No one assigned

Labels

SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@MaxGekk @cloud-fan @HyukjinKwon