[SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode #35797
Conversation
| .version("3.0.0") | ||
| .booleanConf | ||
| .createWithDefault(false) | ||
| .createWithDefault(sys.env.get("SPARK_ANSI_SQL_MODE").contains("true")) |
There was a problem hiding this comment.
will sys.env.get("SPARK_ANSI_SQL_MODE") be null?
There was a problem hiding this comment.
The result type of sys.env.get("SPARK_ANSI_SQL_MODE") is Option[String]. It is totally fine here.
|
|
||
| jobs: | ||
| ansi_sql_test: | ||
| uses: ./.github/workflows/build_and_test.yml |
There was a problem hiding this comment.
oh wow. This is nice. Now we can reuse workflow files. cc @dongjoon-hyun FYI
There was a problem hiding this comment.
Yes. I suggest that we move all the scheduled workflows to separated files.
There was a problem hiding this comment.
Yeah, we should separate them. This is nice. @Yikun too FYI
There was a problem hiding this comment.
Ya, it looks like a nice tip. Thanks, @HyukjinKwon .
There was a problem hiding this comment.
Cool, really good, also is an inspiration to reuse workflow in downstream repo.
Looks like there are some limitations we can only separated our workflow which is not using strategy.
There was a problem hiding this comment.
@Yikun the limitation is for the caller. The strategy in "build_and_test.yml" still works.
gengliangwang
changed the title
|
This one is ready. The test result https://github.com/gengliangwang/spark/actions/runs/1961390528 is the same as changing the SQLConf manually: #34970 |
|
@cloud-fan @HyukjinKwon @dongjoon-hyun @Yikun Thanks for the reviews! |
5 participants
What changes were proposed in this pull request?
Add Github action test job for ANSI SQL mode. It will be triggered after each commit push on the master branch of apache/spark.
To make the implementation easy, ANSI SQL mode becomes enabled if ENV variable
SPARK_ANSI_SQL_MODEis set astrueWhy are the changes needed?
Testing the ANSI SQL mode on each commit push, so that we can find issues in time.
Does this PR introduce any user-facing change?
Yes, ANSI SQL mode becomes enabled if ENV variable
SPARK_ANSI_SQL_MODEis set astrue. This won't be in documentation since it is for testing purpose.How was this patch tested?
Manually try on my repo: https://github.com/gengliangwang/spark/actions/runs/1961176426