Skip to content

[SPARK-9154][SQL] codegen StringFormat #7546

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
tarekbecker wants to merge 4 commits into from
Closed

[SPARK-9154][SQL] codegen StringFormat #7546

tarekbecker wants to merge 4 commits into from

Conversation

@tarekbecker
Copy link
Contributor

@tarekbecker tarekbecker commented Jul 20, 2015

@tarekbecker
Copy link
Contributor Author

tarekbecker commented Jul 20, 2015

Jenkins, ok to test

@SparkQA
Copy link

SparkQA commented Jul 21, 2015

Test build #1128 has finished for PR 7546 at commit 10b4de8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class StringFormat(children: Expression*) extends Expression

@SparkQA
Copy link

SparkQA commented Jul 21, 2015

Test build #1131 has finished for PR 7546 at commit 10b4de8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

marmbrus
marmbrus reviewed Jul 21, 2015
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala Outdated
Copy link
Contributor

@marmbrus marmbrus Jul 21, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably add ImplicitCastInputTypes with a string for the first argument and AnyDataType for remaining children.

Copy link
Contributor Author

@tarekbecker tarekbecker Jul 21, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do have to split the signature for this to StringFormat(string: Expression, args: Expression*), don't I?

Copy link
Contributor

@marmbrus marmbrus Jul 21, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't think so. We just zip with children to check the types so I think it can work either way. Just remember to subtract 1 when figuring out how many AnyDataTypes to fill in.

tarekbecker
tarekbecker reviewed Jul 21, 2015
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala
Copy link
Contributor Author

@tarekbecker tarekbecker Jul 21, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marmbrus Is this what you proposed?

Copy link
Contributor

@marmbrus marmbrus Jul 21, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works. I think StringType :: List.fill(children.size - 1)(AnyDataType) might be a little more clear.

@marmbrus
Copy link
Contributor

marmbrus commented Jul 21, 2015

Thanks! I'm going to merge to master.

@asfgit asfgit closed this in 7f072c3 Jul 21, 2015
@marmbrus marmbrus mentioned this pull request Jul 21, 2015
Closed
asfgit pushed a commit that referenced this pull request Jul 21, 2015
@marmbrus
Revert "[SPARK-9154] [SQL] codegen StringFormat"
87d890c 
This reverts commit 7f072c3.

Revert #7546

Author: Michael Armbrust <[email protected]>

Closes #7570 from marmbrus/revert9154 and squashes the following commits:

ed2c32a [Michael Armbrust] Revert "[SPARK-9154] [SQL] codegen StringFormat"
@tarekbecker tarekbecker mentioned this pull request Jul 21, 2015
Closed
asfgit pushed a commit that referenced this pull request Jul 21, 2015
@tarekbecker @marmbrus
[SPARK-9154] [SQL] codegen StringFormat
d4c7a7a 
Jira: https://issues.apache.org/jira/browse/SPARK-9154

fixes bug of #7546

marmbrus I can't reopen the other PR, because I didn't closed it. Can you trigger Jenkins?

Author: Tarek Auel <[email protected]>

Closes #7571 from tarekauel/SPARK-9154 and squashes the following commits:

dcae272 [Tarek Auel] [SPARK-9154][SQL] build fix
1487602 [Tarek Auel] Merge remote-tracking branch 'upstream/master' into SPARK-9154
f512c5f [Tarek Auel] [SPARK-9154][SQL] build fix
a943d3e [Tarek Auel] [SPARK-9154] implicit input cast, added tests for null, support for null primitives
10b4de8 [Tarek Auel] [SPARK-9154][SQL] codegen removed fallback trait
cd8322b [Tarek Auel] [SPARK-9154][SQL] codegen string format
086caba [Tarek Auel] [SPARK-9154][SQL] codegen string format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@tarekbecker @SparkQA @marmbrus @davies