Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE]: ensure Spark.sql-like code is not migrated accidentally #1213

Closed
1 task done
Tracked by #1204
ericvergnaud opened this issue Apr 1, 2024 · 0 comments · Fixed by #2114
Closed
1 task done
Tracked by #1204

[FEATURE]: ensure Spark.sql-like code is not migrated accidentally #1213

ericvergnaud opened this issue Apr 1, 2024 · 0 comments · Fixed by #2114
Labels
migrate/code Abstract Syntax Trees and other dark magic tech debt chores and design flaws

Comments

@ericvergnaud
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Problem statement

Our current implementation of code migration assumes that a Spark.sql method call is eligible for migration based on its names and a few other checks: number of arguments, type of arguments... It does not check whether the method genuinely belongs to the Spark.sql API.
This may very well be good enough, but if user code contains code that closely mimics some Spark.sql API calls, then they could be migrated accidentally.

Proposed Solution

Implement context analysis in SparkSql class

Additional Context

No response

@ericvergnaud ericvergnaud added enhancement New feature or request needs-triage labels Apr 1, 2024
@github-project-automation github-project-automation bot moved this to Triage in UCX Apr 1, 2024
@nfx nfx added migrate/code Abstract Syntax Trees and other dark magic and removed enhancement New feature or request needs-triage labels Apr 1, 2024
@ericvergnaud ericvergnaud changed the title [FEATURE]: ensure Spark.sql code is not migrated accidentally [FEATURE]: ensure Spark.sql-like code is not migrated accidentally Apr 2, 2024
@nfx nfx added the tech debt chores and design flaws label Apr 22, 2024
@nfx nfx moved this from Triage to Quarter Backlog in UCX Apr 22, 2024
@nfx nfx closed this as completed in #2114 Jul 9, 2024
@nfx nfx closed this as completed in bd99d7c Jul 9, 2024
@github-project-automation github-project-automation bot moved this from Quarter Backlog to Archive in UCX Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
migrate/code Abstract Syntax Trees and other dark magic tech debt chores and design flaws
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants