[FEATURE]: Migrate spark.table("db.table") to spark.table("catalog.db.table")

[nfx]

Is there an existing issue for this?

• I have searched the existing issues

Problem statement

Every table in UC needs a catalog

Proposed Solution

Transform AST/CST with the migrated table index using the fixer framework declared in #1067

• spark.table(...)
• spark.read.table(...)
• ...write.saveAsTable(...)
• ...

Add another Linter/Fixer to https://github.com/databrickslabs/ucx/blob/main/src/databricks/labs/ucx/code/pyspark.py

Additional Context

No response

[ericvergnaud]

As the list of candidate function calls eligible to migration grows, our currently minimalistic approach (checking just the name and arguments of the function being called) might increase the risk of unwanted migrations.
Also, if the argument is not a string literal, the migration will be skipped.
Is that something we want to cater for, and if it is, should it be done as part of this ticket ?

[nfx]

As the list of candidate function calls eligible to migration grows, our currently minimalistic approach (checking just the name and arguments of the function being called) might increase the risk of unwanted migrations.

that's correct.

Also, if the argument is not a string literal, the migration will be skipped.

created a separate issue to track this:

• [FEATURE]: If a code computes a value dynamically, do value inference, at least at the state of linting #1205

[nfx]

@jimidle ^

[ericvergnaud]

@jimidle you could leverage what I did for sys.path in PythonLinter once it's merged to main - fyi @nfx

[jimidle]

@jimidle you could leverage what I did for sys.path in PythonLinter once it's merged to main - fyi @nfx

OK - I will take a look once you merge it. @nfx Do you have a preference for using the fixer framework over Eric's suggestion?

[ericvergnaud]

My suggestion is for improved linting i.e. #1205, not for fixing. It's now merged.

[nfx]

@jimidle @ericvergnaud sys.path manipulation and this feature are orthogonal. We already have the fixer framework and the example is with sql queries.

The implementation that only looks at string constants is trivial and should not take more than few hours to implement and test.

[nfx]

fixed in #1210