Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[EPIC] Lint all notebooks that belong to a job in scope of the assessment workflow #1085

Open
22 of 26 tasks
Tracked by #1204
, #1490
...
nfx opened this issue Mar 21, 2024 · 0 comments
Open
22 of 26 tasks
Tracked by #1204
, #1490
...

[EPIC] Lint all notebooks that belong to a job in scope of the assessment workflow #1085

nfx opened this issue Mar 21, 2024 · 0 comments
Labels
CUJ critial user journey EPIC migrate/code Abstract Syntax Trees and other dark magic step/assessment go/uc/upgrade - Assessment Step

Comments

@nfx
Copy link
Collaborator

nfx commented Mar 21, 2024 •
edited
Loading

Display a dashboard with the incompatible code patterns per their occurrence

Relevant issues:

image
@nfx nfx added enhancement New feature or request step/assessment go/uc/upgrade - Assessment Step migrate/jobs Step 5 - Upgrading Jobs for External Tables migrate/code Abstract Syntax Trees and other dark magic labels Mar 21, 2024
@nfx nfx added this to UCX Mar 21, 2024
@github-project-automation github-project-automation bot moved this to Triage in UCX Mar 21, 2024
This was referenced Apr 1, 2024
Open
Closed
@nfx nfx moved this from Triage to Month Backlog in UCX Apr 10, 2024
@nfx nfx removed the enhancement New feature or request label Apr 22, 2024
@nfx nfx changed the title [FEATURE]: Lint all notebooks that belong to a job in scope of the assessment workflow [EPIC]: Lint all notebooks that belong to a job in scope of the assessment workflow Apr 22, 2024
@nfx nfx added the CUJ critial user journey label Apr 22, 2024
@nfx nfx added this to the Q2 milestone Apr 22, 2024
@nfx nfx removed the migrate/jobs Step 5 - Upgrading Jobs for External Tables label Apr 22, 2024
@nfx nfx pinned this issue Apr 22, 2024
@nfx nfx mentioned this issue Apr 22, 2024
Closed
9 tasks
@nfx nfx changed the title [EPIC]: Lint all notebooks that belong to a job in scope of the assessment workflow [EPIC] Lint all notebooks that belong to a job in scope of the assessment workflow Apr 22, 2024
@nfx nfx removed this from the Q2 milestone Apr 22, 2024
@nfx nfx changed the title [EPIC] Lint all notebooks that belong to a job in scope of the assessment workflow [EPIC] Lint all notebooks that belong to a job in scope of the assessment workflow: databricks labs ucx lint-local-code Apr 23, 2024
@nfx nfx changed the title [EPIC] Lint all notebooks that belong to a job in scope of the assessment workflow: databricks labs ucx lint-local-code [EPIC] Lint all notebooks that belong to a job in scope of the assessment workflow Apr 23, 2024
@nfx nfx moved this from Month Backlog to Active Backlog in UCX Apr 24, 2024
@nfx nfx mentioned this issue May 3, 2024
Merged
nfx added a commit that referenced this issue May 7, 2024
@nfx
Added baseline for workflow linter (#1613)
1ae345a 
```mermaid
flowchart TD
    job -->|has many| job_task
    job_task -.-> notebook_task
    job_task -.-> wheel_task 

    job -.-> git_source

    job_task -.->|execute on| interactive_cluster
    interactive_cluster -.-> library

    job_task -.-> library
    library -.-> wheel_on_dbfs
    library -.-> wheel_on_wsfs
    library -.-> wheel_on_volumes
    library -.-> egg_on_dbfs
    library -.-> egg_on_wsfs
    library -.-> pypi
    wheel_task -.-> wheel_on_dbfs
    wheel_task -.-> wheel_on_wsfs

    wheel_on_dbfs -.-> python_file
    wheel_on_wsfs -.-> python_file
    egg_on_dbfs -.-> python_file
    egg_on_wsfs -.-> python_file
    pypi -.-> python_file
    wsfs_file -.-> python_file
    python_file -.->|import| python_file
    notebook_task -.-> notebook
    notebook -.->|import| python_file
    notebook -.->|can run| notebook

    job_task -.-> dependency_graph
    python_file --> dependency_graph
    notebook --> dependency_graph

    git_source -.-> python_file
    git_source -.-> notebook
    lint_local_code_cli --> dependency_graph

    workflow_linter --> dependency_graph
    workflow_linter -.-> job_problems
    dependency_graph -.-> job_problems
    job_problems -.->|viz| redash_dashboard
```

This PR adds baseline for linting workflows

Related to:
- #1542 
- #1541
- #1540
- #1539
- #1382
- #1204
- #1203
- #1085

closes #1559
closes #1468
closes #1286
@nfx nfx unpinned this issue Jul 25, 2024
@gueniai gueniai added the EPIC label Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CUJ critial user journey EPIC migrate/code Abstract Syntax Trees and other dark magic step/assessment go/uc/upgrade - Assessment Step
Projects
UCX
Status: No status
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nfx @gueniai