LSP server Integration via vscode-python-tools-extension + pivot away from FastAPI to direct ZenML library use #6

marwan37 · 2024-03-18T07:43:30Z

This PR marks a significant architectural evolution, incorporating the vscode-python-tools-extension template to enhance TypeScript and ZenML Python library integration. Going beyond typical LSP functions like linting/formatting, it lays down a proven framework for environment and interpreter detection, and an event-driven structure that improves existing features and prepares for future updates.

Rationale behind the change
Previously, we depended on FastAPI and background shell processes to interact with the ZenML Python library. This raised sustainability concerns and limited flexibility to specific ZenML versions. The new approach mitigates these issues, offers flexibility to use any local ZenML version (minimum version pending), and provides a cleaner, and more maintainable solution.

Credits to @strickvl for discovering the python tools extension and suggesting the integration.

Detailed Changes

New: Python Tool (LSP Server Side)

All Python-related LSP files, packages, and code is encapsulated within the bundled/tool directory.
Dependencies are listed in requirements.in and managed through nox.
Core logic and tool-specific specific setup is in the lsp_server.py file.
Additions to the template include:
- zenml_client.py centralizes ZenML library interactions.
- zenml_language_server.py extends the pygls LanguageServer to include ZenML version checks and custom notifications.
- Incorporated watchdog to enable real-time config.yaml monitoring, maintaining extension sync with external ZenML environment or configuration adjustments.

On the TypeScript Side

Preserved the core structure within the common directory as recommended by the template.
Commands overhauled to interact with the LSP server instead of ZenML's FastAPI server.
UI command for pipeline run deletions, and status icons for various pipeline states.
Added comprehensive tests to evaluate the new integrations.
New integrations include:
- The LSClient class for Language Server interaction.
- The EventBus class, an extension of EventEmitter, orchestrates notifications from the LSP server.
- The ZenExtension class to encapsulate the new extension activation process.

Demo: Syncing CLI ZenML operations with the extension automatically

Demo: Prompts to select python interpreter until the correct one with ZenML installed is detected

Summary by CodeRabbit

New Features
- Enhanced ZenML VS Code extension with seamless integration with ZenML CLI, providing Activity Bar view, Python Tool Integration, Real-Time Configuration Monitoring, and Status Bar updates.
- Introduced Environment View and additional commands for server, stack, pipeline, and environment operations.
Enhancements
- Improved Python interpreter management and server connection handling.
- Optimized pipeline and environment views for real-time updates and detailed information.
- Streamlined extension settings for workspace configurations.
Documentation
- Updated README.md with comprehensive details on extension features, a Quick Start Guide, and development instructions.
Tests
- Added tests for server configuration updates, event handling, and server status updates.

…tside virtual environments

…ration

…execution

…flect changes

…from zenml

…client

…atures

…ing from servers

…d activity bar

…connect from servers

…tly updated

coderabbitai

Review Status

Actionable comments generated: 0

Configuration used: .coderabbit.yml

Commits

Files that changed from the base of the PR and between d1468a3 and c19000f.

Files selected for processing (7)

bundled/tool/lsp_zenml.py (1 hunks)
bundled/tool/zenml_wrappers.py (1 hunks)
package.json (1 hunks)
src/commands/pipelines/registry.ts (1 hunks)
src/types/PipelineTypes.ts (1 hunks)
src/views/activityBar/common/PaginationTreeItems.ts (1 hunks)
src/views/activityBar/pipelineView/PipelineDataProvider.ts (1 hunks)

Files skipped from review as they are similar to previous changes (7)

bundled/tool/lsp_zenml.py
bundled/tool/zenml_wrappers.py
package.json
src/commands/pipelines/registry.ts
src/types/PipelineTypes.ts
src/views/activityBar/common/PaginationTreeItems.ts
src/views/activityBar/pipelineView/PipelineDataProvider.ts

…ckage.nls.json

coderabbitai

Review Status

Actionable comments generated: 0

Configuration used: .coderabbit.yml

Commits

Files that changed from the base of the PR and between c19000f and 46f29fa.

Files selected for processing (4)

package.json (1 hunks)
package.nls.json (1 hunks)
src/commands/pipelines/registry.ts (1 hunks)
src/views/activityBar/pipelineView/PipelineDataProvider.ts (1 hunks)

Files skipped from review as they are similar to previous changes (4)

package.json
package.nls.json
src/commands/pipelines/registry.ts
src/views/activityBar/pipelineView/PipelineDataProvider.ts

coderabbitai

Review Status

Actionable comments generated: 0

Configuration used: .coderabbit.yml

Commits

Files that changed from the base of the PR and between 46f29fa and 286857a.

Files selected for processing (2)

README.md (1 hunks)
package.json (1 hunks)

Files skipped from review as they are similar to previous changes (1)

package.json

Additional comments: 5

README.md (5)

5-15: The introduction and current features sections provide a clear and comprehensive overview of the extension's capabilities and integration with the ZenML CLI. Well done on detailing these aspects.

17-51: The new additions and commands sections are well-detailed, providing users with clear instructions on how to utilize the new features and commands introduced in this update. Great job on this section.

53-115: The limitations, quick start guide, and setup instructions sections are clear and comprehensive, effectively guiding users through the initial setup and highlighting important compatibility information.

117-152: The usage, development, and testing instructions sections are well-organized and informative, providing users and developers with clear guidance on how to utilize, develop, and test the extension effectively.

154-185: The ZenML version and installation, troubleshooting, contributing, and pull requests sections are comprehensive and provide valuable information for users and contributors alike. Excellent work on compiling these helpful resources.

coderabbitai

Review Status

Actionable comments generated: 0

Configuration used: .coderabbit.yml

Commits

Files that changed from the base of the PR and between 286857a and e512dba.

Files selected for processing (1)

package.json (1 hunks)

Files skipped from review as they are similar to previous changes (1)

package.json

coderabbitai

Review Status

Actionable comments generated: 1

Configuration used: .coderabbit.yml

Commits

Files that changed from the base of the PR and between e512dba and 0ebe969.

Files selected for processing (3)

README.md (1 hunks)
src/common/constants.ts (1 hunks)
src/test/ts_tests/extension.test.ts (1 hunks)

Files skipped from review as they are similar to previous changes (2)

src/common/constants.ts
src/test/ts_tests/extension.test.ts

README.md

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

…vscode-zenml into python-lsp-integration

coderabbitai

Review Status

Actionable comments generated: 1

Configuration used: .coderabbit.yml

Commits

Files that changed from the base of the PR and between 0ebe969 and 4aaae68.

Files selected for processing (1)

README.md (1 hunks)

Additional comments: 3

README.md (3)

12-15: The features section clearly outlines the new capabilities of the ZenML VSCode extension, aligning well with the PR objectives.

35-37: The requirements section is updated to reflect the necessary conditions for using the ZenML VSCode extension effectively, including ZenML and Python version requirements.

39-47: The sections on feedback, contributions, and further assistance are well-structured, providing clear paths for community engagement and support.

coderabbitai · 2024-03-27T20:13:26Z

README.md

+Note that you'll need to have ZenML installed in your Python environment to use
+this extension and your Python version needs to be 3.8 or greater.


Consider adding a comma for clarity in the sentence about Python version requirements.

- this extension and your Python version needs to be 3.8 or greater. + this extension, and your Python version needs to be 3.8 or greater.

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change

Note that you'll need to have ZenML installed in your Python environment to use

this extension and your Python version needs to be 3.8 or greater.

Note that you'll need to have ZenML installed in your Python environment to use

this extension, and your Python version needs to be 3.8 or greater.

coderabbitai

Review Status

Actionable comments generated: 2

Configuration used: .coderabbit.yml

Commits

Files that changed from the base of the PR and between 4aaae68 and 2985e77.

Files selected for processing (3)

CONTRIBUTING.md (1 hunks)
README.md (1 hunks)
src/views/statusBar/index.ts (1 hunks)

Files skipped from review as they are similar to previous changes (1)

src/views/statusBar/index.ts

coderabbitai · 2024-03-27T20:32:37Z

README.md

+Note that you'll need to have [ZenML](https://github.com/zenml-io/zenml) installed in your Python environment to use
+this extension and your Python version needs to be 3.8 or greater.


The sentence about the Python version requirement could be improved for clarity by adding a comma.

- this extension and your Python version needs to be 3.8 or greater. + this extension, and your Python version needs to be 3.8 or greater.

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change

Note that you'll need to have [ZenML](https://github.com/zenml-io/zenml) installed in your Python environment to use

this extension and your Python version needs to be 3.8 or greater.

Note that you'll need to have [ZenML](https://github.com/zenml-io/zenml) installed in your Python environment to use

this extension, and your Python version needs to be 3.8 or greater.

coderabbitai · 2024-03-27T20:32:37Z

CONTRIBUTING.md

+### Additional Resources
+
+- [ZenML VSCode Extension Repository](https://github.com/zenml-io/vscode-zenml)
+- [ZenML Documentation](https://docs.zenml.io)


The URL for the ZenML Documentation has been updated. Ensure it points to the correct location.

- - [ZenML Documentation](https://docs.zenml.io) + - [ZenML Documentation](https://zenml.io/docs)

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change

- [ZenML Documentation](https://docs.zenml.io)

- [ZenML Documentation](https://zenml.io/docs)

coderabbitai

Review Status

Actionable comments generated: 2

Configuration used: .coderabbit.yml

Commits

Files that changed from the base of the PR and between 2985e77 and 038cb22.

Files selected for processing (1)

CONTRIBUTING.md (1 hunks)

coderabbitai · 2024-03-27T20:35:59Z

CONTRIBUTING.md

+### Additional Resources
+
+- [ZenML VSCode Extension Repository](https://github.com/zenml-io/vscode-zenml)
+- [ZenML Documentation](https://docs.zenml.io)


The URL for the ZenML Documentation is outdated. Please update it to point to the correct location.

- - [ZenML Documentation](https://docs.zenml.io) + - [ZenML Documentation](https://zenml.io/docs)

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change

- [ZenML Documentation](https://docs.zenml.io)

- [ZenML Documentation](https://zenml.io/docs)

coderabbitai · 2024-03-27T20:35:59Z

CONTRIBUTING.md

+
+## Troubleshooting Common Issues
+
+- Ensure all dependencies are up to date and compatible.


Consider adding hyphens to "up to date" to form the correct adjective "up-to-date".

- Ensure all dependencies are up to date and compatible. + Ensure all dependencies are up-to-date and compatible.

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change

- Ensure all dependencies are up to date and compatible.

Ensure all dependencies are up-to-date and compatible.

marwan37 added 30 commits March 7, 2024 11:48

Setup basic vscode extension skeleton

a5d7fba

Add CONTRIBUTING.md file for zenml vscode extension

7e064a8

Add status bar that displays the currently active stack

7ef69d2

Add tests for status bar functionality

ecfd9c3

Add setup/contributing guide directly to README file

1d2a4ec

Update README: Clarify ZenML stack setup and CLI command execution ou…

0541e9a

…tside virtual environments

Fix parsing of quoted stack names when getting active stack

1e9ae09

Remove CHANGELOG.md file

70ba401

Rename zenml-studio to zenml

e953e55

Merge quickstart guide relevant info into README file

4659063

Remove redundant comments about parsing the output in statusBar.ts

31f53a1

Update README: Clarify ZenML VSCode extension functionality and integ…

1c8a8e9

…ration

Add info regarding click-to-refresh for the status bar

9f99c48

Add python scripts integrating zenml python client

305c7ce

Refactor stackCommands to run the python script instead of the cli

9fe0ad8

Add serverCommands to check the server status using the python client

f5e8fba

Update Shell class to integrate environment checks and python script …

b6f2b4f

…execution

Add tests for shell and serverCommands and update previous ones to re…

6027054

…flect changes

Add configuration block for venvPath, and sinon package

9e95887

Register server command in extension

6982e56

Delete helpers file as parsing output is no longer needed

202b73b

Update status bar to display server status icon, with stack name only

fea58b2

Refactor check_server_status script to use get_active_server_details …

ac4be27

…from zenml

Update files to reflect changes to data structure returned by python …

939afec

…client

Update README to reflect recent changes, limitations, and upcoming fe…

41e86b1

…atures

Refactor: Centralize script path resolution in Shell class

e3d4dfa

Add commands to allow for connecting to remote servers and disconnect…

2cb4cbf

…ing from servers

Add two view containers for 'Server' and 'Stacks' with tree items

bf15a54

Add service to centralize checking of server status for the status an…

a7fd6cd

…d activity bar

Add python scripts to fetch stacks, get store config, and connect/dis…

a1b67d3

…connect from servers

Allow setting max pipeline run items per page, and list by most recen…

c19000f

…tly updated

coderabbitai bot reviewed Mar 27, 2024

View reviewed changes

marwan37 added 2 commits March 27, 2024 13:53

Add command tree items for next/prev pipeline runs

b18bcfe

Remove preview=true from package.json, and add missing commands to pa…

46f29fa

…ckage.nls.json

coderabbitai bot reviewed Mar 27, 2024

View reviewed changes

Add details to package.json to prepare for publishing

481d8a9

strickvl added the enhancement New feature or request label Mar 27, 2024

add gif to readme

286857a

coderabbitai bot reviewed Mar 27, 2024

View reviewed changes

strickvl added 2 commits March 27, 2024 20:44

use publisher ID

e512dba

use publisher ID everywhere

8b8c891

coderabbitai bot reviewed Mar 27, 2024

View reviewed changes

Edit readme

0ebe969

coderabbitai bot reviewed Mar 27, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

marwan37 and others added 5 commits March 27, 2024 14:51

Update README.md

a651a22

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

fix typo

3a2c4c3

Merge branch 'python-lsp-integration' of https://github.com/marwan37/…

0174aa9

…vscode-zenml into python-lsp-integration

add license shield

89b2ab3

Add installation and version requirements for ZenML extension

4aaae68

coderabbitai bot reviewed Mar 27, 2024

View reviewed changes

strickvl and others added 6 commits March 27, 2024 21:13

Small change

31157c6

add links

d3a0bca

display active stack in status bar again

7e524ab

Add contributing.md

2985e77

edit contrib

038cb22

Small nits for contributing document

b3a663f

strickvl approved these changes Mar 27, 2024

View reviewed changes

coderabbitai bot reviewed Mar 27, 2024

View reviewed changes

strickvl merged commit be2fc19 into zenml-io:develop Mar 27, 2024
1 check passed

coderabbitai bot reviewed Mar 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LSP server Integration via vscode-python-tools-extension + pivot away from FastAPI to direct ZenML library use #6

LSP server Integration via vscode-python-tools-extension + pivot away from FastAPI to direct ZenML library use #6

marwan37 commented Mar 18, 2024 •

edited by coderabbitai bot

Loading

coderabbitai bot left a comment

coderabbitai bot left a comment

coderabbitai bot left a comment

coderabbitai bot left a comment

coderabbitai bot left a comment

coderabbitai bot left a comment

coderabbitai bot Mar 27, 2024

coderabbitai bot left a comment

coderabbitai bot Mar 27, 2024

coderabbitai bot Mar 27, 2024

coderabbitai bot left a comment

coderabbitai bot Mar 27, 2024

coderabbitai bot Mar 27, 2024

		Note that you'll need to have ZenML installed in your Python environment to use
		this extension and your Python version needs to be 3.8 or greater.

		Note that you'll need to have [ZenML](https://github.com/zenml-io/zenml) installed in your Python environment to use
		this extension and your Python version needs to be 3.8 or greater.

	- [ZenML Documentation](https://docs.zenml.io)
	- [ZenML Documentation](https://zenml.io/docs)


		## Troubleshooting Common Issues

		- Ensure all dependencies are up to date and compatible.

	- Ensure all dependencies are up to date and compatible.
	Ensure all dependencies are up-to-date and compatible.

LSP server Integration via vscode-python-tools-extension + pivot away from FastAPI to direct ZenML library use #6

LSP server Integration via vscode-python-tools-extension + pivot away from FastAPI to direct ZenML library use #6

Conversation

marwan37 commented Mar 18, 2024 • edited by coderabbitai bot Loading

Detailed Changes

New: Python Tool (LSP Server Side)

On the TypeScript Side

Summary by CodeRabbit

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot Mar 27, 2024

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot Mar 27, 2024

Choose a reason for hiding this comment

coderabbitai bot Mar 27, 2024

Choose a reason for hiding this comment

coderabbitai bot left a comment

Choose a reason for hiding this comment

coderabbitai bot Mar 27, 2024

Choose a reason for hiding this comment

coderabbitai bot Mar 27, 2024

Choose a reason for hiding this comment

marwan37 commented Mar 18, 2024 •

edited by coderabbitai bot

Loading