-
Notifications
You must be signed in to change notification settings - Fork 222
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[New Feature]: Improve dependency management for MI Connectors #3614
Comments
Following initial discussions with the teams overseeing the MI VSCode extension and the Connector Store, an API specification has been drafted outlining the required functionalities from the Connector Store side. The APIs will provide access to the following features:
The specification can be found here A final specification will be prepared after reviewing the above draft |
One goal of this feature is to automate the connector release process, which is currently done manually. At present, the procedure involves creating the connector ZIP artifacts locally and uploading them to manually created Git release tags. To streamline this process, the original plan was to implement Jenkins workflows to:
As a preliminary step, before setting these workflows up in the main connector repositories and deploying artifacts to the public Maven repository, the process was tested using a local setup and a fork of a connector repository. This involved:
However, a decision was taken to transition from Jenkins to GitHub workflows for this task, aligning with the ongoing effort to migrate all MI-related automation tasks from Jenkins to GitHub workflows. |
After finalizing the API spec for the connector store[1], as the proposed solution would require collaboration and consensus across multiple teams, and considering the time constraints, a milestone plan has also been outlined in this design document.[2] [1] https://gist.github.com/ShammiL/606f1bc87af66a33b391864ff6fb12e8 As our next step, we will be initiating discussions with the teams we believe will be impacted by the proposed changes to ensure alignment and address any concerns. Simultaneously, other areas highlighted in the design will also be addressed. |
After a discussion with members of the IS team looking over IS connectors, it was concluded that the changes proposed for the connector store in Milestone 1 will have no impact on the IS connectors or their current release workflows. Along with that, we also came to the following conclusions:
However, a concern was raised about allocating resources for the implementations proposed for the connector store, due to the limited availability of members on the internal apps team. |
A release workflow has been implemented in the Redis Connector repository, marking the initial step in integrating release workflows across all MI connector repositories. The workflow makes use of the Maven release plugin to perform the release. This workflow will:
The individual steps and the overall workflow were initially tested in a local environment and then on a forked repository of the Redis Connector. A locally configured Nexus server was used to simulate the WSO2 release repository. Once the workflow has been tested on this repository, the same changes will be applied to all MI connector repositories |
Due to the lack of resources to be allocated from the digital transformation team, the MI team will implement the new GET endpoint to retrieve all MI connector details to be used by the MI VS Code extension. [1]. This implementation will cover the following:
[1] https://github.com/wso2-enterprise/integration-product-management/issues/248 |
The implementation of the endpoint for retrieving a list of MI connectors can be found here - https://github.com/wso2-enterprise/digiops-marketing/pull/220 |
The implementation of the daily sync job for updating the connector store database has been updated to read MI metadata and operations - https://github.com/wso2-enterprise/digiops-engineering/pull/164 |
The GET endpoint and the daily sync for populating the connector store database has been updated with the required changes are available on the staging environments now [1] [2] [1] https://github.com/wso2-enterprise/digiops-marketing/pull/220 The metadata files of a selected set of connectors have been updated with the new metadata values and the release workflows have also been added for these connectors. |
Description
The current implementations in MI for adding connectors to an integration project and resolving connector dependencies involve several complex and manual steps from both the connector developer and the user sides. To mitigate the problems in the existing implementation, the current mechanisms need to be reformed.
Describe your problem(s)
Describe your solution(s)
Related issue(s) (optional)
No response
Suggested label(s) (optional)
No response
Suggested assignee(s) (optional)
No response
The text was updated successfully, but these errors were encountered: