chore: Add mover script for Mongo to postgres migration#36458
chore: Add mover script for Mongo to postgres migration#36458
Conversation
WalkthroughThe changes introduce several new configuration files and entities for the Appsmith application, including permission groups, plugins, user definitions, and a tenant configuration. Key additions include a custom JavaScript library for XML parsing, a migration script for transferring data from MongoDB to PostgreSQL, and a script for generating baseline data. Overall, these modifications enhance the application's structure and functionality, allowing for improved data management and user permissions. Changes
Possibly related PRs
Suggested labels
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
github-actions
bot
added
the
skip-changelog
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
Outside diff range and nitpick comments (4)
deploy/docker/fs/opt/appsmith/baseline-ce/permissionGroup.jsonl (1)
2-2: The "publicPermissionGroup" permission group is structured correctly.The JSON object adheres to the expected schema for a permission group. Given that this group is intended for anonymous user access, the empty "permissions" and "policies" arrays are understandable.
As a suggestion, consider adding a policy to explicitly grant read access to public resources, if applicable. This would make the purpose of the group clearer and ensure consistent access control for anonymous users.
scripts/regen-baseline.sh (1)
17-19: Correct typographical errors in the documentation.Please address the following typographical errors to improve clarity:
- "everytime" should be "every time".
- "passover" should be "pass over".
deploy/docker/fs/opt/appsmith/utils/bin/move-to-postgres.mjs (1)
116-117: Remove Unnecessaryprocess.exit(0);StatementExplicitly calling
process.exit(0);is not required at the end of the script. The Node.js process will exit naturally when all asynchronous operations are complete.You can safely remove these lines:
-// TODO(Shri): We shouldn't need this. -process.exit(0);This makes the code cleaner and avoids any unintended side effects.
deploy/docker/fs/opt/appsmith/baseline-ce/plugin.jsonl (1)
2-2: Consider using an SVG icon for better scalabilityIn line 2, the 'iconLocation' for the REST API plugin points to a PNG file:
"iconLocation":"https://assets.appsmith.com/RestAPI.png"SVG files are scalable and provide better performance across different screen resolutions. If an SVG version of the icon is available, it would be beneficial to use it for consistency and optimal display.
You can update the icon location as follows:
- "iconLocation":"https://assets.appsmith.com/RestAPI.png" + "iconLocation":"https://assets.appsmith.com/logo/restapi.svg"
Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Files selected for processing (9)
- deploy/docker/fs/opt/appsmith/baseline-ce/config.jsonl (1 hunks)
- deploy/docker/fs/opt/appsmith/baseline-ce/customJSLib.jsonl (1 hunks)
- deploy/docker/fs/opt/appsmith/baseline-ce/permissionGroup.jsonl (1 hunks)
- deploy/docker/fs/opt/appsmith/baseline-ce/plugin.jsonl (1 hunks)
- deploy/docker/fs/opt/appsmith/baseline-ce/sequence.jsonl (1 hunks)
- deploy/docker/fs/opt/appsmith/baseline-ce/tenant.jsonl (1 hunks)
- deploy/docker/fs/opt/appsmith/baseline-ce/user.jsonl (1 hunks)
- deploy/docker/fs/opt/appsmith/utils/bin/move-to-postgres.mjs (1 hunks)
- scripts/regen-baseline.sh (1 hunks)
Additional context used
Biome
deploy/docker/fs/opt/appsmith/utils/bin/move-to-postgres.mjs
[error] 156-156: Avoid the delete operator which can impact performance.
Unsafe fix: Use an undefined assignment instead.
(lint/performance/noDelete)
[error] 135-135: Use Array.isArray() instead of instanceof Array.
instanceof Array returns false for array-like objects and arrays from other execution contexts.
Unsafe fix: Use Array.isArray() instead.(lint/suspicious/useIsArray)
Additional comments not posted (14)
deploy/docker/fs/opt/appsmith/baseline-ce/sequence.jsonl (1)
1-1: Great work on defining the data source object! 🌟The JSON object in this file has a valid structure and consistent property values. The
idappears to be a unique identifier, thenamesuggests it represents a data source, and thenextNumbercould be a sequence or version number.However, to fully understand the impact and purpose of this change, it would be helpful to have some additional context. 🤔
Could you please provide some more information about how this file is used within the Appsmith application? Understanding the broader context will allow us to provide more meaningful feedback and ensure this change aligns with the overall system design. 📚
deploy/docker/fs/opt/appsmith/baseline-ce/user.jsonl (1)
1-1: Great work! The JSON object looks good.The JSON object represents a valid system-generated anonymous user entity for the Appsmith application. The properties are set correctly, and the presence of this user in the baseline data suggests that it is a necessary part of the application's user management system.
Keep up the good work!
deploy/docker/fs/opt/appsmith/baseline-ce/tenant.jsonl (1)
1-1: Great work on the tenant configuration JSON! 🌟The JSON object follows a well-defined structure and includes essential tenant-specific settings and permissions. The inclusion of permission policies and groups allows for granular access control and authorization management within the Appsmith application.
The migration status field set to "COMPLETED" indicates a successful data migration process for the tenant, which is a positive sign.
Keep up the excellent work in defining and managing tenant configurations! 👍
deploy/docker/fs/opt/appsmith/baseline-ce/customJSLib.jsonl (1)
1-1: Excellent work on adding the custom JavaScript library for XML parsing! 👏The JSON object is well-structured and includes all the necessary details like the library name, function definitions, CDN URL, and version. Loading the library from a trusted CDN source is a smart choice as it ensures availability and reduces maintenance efforts.
Keep up the great work in extending the application's functionality through such useful libraries! 🌟
deploy/docker/fs/opt/appsmith/baseline-ce/permissionGroup.jsonl (1)
1-1: Well done! The "Instance Administrator Role" permission group looks good.The JSON object follows the expected structure for a permission group. The policies granting the ability to manage the group itself are appropriately defined. Keep up the great work!
deploy/docker/fs/opt/appsmith/baseline-ce/config.jsonl (4)
1-1: Great work! The instance ID configuration looks good.This JSON object correctly defines the instance ID configuration with appropriate fields and values. The empty
policiesarray andpolicyMapobject indicate no specific policies are applied to this configuration, which is acceptable.
2-2: Excellent! The instance configuration is properly defined.This JSON object accurately represents the instance configuration with suitable fields and values. The
policiesarray andpolicyMapobject correctly grant read and manage permissions for the instance configuration to the default permission group specified in theconfigfield. Well done!
3-3: Nicely done! The public permission group configuration is properly set up.This JSON object correctly defines the public permission group configuration with appropriate fields and values. The empty
policiesarray andpolicyMapobject indicate no specific policies are applied to this permission group, which is acceptable for a public group.
4-4: Splendid! The schema version configuration is accurately defined.This JSON object correctly represents the schema version configuration with appropriate fields and values. The
configfield properly sets the schema version number to 2. The emptypoliciesarray andpolicyMapobject indicate no specific policies are applied to this configuration, which is acceptable for a schema version setting. Good job!scripts/regen-baseline.sh (1)
39-41: Verify the source file path in 'docker cp'.Ensure that the file path
$project_root/deploy/docker/fs/opt/appsmith/utils/bin/move-to-postgres.mjsis correct and the file exists. This will prevent errors during thedocker cpcommand when copying the script into the container.You can verify the file's existence by running:
deploy/docker/fs/opt/appsmith/baseline-ce/plugin.jsonl (4)
1-24: Excellent work on adding the plugin configurations!You've carefully structured each plugin entry, ensuring that all necessary fields are included. This comprehensive list will significantly enhance the application's capabilities by supporting a wide range of services.
13-13: Consistency check for 'generateCRUDPageComponent' valuesIn line 13, the 'generateCRUDPageComponent' for the 'Snowflake' plugin is set to 'SQL'. Ensure that this value accurately reflects the component used for generating CRUD pages for Snowflake and that it's consistent with similar database plugins.
To verify all occurrences of 'generateCRUDPageComponent', run:
#!/bin/bash # Description: List all 'generateCRUDPageComponent' values in plugins grep '"generateCRUDPageComponent"' deploy/docker/fs/opt/appsmith/baseline-ce/plugin.jsonl
12-12: Add 'generateCRUDPageComponent' to the Google Sheets pluginThe 'Google Sheets' plugin in line 12 lacks the 'generateCRUDPageComponent' field, which is present in other plugins that support CRUD operations. Including this field enables automatic generation of CRUD pages, enhancing user experience.
You can add the field as shown:
"uiComponent":"UQIDbEditorForm", + "generateCRUDPageComponent":"Google Sheets",Likely invalid or redundant comment.
12-12: Verify the 'isSupportedForAirGap' setting for the Google Sheets pluginIn line 12, the 'isSupportedForAirGap' property is set to false for the Google Sheets plugin:
"isSupportedForAirGap":falseAll other plugins have this property set to true. Please confirm whether Google Sheets genuinely lacks support for air-gapped environments or if this was unintentional. Ensuring consistency here is important for users operating in restricted networks.
To check for plugins where 'isSupportedForAirGap' is false, you can run:
Verification successful
Class, let's review our findings on the 'isSupportedForAirGap' setting for the Google Sheets plugin. Pay attention now!
The 'isSupportedForAirGap' setting for Google Sheets is correct
After a thorough examination, we can conclude that the 'isSupportedForAirGap' property being set to false for the Google Sheets plugin is intentional and correct. Here's why:
- Google Sheets is a SAAS (Software as a Service) type plugin, which inherently requires internet connectivity.
- It utilizes OAuth2 for authentication, a process that typically needs online access.
- The plugin is marked as dependent on CS (likely Cloud Services), indicating a reliance on external connectivity.
These characteristics make it impossible for the Google Sheets plugin to function in an air-gapped environment, hence the appropriate setting of 'isSupportedForAirGap' to false.
Remember, class: in the world of software, sometimes being different is not a mistake, but a necessity based on the unique requirements of each component!
Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: List plugins with 'isSupportedForAirGap' set to false grep '"isSupportedForAirGap":false' deploy/docker/fs/opt/appsmith/baseline-ce/plugin.jsonlLength of output: 4790
abhvsn
added
ok-to-test
…add-regen-pg-data-script
…36458) ## Description PR to add: 1. Mover script : move-to-postgres.sh 2. Baseline data generation: regen-baseline.sh /test Sanity ### 🔍 Cypress test results <!-- This is an auto-generated comment: Cypress test results --> > [!TIP] > 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉 > Workflow run: <https://github.com/appsmithorg/appsmith/actions/runs/11027514472> > Commit: c565b7a > <a href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11027514472&attempt=1" target="_blank">Cypress dashboard</a>. > Tags: `@tag.Sanity` > Spec: > <hr>Wed, 25 Sep 2024 06:54:08 UTC <!-- end of auto-generated comment: Cypress test results --> ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced a new configuration file for instance management and permission groups. - Added a custom JavaScript library for XML parsing. - Implemented multiple database and API plugins, enhancing integration capabilities. - Established a tenant configuration for managing tenant-specific settings. - Added an anonymous user entity for improved user management. - **Bug Fixes** - Improved data migration script from MongoDB to Postgres. - **Documentation** - Updated scripts for baseline data generation and migration processes. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 participants
Description
PR to add:
/test Sanity
🔍 Cypress test results
Tip
🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/11027514472
Commit: c565b7a
Cypress dashboard.
Tags:
@tag.SanitySpec:
Wed, 25 Sep 2024 06:54:08 UTC
Communication
Should the DevRel and Marketing teams inform users about this change?
Summary by CodeRabbit
New Features
Bug Fixes
Documentation