New module for backup_restore_api #238
Conversation
WalkthroughThe update includes the addition and improvement of backup and restore functionalities for Dgraph and Postgres databases, changes in Kafka topic configurations, and updates to various application components. Notably, new scripts and functions handle data backup and restoration, several configurations were simplified or deprecated, and specific code logic was streamlined to enhance efficiency and maintainability. Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
|
Task linked: CU-86bymxdnb Create backup api |
There was a problem hiding this comment.
@sushantpatil1214 please review this new component and only include methods, files and libraries required
JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/AppConfig.java
Outdated
Show resolved
Hide resolved
JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/AppConfig.java
Outdated
Show resolved
Hide resolved
JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java
Outdated
Show resolved
Hide resolved
.../JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/JsonFieldsConfig.java
Outdated
Show resolved
Hide resolved
...ps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlAuditTrail.java
Outdated
Show resolved
Hide resolved
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 23
Out of diff range and nitpick comments (5)
JeMPI_Apps/build-all-ci.sh (1)
Line range hint
13-17: Refactor thetag_imagefunction to improve script safety and readability.tag_image() { if [ -n "$tag_to_use" ]; then IFS=':' read -ra image_details <<< "$2" docker tag "$1" "${image_details[0]}:$tag_to_use" else docker tag "$1" "$2" fi }This change includes using
-ninstead of! -z, adding the-roption toreadto prevent backslash mangling, and quoting variables to prevent globbing and word splitting.devops/linux/docker/conf/haproxy/haproxy.cfg (1)
Line range hint
43-71: Correct the typo in the HAProxy configuration for the Backup Restore API.- use_backend fe_backup-restore-api if { path_beg /JeMPI } + use_backend be_backup-restore-api if { path_beg /JeMPI }This change corrects the backend reference in the frontend configuration to ensure proper routing of requests.
devops/linux/docker/backup_restore/postgres-backup.sh (1)
Line range hint
4-51: Refactor the PostgreSQL backup script to improve safety and address unused variables.# Ensure variables are quoted to prevent word splitting and globbing. PGPASSWORD="$DB_PASSWORD" pg_dump -h "$DB_HOST" -U "$DB_USER" -d "$db" -F c -f "${BACKUP_PATH}/${db}--$BACKUP_DATE_TIME.sql" # Address the unused variable `backup_file` by removing it or documenting its purpose if it is intended for future use.These changes ensure that the script is safer and more maintainable.
devops/linux/docker/deployment/deploy-local.sh (2)
Line range hint
1-1: Add a shebang line at the beginning of the script to specify the shell interpreter.+ #!/bin/bash
221-227: Ensure that the backup process logs are detailed and include timestamps for better traceability.Consider adding more detailed logging for each step of the backup process to help with troubleshooting and ensure that all operations are logged with timestamps.
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (41)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/.gitignore (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/build.sh (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/checkstyle/suppression.xml (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/docker/.gitignore (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/docker/Dockerfile (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/pom.xml (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/push.sh (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/AppConfig.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/Ask.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/JsonFieldsConfig.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/MapError.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlAuditTrail.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlClient.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlNotifications.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/Routes.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/api/BackupRestoreAPI.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/api/HttpServer.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/resources/application.conf (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/resources/log4j.properties (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/resources/log4j2.properties (1 hunks)
- JeMPI_Apps/build-all-ci.sh (1 hunks)
- JeMPI_Apps/build-all-java.sh (3 hunks)
- JeMPI_Apps/pom.xml (1 hunks)
- devops/linux/docker/backup_restore/dgraph-backup-api.sh (1 hunks)
- devops/linux/docker/backup_restore/dgraph-backup.py (1 hunks)
- devops/linux/docker/backup_restore/postgres-backup.sh (4 hunks)
- devops/linux/docker/conf/env/conf-env-high-1-pc.template (1 hunks)
- devops/linux/docker/conf/env/conf-env-low-1-pc.template (1 hunks)
- devops/linux/docker/conf/env/create-env-linux-high-1.sh (1 hunks)
- devops/linux/docker/conf/env/create-env-linux-low-1.sh (1 hunks)
- devops/linux/docker/conf/env/create-env-wsl-low-1.sh (1 hunks)
- devops/linux/docker/conf/haproxy/haproxy.cfg (3 hunks)
- devops/linux/docker/conf/images/conf-app-images.sh (1 hunks)
- devops/linux/docker/conf/stack/docker-stack-low-0.yml (2 hunks)
- devops/linux/docker/conf/stack/docker-stack-low-1.yml (2 hunks)
- devops/linux/docker/deployment/deploy-local.sh (5 hunks)
- devops/linux/docker/deployment/down/d-stack-stop-backup-restore-api-services.sh (1 hunks)
- devops/linux/docker/deployment/down/d-stack-stop-services.sh (1 hunks)
- devops/linux/docker/deployment/reboot/d-stack-start-backup-restore-api-services.sh (1 hunks)
- devops/linux/docker/deployment/reboot/d-stack-start-services.sh (1 hunks)
Files skipped from review due to trivial changes (10)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/.gitignore
- JeMPI_Apps/JeMPI_BackupRestoreAPI/checkstyle/suppression.xml
- JeMPI_Apps/JeMPI_BackupRestoreAPI/docker/.gitignore
- JeMPI_Apps/JeMPI_BackupRestoreAPI/docker/Dockerfile
- JeMPI_Apps/JeMPI_BackupRestoreAPI/pom.xml
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/resources/application.conf
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/resources/log4j.properties
- JeMPI_Apps/pom.xml
- devops/linux/docker/conf/env/conf-env-high-1-pc.template
- devops/linux/docker/conf/env/conf-env-low-1-pc.template
Additional Context Used
ShellCheck (142)
JeMPI_Apps/JeMPI_BackupRestoreAPI/build.sh (7)
[info] 6-6: Not following: ./conf/images/conf-app-images.sh was not specified as input (see shellcheck -x).
[info] 6-6: Double quote to prevent globbing and word splitting.
[info] 7-7: Not following: ../build-check-jdk.sh was not specified as input (see shellcheck -x).
[warning] 9-9: JAR_FILE appears unused. Verify use (or export if used externally).
[warning] 10-10: APP_IMAGE appears unused. Verify use (or export if used externally).
[warning] 11-11: APP appears unused. Verify use (or export if used externally).
[info] 12-12: Not following: ../build-app-image.sh was not specified as input (see shellcheck -x).
JeMPI_Apps/JeMPI_BackupRestoreAPI/push.sh (11)
[info] 6-6: Not following: ./conf.env was not specified as input (see shellcheck -x).
[info] 6-6: Double quote to prevent globbing and word splitting.
[info] 7-7: Not following: ./conf/images/conf-app-images.sh was not specified as input (see shellcheck -x).
[info] 7-7: Double quote to prevent globbing and word splitting.
[info] 9-9: Double quote to prevent globbing and word splitting.
[info] 9-9: Double quote to prevent globbing and word splitting.
[info] 9-9: Double quote to prevent globbing and word splitting.
[info] 10-10: Double quote to prevent globbing and word splitting.
[info] 10-10: Double quote to prevent globbing and word splitting.
[info] 11-11: Double quote to prevent globbing and word splitting.
[info] 11-11: Double quote to prevent globbing and word splitting.
JeMPI_Apps/build-all-ci.sh (25)
[style] 13-13: Use -n instead of ! -z.
[info] 14-14: read without -r will mangle backslashes.
[info] 15-15: Double quote to prevent globbing and word splitting.
[info] 15-15: Double quote to prevent globbing and word splitting.
[info] 15-15: Double quote to prevent globbing and word splitting.
[info] 17-17: Double quote to prevent globbing and word splitting.
[info] 17-17: Double quote to prevent globbing and word splitting.
[info] 37-37: Double quote to prevent globbing and word splitting.
[info] 37-37: Double quote to prevent globbing and word splitting.
[info] 41-41: Double quote to prevent globbing and word splitting.
[info] 41-41: Double quote to prevent globbing and word splitting.
[info] 45-45: Double quote to prevent globbing and word splitting.
[info] 45-45: Double quote to prevent globbing and word splitting.
[info] 49-49: Double quote to prevent globbing and word splitting.
[info] 49-49: Double quote to prevent globbing and word splitting.
[info] 53-53: Double quote to prevent globbing and word splitting.
[info] 53-53: Double quote to prevent globbing and word splitting.
[info] 57-57: Double quote to prevent globbing and word splitting.
[info] 57-57: Double quote to prevent globbing and word splitting.
[info] 61-61: Double quote to prevent globbing and word splitting.
[info] 61-61: Double quote to prevent globbing and word splitting.
[info] 65-65: Double quote to prevent globbing and word splitting.
[info] 65-65: Double quote to prevent globbing and word splitting.
[info] 69-69: Double quote to prevent globbing and word splitting.
[info] 69-69: Double quote to prevent globbing and word splitting.
devops/linux/docker/backup_restore/dgraph-backup-api.sh (7)
[info] 8-8: Double quote to prevent globbing and word splitting.
[info] 16-16: Double quote to prevent globbing and word splitting.
[info] 21-21: Not following: d-stack-stop-services.sh was not specified as input (see shellcheck -x).
[info] 29-29: Not following: d-stack-start-backup-restore-api-services.sh was not specified as input (see shellcheck -x).
[info] 38-38: Double quote to prevent globbing and word splitting.
[info] 49-49: Not following: d-stack-start-services.sh was not specified as input (see shellcheck -x).
[info] 57-57: Not following: d-stack-stop-backup-restore-api-services.sh was not specified as input (see shellcheck -x).
devops/linux/docker/backup_restore/postgres-backup.sh (8)
[info] 2-2: Not following: ../conf.env was not specified as input (see shellcheck -x).
[warning] 11-11: DB_NAME appears unused. Verify use (or export if used externally).
[warning] 37-37: backup_file appears unused. Verify use (or export if used externally).
[info] 40-40: Double quote to prevent globbing and word splitting.
[info] 40-40: Double quote to prevent globbing and word splitting.
[info] 40-40: Double quote to prevent globbing and word splitting.
[info] 51-51: Double quote to prevent globbing and word splitting.
[info] 51-51: Double quote to prevent globbing and word splitting.
devops/linux/docker/conf/env/create-env-linux-high-1.sh (7)
[warning] 4-4: Declare and assign separately to avoid masking return values.
[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
[warning] 6-6: Declare and assign separately to avoid masking return values.
[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
[warning] 11-11: Declare and assign separately to avoid masking return values.
[warning] 13-13: Declare and assign separately to avoid masking return values.
[info] 85-85: Double quote to prevent globbing and word splitting.
devops/linux/docker/conf/env/create-env-linux-low-1.sh (7)
[warning] 4-4: Declare and assign separately to avoid masking return values.
[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
[warning] 6-6: Declare and assign separately to avoid masking return values.
[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
[warning] 11-11: Declare and assign separately to avoid masking return values.
[warning] 13-13: Declare and assign separately to avoid masking return values.
[info] 79-79: Double quote to prevent globbing and word splitting.
devops/linux/docker/conf/env/create-env-wsl-low-1.sh (7)
[warning] 4-4: Declare and assign separately to avoid masking return values.
[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
[warning] 6-6: Declare and assign separately to avoid masking return values.
[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
[warning] 11-11: Declare and assign separately to avoid masking return values.
[warning] 13-13: Declare and assign separately to avoid masking return values.
[info] 79-79: Double quote to prevent globbing and word splitting.
devops/linux/docker/conf/images/conf-app-images.sh (1)
[error] 1-1: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
devops/linux/docker/deployment/deploy-local.sh (54)
[error] 1-1: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
[warning] 4-4: Declare and assign separately to avoid masking return values.
[info] 24-24: read without -r will mangle backslashes.
[info] 28-28: read without -r will mangle backslashes.
[info] 49-49: Double quote to prevent globbing and word splitting.
[info] 62-62: Not following: ./.sdkman/bin/sdkman-init.sh was not specified as input (see shellcheck -x).
[warning] 77-77: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 79-79: Not following: hostname-setup.sh was not specified as input (see shellcheck -x).
[warning] 80-80: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 86-86: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[warning] 87-87: ShellCheck can't follow non-constant source. Use a directive to specify location.
[warning] 88-88: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 92-92: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 93-93: Not following: x-swarm-a-set-insecure-registries.sh was not specified as input (see shellcheck -x).
[warning] 94-94: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 100-100: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 101-101: Not following: create.sh was not specified as input (see shellcheck -x).
[info] 101-101: Double quote to prevent globbing and word splitting.
[warning] 102-102: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 111-111: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 112-112: Not following: b-swarm-1-init-node1.sh was not specified as input (see shellcheck -x).
[warning] 113-113: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 119-119: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 121-121: Not following: c-registry-1-create.sh was not specified as input (see shellcheck -x).
[info] 125-125: Not following: a-images-1-pull-from-hub.sh was not specified as input (see shellcheck -x).
[info] 129-129: Not following: c-registry-2-push-hub-images.sh was not specified as input (see shellcheck -x).
[warning] 130-130: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 137-137: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 138-138: Not following: d-stack-1-build-all-reboot.sh was not specified as input (see shellcheck -x).
[warning] 139-139: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 143-143: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 144-144: Not following: d-stack-1-create-db-build-all-reboot.sh was not specified as input (see shellcheck -x).
[warning] 145-145: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[info] 149-149: read without -r will mangle backslashes.
[warning] 153-153: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[warning] 157-157: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 160-160: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
[info] 165-165: read without -r will mangle backslashes.
[warning] 169-169: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[warning] 173-173: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 209-209: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 210-210: Not following: d-stack-3-reboot.sh was not specified as input (see shellcheck -x).
[warning] 211-211: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 215-215: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 216-216: Not following: d-stack-3-down.sh was not specified as input (see shellcheck -x).
[warning] 217-217: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 223-223: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 224-224: Not following: dgraph-backup-api.sh was not specified as input (see shellcheck -x).
[info] 224-224: Double quote to prevent globbing and word splitting.
[info] 225-225: Double quote to prevent globbing and word splitting.
[warning] 226-226: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
[warning] 241-241: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
[info] 243-243: Not following: b-swarm-2-leave.sh was not specified as input (see shellcheck -x).
[warning] 244-244: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
devops/linux/docker/deployment/down/d-stack-stop-backup-restore-api-services.sh (2)
[info] 8-8: Double quote to prevent globbing and word splitting.
[info] 10-10: Not following: ./conf.env was not specified as input (see shellcheck -x).
devops/linux/docker/deployment/down/d-stack-stop-services.sh (2)
[info] 8-8: Double quote to prevent globbing and word splitting.
[info] 10-10: Not following: ./conf.env was not specified as input (see shellcheck -x).
devops/linux/docker/deployment/reboot/d-stack-start-backup-restore-api-services.sh (2)
[info] 8-8: Double quote to prevent globbing and word splitting.
[info] 10-10: Not following: ./conf.env was not specified as input (see shellcheck -x).
devops/linux/docker/deployment/reboot/d-stack-start-services.sh (2)
[info] 8-8: Double quote to prevent globbing and word splitting.
[info] 10-10: Not following: ./conf.env was not specified as input (see shellcheck -x).
Additional comments not posted (20)
JeMPI_Apps/JeMPI_BackupRestoreAPI/build.sh (1)
9-11: Verify the usage ofJAR_FILE,APP_IMAGE, andAPPto ensure they are not unused variables.Verification successful
The variables
JAR_FILE,APP_IMAGE, andAPPdefined inJeMPI_Apps/JeMPI_BackupRestoreAPI/build.share indeed used in various scripts across the repository. Here are some specific instances:
JeMPI_Apps/build-app-image.shJeMPI_Apps/build-scala-app-image.shJeMPI_Apps/JeMPI_UI/push.shJeMPI_Apps/JeMPI_Linker/build.shJeMPI_Apps/JeMPI_Linker/push.shJeMPI_Apps/JeMPI_BackupRestoreAPI/build.shJeMPI_Apps/JeMPI_EM_Scala/build.shJeMPI_Apps/JeMPI_EM_Scala/push.shJeMPI_Apps/JeMPI_ETL/build.shJeMPI_Apps/JeMPI_ETL/push.shJeMPI_Apps/JeMPI_Bootstrapper/build.shJeMPI_Apps/JeMPI_Bootstrapper/push.shJeMPI_Apps/JeMPI_API_KC/build.shJeMPI_Apps/JeMPI_Controller/build.shJeMPI_Apps/JeMPI_Controller/push.shJeMPI_Apps/JeMPI_AsyncReceiver/build.shJeMPI_Apps/JeMPI_AsyncReceiver/push.shJeMPI_Apps/JeMPI_API/build.shThese variables are used for various purposes such as copying JAR files, exporting environment variables, tagging Docker images, and pushing Docker images to a registry.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Check for usage of JAR_FILE, APP_IMAGE, and APP in the current script directory grep -E 'JAR_FILE|APP_IMAGE|APP' *.shLength of output: 79
Script:
#!/bin/bash # Search for usage of JAR_FILE, APP_IMAGE, and APP in all shell scripts in the repository rg 'JAR_FILE|APP_IMAGE|APP' --glob '*.sh'Length of output: 9617
JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/resources/log4j2.properties (1)
1-14: The logging configuration is set up correctly and follows standard practices.JeMPI_Apps/build-all-java.sh (3)
8-8: The configuration file is correctly copied to the BackupRestoreAPI project's resources directory.
39-41: The integration of the BackupRestoreAPI project into the build process is handled correctly.
67-69: The integration of the BackupRestoreAPI project into the deployment process is handled correctly.JeMPI_Apps/build-all-ci.sh (1)
27-27: Ensure the new configuration file is properly integrated into the BackupRestoreAPI module.devops/linux/docker/conf/images/conf-app-images.sh (1)
52-54: The environment variable declarations for the Backup Restore API are correctly added.JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/MapError.java (1)
21-39: The error mapping logic is implemented correctly and covers a comprehensive range of error scenarios.devops/linux/docker/conf/env/create-env-linux-low-1.sh (1)
36-36: The addition ofBACKUP_RESTORE_API_HTTP_PORTaligns with the introduction of the new backup and restore API module.devops/linux/docker/conf/env/create-env-wsl-low-1.sh (1)
36-36: Consistent addition ofBACKUP_RESTORE_API_HTTP_PORTacross different environments supports the new API module effectively.devops/linux/docker/conf/env/create-env-linux-high-1.sh (1)
42-42: EnsuringBACKUP_RESTORE_API_HTTP_PORTis set in high-availability configurations aligns with scalable deployment needs.JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/api/BackupRestoreAPI.java (1)
16-71: TheBackupRestoreAPIclass is well-implemented with robust logging and error handling, suitable for managing the API's lifecycle.JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/JsonFieldsConfig.java (1)
15-90: TheJsonFieldsConfigclass is effectively implemented with comprehensive error handling and useful utilities for JSON field management.JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/api/HttpServer.java (1)
26-87: TheHttpServerclass is well-implemented with robust handling of HTTP requests, responses, and CORS settings.JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/Routes.java (1)
23-85: TheRoutesclass is effectively implemented, managing API routes with clear and robust error handling.JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java (1)
27-146: Ensure that theBackEndclass properly handles errors and logs them appropriately.Verification successful
The
BackEndclass properly handles errors and logs them appropriately.
- Verified the presence of
catchblocks and correspondingLOGGER.errorstatements.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify error handling in the `BackEnd` class. # Test: Search for error handling patterns. Expect: Proper logging and exception management. rg --type java 'try' 'JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java'Length of output: 435
Script:
#!/bin/bash # Description: Verify error handling and logging in the `BackEnd` class. # Test: Search for catch blocks and logging statements. Expect: Proper logging in catch blocks. rg --type java 'catch' 'JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java' rg --type java 'LOGGER.error' 'JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java'Length of output: 1193
devops/linux/docker/conf/stack/docker-stack-low-0.yml (2)
578-616: The configuration for thebackup-restore-apiservice appears comprehensive and aligns with the PR objectives.
74-77: Verify the necessity and correctness of the new port configuration for HAProxy.Verification successful
The verification confirms that port 50010 is referenced in the Docker Compose files and is intended for use by HAProxy. There are no indications of conflicts with other services using the same port.
docker-stack-low-0.yml: Port 50010 is mapped for HAProxy.docker-stack-low-1.yml: Port 50010 is also mapped for HAProxy.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify if port 50010 is required and correctly configured for HAProxy. # Test: Search for references to port 50010 in the configuration files. rg --type yaml '50010' devops/linux/docker/conf/Length of output: 358
Script:
#!/bin/bash # Description: Verify HAProxy configuration and usage of port 50010. # Test 1: Search for HAProxy configurations in the Docker Compose files. rg --type yaml 'haproxy' devops/linux/docker/conf/ # Test 2: Check for any other services using port 50010. rg --type yaml '50010' devops/linux/docker/conf/Length of output: 1423
devops/linux/docker/conf/stack/docker-stack-low-1.yml (2)
578-616: The configuration for thebackup-restore-apiservice appears comprehensive and aligns with the PR objectives.
74-77: Verify the necessity and correctness of the new port configuration for HAProxy.Verification successful
The port configuration for HAProxy on port 50010 is correctly set up and necessary.
- Port 50010 is referenced in the HAProxy configuration file (
haproxy.cfg) with the linebind *:50010.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify if port 50010 is required and correctly configured for HAProxy. # Test: Search for references to port 50010 in the configuration files. rg --type yaml '50010' devops/linux/docker/conf/Length of output: 358
Script:
#!/bin/bash # Description: Verify if HAProxy is configured to use port 50010 and ensure there are no conflicts. # Test: Search for HAProxy configurations and references to port 50010 in the repository. rg -i 'haproxy' devops/linux/docker/conf/ rg -i '50010' devops/linux/docker/conf/Length of output: 4014
...ps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlAuditTrail.java
Outdated
Show resolved
Hide resolved
...ps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlAuditTrail.java
Outdated
Show resolved
Hide resolved
...I_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlClient.java
Outdated
Show resolved
Hide resolved
...ps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlAuditTrail.java
Outdated
Show resolved
Hide resolved
JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/AppConfig.java
Show resolved
Hide resolved
devops/linux/docker/deployment/reboot/d-stack-start-services.sh
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
Out of diff range and nitpick comments (1)
devops/linux/docker/deployment/build_and_reboot/d-stack-2-build-java.sh (1)
Line range hint
6-6: Consider verifying the existence of../../conf.envbefore sourcing it to prevent runtime errors if the file is missing.if [ -f "../../conf.env" ]; then source ../../conf.env else echo "Error: Configuration file not found." exit 1 fi
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/pom.xml (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java (1 hunks)
- devops/linux/docker/deployment/build_and_reboot/d-stack-2-build-java.sh (1 hunks)
Files skipped from review as they are similar to previous changes (2)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/pom.xml
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java
Additional Context Used
ShellCheck (2)
devops/linux/docker/deployment/build_and_reboot/d-stack-2-build-java.sh (2)
[info] 6-6: Not following: ../../conf.env was not specified as input (see shellcheck -x).
[info] 13-13: Not following: ./build-all-java.sh was not specified as input (see shellcheck -x).
There was a problem hiding this comment.
Actionable comments posted: 0
Outside diff range and nitpick comments (4)
JeMPI_Apps/build-all-ci.sh (3)
Line range hint
13-13: Consider using-ninstead of! -zfor string non-empty check for better readability.- if [ ! -z "$tag_to_use" ]; then + if [ -n "$tag_to_use" ]; then
Line range hint
14-14: Add-rtoreadto prevent backslashes from being mangled.- IFS=':' read -a image_details <<< "$2" + IFS=':' read -ra image_details <<< "$2"
Line range hint
15-15: Ensure to double-quote variables to prevent globbing and word splitting.- docker tag $1 $2 + docker tag "$1" "$2"Also applies to: 17-17, 38-38, 42-42, 46-46, 50-50, 54-54, 58-58, 62-62, 66-66, 70-70
devops/linux/docker/conf/images/conf-app-images.sh (1)
Line range hint
1-1: Add a shebang to specify the intended shell.+ #!/bin/bash
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (7)
- JeMPI_Apps/build-all-ci.sh (1 hunks)
- JeMPI_Apps/build-all-java.sh (3 hunks)
- JeMPI_Apps/pom.xml (1 hunks)
- devops/linux/docker/conf/env/conf-env-low-1-pc.template (1 hunks)
- devops/linux/docker/conf/images/conf-app-images.sh (1 hunks)
- devops/linux/docker/conf/stack/docker-stack-low-0.yml (2 hunks)
- devops/linux/docker/conf/stack/docker-stack-low-1.yml (2 hunks)
Files skipped from review as they are similar to previous changes (5)
- JeMPI_Apps/build-all-java.sh
- JeMPI_Apps/pom.xml
- devops/linux/docker/conf/env/conf-env-low-1-pc.template
- devops/linux/docker/conf/stack/docker-stack-low-0.yml
- devops/linux/docker/conf/stack/docker-stack-low-1.yml
Additional Context Used
ShellCheck (26)
JeMPI_Apps/build-all-ci.sh (25)
[style] 13-13: Use -n instead of ! -z.
[info] 14-14: read without -r will mangle backslashes.
[info] 15-15: Double quote to prevent globbing and word splitting.
[info] 15-15: Double quote to prevent globbing and word splitting.
[info] 15-15: Double quote to prevent globbing and word splitting.
[info] 17-17: Double quote to prevent globbing and word splitting.
[info] 17-17: Double quote to prevent globbing and word splitting.
[info] 38-38: Double quote to prevent globbing and word splitting.
[info] 38-38: Double quote to prevent globbing and word splitting.
[info] 42-42: Double quote to prevent globbing and word splitting.
[info] 42-42: Double quote to prevent globbing and word splitting.
[info] 46-46: Double quote to prevent globbing and word splitting.
[info] 46-46: Double quote to prevent globbing and word splitting.
[info] 50-50: Double quote to prevent globbing and word splitting.
[info] 50-50: Double quote to prevent globbing and word splitting.
[info] 54-54: Double quote to prevent globbing and word splitting.
[info] 54-54: Double quote to prevent globbing and word splitting.
[info] 58-58: Double quote to prevent globbing and word splitting.
[info] 58-58: Double quote to prevent globbing and word splitting.
[info] 62-62: Double quote to prevent globbing and word splitting.
[info] 62-62: Double quote to prevent globbing and word splitting.
[info] 66-66: Double quote to prevent globbing and word splitting.
[info] 66-66: Double quote to prevent globbing and word splitting.
[info] 70-70: Double quote to prevent globbing and word splitting.
[info] 70-70: Double quote to prevent globbing and word splitting.
devops/linux/docker/conf/images/conf-app-images.sh (1)
[error] 1-1: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
Additional comments not posted (2)
JeMPI_Apps/build-all-ci.sh (1)
27-27: LGTM! Consistent with the configuration setup for other services.devops/linux/docker/conf/images/conf-app-images.sh (1)
52-54: LGTM! Consistent with the environment variable setup for other services.
…restore-endpoint-in-backup-api
…restore-endpoint-in-backup-api
There was a problem hiding this comment.
Actionable comments posted: 1
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- devops/linux/docker/deployment/deploy-local.sh (5 hunks)
Additional context used
Shellcheck
devops/linux/docker/deployment/deploy-local.sh
[error] 1-1: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive. (SC2148)
[warning] 4-4: Declare and assign separately to avoid masking return values. (SC2155)
[warning] 77-77: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 80-80: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 86-86: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 87-87: ShellCheck can't follow non-constant source. Use a directive to specify location. (SC1090)
[warning] 88-88: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 92-92: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 94-94: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 100-100: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 102-102: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 111-111: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 113-113: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 119-119: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 130-130: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 137-137: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 139-139: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 143-143: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 145-145: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 153-153: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 157-157: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 160-160: Use 'cd ... || exit' or 'cd ... || return' in case cd fails. (SC2164)
[warning] 169-169: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 173-173: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 207-207: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 209-209: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 213-213: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 215-215: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 221-221: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 224-224: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 239-239: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 242-242: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
Additional comments not posted (2)
devops/linux/docker/deployment/deploy-local.sh (2)
9-9: Ensure consistency in environment configuration file naming.This change aligns the environment configuration file name with the new setup. Confirm that
create-env-linux-low-1.shexists and is correctly configured.
219-225: Review the new backup functionality for syntax and operational efficiency.The addition of a timestamp to the backup process enhances traceability and organization of backup files. Ensure that the backup scripts
dgraph-backup-api.shandpostgres-backup.share properly handling the passed timestamp.Tools
Shellcheck
[warning] 221-221: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 224-224: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
There was a problem hiding this comment.
Actionable comments posted: 5
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (6)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/JsonFieldsConfig.java (1 hunks)
- JeMPI_Apps/build-all-java.sh (3 hunks)
- devops/linux/docker/conf/stack/docker-stack-high-0.yml (1 hunks)
- devops/linux/docker/conf/stack/docker-stack-low-0.yml (2 hunks)
- devops/linux/docker/conf/stack/docker-stack-low-1.yml (2 hunks)
- devops/linux/docker/deployment/deploy-local.sh (6 hunks)
Additional context used
Shellcheck
devops/linux/docker/deployment/deploy-local.sh
[error] 1-1: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive. (SC2148)
[warning] 4-4: Declare and assign separately to avoid masking return values. (SC2155)
[warning] 77-77: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 80-80: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 86-86: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 87-87: ShellCheck can't follow non-constant source. Use a directive to specify location. (SC1090)
[warning] 88-88: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 92-92: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 94-94: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 100-100: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 102-102: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 111-111: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 113-113: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 119-119: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 130-130: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 136-136: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 138-138: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 142-142: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 144-144: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 152-152: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 156-156: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 159-159: Use 'cd ... || exit' or 'cd ... || return' in case cd fails. (SC2164)
[warning] 168-168: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 172-172: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 206-206: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 208-208: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 212-212: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 214-214: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 220-220: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 223-223: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
[warning] 238-238: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails. (SC2164)
[warning] 241-241: Use 'popd ... || exit' or 'popd ... || return' in case popd fails. (SC2164)
yamllint
devops/linux/docker/conf/stack/docker-stack-low-0.yml
[warning] 57-57: wrong indentation: expected 4 but found 6 (indentation)
[warning] 59-59: wrong indentation: expected 4 but found 6 (indentation)
[warning] 62-62: wrong indentation: expected 4 but found 6 (indentation)
[warning] 87-87: wrong indentation: expected 4 but found 6 (indentation)
[error] 97-97: trailing spaces (trailing-spaces)
[warning] 102-102: wrong indentation: expected 4 but found 6 (indentation)
[error] 109-109: trailing spaces (trailing-spaces)
[warning] 119-119: wrong indentation: expected 4 but found 6 (indentation)
[warning] 136-136: wrong indentation: expected 8 but found 10 (indentation)
[error] 137-137: trailing spaces (trailing-spaces)
[error] 210-210: trailing spaces (trailing-spaces)
[error] 231-231: trailing spaces (trailing-spaces)
[error] 269-269: trailing spaces (trailing-spaces)
[error] 282-282: trailing spaces (trailing-spaces)
[error] 303-303: trailing spaces (trailing-spaces)
[error] 330-330: trailing spaces (trailing-spaces)
[error] 351-351: trailing spaces (trailing-spaces)
[error] 388-388: trailing spaces (trailing-spaces)
[error] 437-437: trailing spaces (trailing-spaces)
[error] 488-488: trailing spaces (trailing-spaces)
[warning] 550-550: wrong indentation: expected 4 but found 6 (indentation)
[error] 561-561: trailing spaces (trailing-spaces)
[error] 562-562: trailing spaces (trailing-spaces)
[warning] 583-583: wrong indentation: expected 8 but found 10 (indentation)
[warning] 588-588: wrong indentation: expected 4 but found 6 (indentation)
[error] 593-593: trailing spaces (trailing-spaces)
[error] 614-614: trailing spaces (trailing-spaces)
[error] 660-660: trailing spaces (trailing-spaces)
[warning] 669-669: wrong indentation: expected 4 but found 6 (indentation)
[warning] 685-685: wrong indentation: expected 8 but found 10 (indentation)
[warning] 686-686: too many blank lines (1 > 0) (empty-lines)
devops/linux/docker/conf/stack/docker-stack-low-1.yml
[warning] 57-57: wrong indentation: expected 4 but found 6 (indentation)
[warning] 59-59: wrong indentation: expected 4 but found 6 (indentation)
[warning] 62-62: wrong indentation: expected 4 but found 6 (indentation)
[warning] 87-87: wrong indentation: expected 4 but found 6 (indentation)
[error] 97-97: trailing spaces (trailing-spaces)
[warning] 102-102: wrong indentation: expected 4 but found 6 (indentation)
[error] 109-109: trailing spaces (trailing-spaces)
[warning] 119-119: wrong indentation: expected 4 but found 6 (indentation)
[warning] 136-136: wrong indentation: expected 8 but found 10 (indentation)
[error] 137-137: trailing spaces (trailing-spaces)
[error] 210-210: trailing spaces (trailing-spaces)
[error] 231-231: trailing spaces (trailing-spaces)
[error] 269-269: trailing spaces (trailing-spaces)
[error] 282-282: trailing spaces (trailing-spaces)
[error] 303-303: trailing spaces (trailing-spaces)
[error] 330-330: trailing spaces (trailing-spaces)
[error] 351-351: trailing spaces (trailing-spaces)
[error] 388-388: trailing spaces (trailing-spaces)
[error] 437-437: trailing spaces (trailing-spaces)
[error] 488-488: trailing spaces (trailing-spaces)
[warning] 550-550: wrong indentation: expected 4 but found 6 (indentation)
[error] 561-561: trailing spaces (trailing-spaces)
[error] 562-562: trailing spaces (trailing-spaces)
[warning] 583-583: wrong indentation: expected 8 but found 10 (indentation)
[warning] 588-588: wrong indentation: expected 4 but found 6 (indentation)
[error] 593-593: trailing spaces (trailing-spaces)
[error] 614-614: trailing spaces (trailing-spaces)
[error] 660-660: trailing spaces (trailing-spaces)
[warning] 669-669: wrong indentation: expected 4 but found 6 (indentation)
[error] 674-674: trailing spaces (trailing-spaces)
[warning] 690-690: wrong indentation: expected 8 but found 10 (indentation)
[warning] 691-691: too many blank lines (1 > 0) (empty-lines)
devops/linux/docker/conf/stack/docker-stack-high-0.yml
[warning] 57-57: wrong indentation: expected 4 but found 6 (indentation)
[warning] 59-59: wrong indentation: expected 4 but found 6 (indentation)
[warning] 62-62: wrong indentation: expected 4 but found 6 (indentation)
[warning] 83-83: wrong indentation: expected 4 but found 6 (indentation)
[error] 93-93: trailing spaces (trailing-spaces)
[warning] 98-98: wrong indentation: expected 4 but found 6 (indentation)
[warning] 115-115: wrong indentation: expected 4 but found 6 (indentation)
[warning] 132-132: wrong indentation: expected 8 but found 10 (indentation)
[error] 133-133: trailing spaces (trailing-spaces)
[error] 135-135: trailing spaces (trailing-spaces)
[warning] 138-138: wrong indentation: expected 4 but found 6 (indentation)
[error] 141-141: trailing spaces (trailing-spaces)
[error] 146-146: trailing spaces (trailing-spaces)
[error] 151-151: trailing spaces (trailing-spaces)
[warning] 155-155: wrong indentation: expected 4 but found 6 (indentation)
[warning] 172-172: wrong indentation: expected 8 but found 10 (indentation)
[error] 173-173: trailing spaces (trailing-spaces)
[error] 175-175: trailing spaces (trailing-spaces)
[warning] 178-178: wrong indentation: expected 4 but found 6 (indentation)
[error] 181-181: trailing spaces (trailing-spaces)
[error] 186-186: trailing spaces (trailing-spaces)
[error] 191-191: trailing spaces (trailing-spaces)
[warning] 195-195: wrong indentation: expected 4 but found 6 (indentation)
[warning] 212-212: wrong indentation: expected 8 but found 10 (indentation)
[error] 278-278: trailing spaces (trailing-spaces)
[error] 311-311: trailing spaces (trailing-spaces)
[error] 418-418: trailing spaces (trailing-spaces)
[error] 439-439: trailing spaces (trailing-spaces)
[error] 473-473: trailing spaces (trailing-spaces)
[error] 503-503: trailing spaces (trailing-spaces)
[error] 547-547: trailing spaces (trailing-spaces)
[error] 580-580: trailing spaces (trailing-spaces)
[error] 625-625: trailing spaces (trailing-spaces)
[error] 672-672: trailing spaces (trailing-spaces)
[warning] 730-730: wrong indentation: expected 4 but found 6 (indentation)
[error] 741-741: trailing spaces (trailing-spaces)
[error] 742-742: trailing spaces (trailing-spaces)
[warning] 763-763: wrong indentation: expected 8 but found 10 (indentation)
[warning] 768-768: wrong indentation: expected 4 but found 6 (indentation)
[error] 785-785: trailing spaces (trailing-spaces)
[error] 789-789: trailing spaces (trailing-spaces)
[error] 835-835: trailing spaces (trailing-spaces)
[warning] 844-844: wrong indentation: expected 4 but found 6 (indentation)
[warning] 860-860: wrong indentation: expected 8 but found 10 (indentation)
Additional comments not posted (4)
JeMPI_Apps/build-all-java.sh (2)
35-37: Ensure robust error handling for build script.The script uses
./build.sh || exit 1to handle potential build failures by exiting with an error code. This is a good practice as it ensures that the script stops execution if a critical component fails to build, preventing cascading errors in dependent components.
63-65: Confirm the push script for JeMPI_BackupRestoreAPI.The script pushes the build output to a repository or server, which is crucial for deployment. Ensure that the
./push.shscript is correctly configured to handle the newJeMPI_BackupRestoreAPIcomponent.Verification successful
The
push.shscript forJeMPI_BackupRestoreAPIinJeMPI_Apps/JeMPI_BackupRestoreAPIis configured to tag and push a Docker image to a registry, followed by removing the image locally. This setup appears to be appropriate for handling theJeMPI_BackupRestoreAPIcomponent.
- The script sets error and unset variable handling.
- It sources necessary environment configurations.
- It tags, pushes, and removes the Docker image as expected.
Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the push script for JeMPI_BackupRestoreAPI. # Test: Search for push script configuration. Expect: Correct configuration for JeMPI_BackupRestoreAPI. rg --type bash $'JeMPI_BackupRestoreAPI/./push.sh'Length of output: 78
Script:
#!/bin/bash # Locate the push.sh script within the JeMPI_BackupRestoreAPI directory fd push.sh JeMPI_BackupRestoreAPI --exec cat {}Length of output: 161
Script:
#!/bin/bash # Locate the correct directory for JeMPI_BackupRestoreAPI fd --type d JeMPI_BackupRestoreAPILength of output: 71
Script:
#!/bin/bash # Locate the push.sh script within the JeMPI_Apps/JeMPI_BackupRestoreAPI directory fd push.sh JeMPI_Apps/JeMPI_BackupRestoreAPI --exec cat {}Length of output: 393
devops/linux/docker/conf/stack/docker-stack-low-0.yml (1)
611-654: Validate configuration for backup-restore-api service.The
backup-restore-apiservice configuration includes environment variables, network settings, and deployment constraints. Ensure that all configuration parameters are correctly set and match the intended deployment environment.Verification successful
The configuration for the
backup-restore-apiservice indocker-stack-low-0.ymlhas been successfully extracted and appears to be correctly set up with the necessary environment variables, network settings, and deployment constraints.
- The environment variables are defined appropriately.
- Network settings and volume bindings are specified.
- Deployment constraints, such as mode, replicas, resources, update configuration, restart policy, and placement constraints, are included.
This confirms that the configuration is accurate and matches the intended deployment environment.
Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the configuration for the backup-restore-api service. # Test: Search for configuration settings in the docker-stack file. Expect: Correct settings for backup-restore-api. rg --type yml $'backup-restore-api'Length of output: 62
Script:
#!/bin/bash # Description: Verify the configuration for the backup-restore-api service. # Test: Search for configuration settings in the docker-stack file. Expect: Correct settings for backup-restore-api. rg --type yaml $'backup-restore-api'Length of output: 264
Script:
#!/bin/bash # Description: Extract and review the configuration details of the `backup-restore-api` service in `docker-stack-low-0.yml`. # Extract the lines containing the `backup-restore-api` configuration from `docker-stack-low-0.yml`. rg --type yaml -A 30 $'backup-restore-api' devops/linux/docker/conf/stack/docker-stack-low-0.ymlLength of output: 1126
Tools
yamllint
[error] 614-614: trailing spaces (trailing-spaces)
devops/linux/docker/conf/stack/docker-stack-low-1.yml (1)
74-77: Addition of New Service Port ConfigurationThe new port
50010has been added for thebackup-restore-apiservice. Ensure that this port is not conflicting with any existing services and is properly documented in the service configuration.
.../JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/JsonFieldsConfig.java
Outdated
Show resolved
Hide resolved
.../JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/JsonFieldsConfig.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- JeMPI_Apps/JeMPI_Controller/src/main/java/org/jembi/jempi/controller/SPInteractions.java (1 hunks)
Additional comments not posted (2)
JeMPI_Apps/JeMPI_Controller/src/main/java/org/jembi/jempi/controller/SPInteractions.java (2)
48-52: Improved null handling in upload workflow conditionThe addition of a null check before accessing
uploadConfigis a good practice to avoid potentialNullPointerExceptions. This change ensures that the application behaves correctly even whenuploadConfigis not set, which could be a common scenario in dynamic configurations.This change aligns with best practices for defensive programming and helps ensure the robustness of the application.
59-63: Refined logic for upload workflow conditionThe updated condition now handles cases where
uploadConfigmight be null, which previously would have led to aNullPointerException. By using a logical OR, the method now safely checks for the condition, allowing the workflow to proceed correctly even ifuploadConfigis absent or does not match the expected workflow.This change not only prevents runtime errors but also makes the code more maintainable by clearly handling cases of missing configuration.
…restore-endpoint-in-backup-api
There was a problem hiding this comment.
Actionable comments posted: 3
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (10)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlClient.java (1 hunks)
- JeMPI_Apps/JeMPI_LibAPI/src/main/java/org/jembi/jempi/libapi/PsqlClient.java (1 hunks)
- devops/linux/docker/backup_restore/dgraph-backup-api.sh (1 hunks)
- devops/linux/docker/backup_restore/dgraph-backup.py (1 hunks)
- devops/linux/docker/backup_restore/dgraph-restore-api.py (1 hunks)
- devops/linux/docker/conf/stack/docker-stack-high-0.yml (13 hunks)
- devops/linux/docker/conf/stack/docker-stack-high-1.yml (19 hunks)
- devops/linux/docker/deployment/deploy-local.sh (7 hunks)
- devops/linux/docker/deployment/reboot/d-stack-start-backup-restore-api-services.sh (1 hunks)
- devops/linux/docker/deployment/reboot/d-stack-start-services.sh (1 hunks)
Files skipped from review as they are similar to previous changes (3)
- devops/linux/docker/backup_restore/dgraph-restore-api.py
- devops/linux/docker/conf/stack/docker-stack-high-0.yml
- devops/linux/docker/deployment/deploy-local.sh
Additional context used
yamllint
devops/linux/docker/conf/stack/docker-stack-high-1.yml
[warning] 131-131: wrong indentation: expected 10 but found 8
(indentation)
[error] 132-132: trailing spaces
(trailing-spaces)
[warning] 171-171: wrong indentation: expected 10 but found 8
(indentation)
[error] 172-172: trailing spaces
(trailing-spaces)
[warning] 211-211: wrong indentation: expected 10 but found 8
(indentation)
[warning] 243-243: wrong indentation: expected 10 but found 8
(indentation)
[warning] 276-276: wrong indentation: expected 10 but found 8
(indentation)
[error] 277-277: trailing spaces
(trailing-spaces)
[warning] 309-309: wrong indentation: expected 10 but found 8
(indentation)
[error] 310-310: trailing spaces
(trailing-spaces)
[warning] 342-342: wrong indentation: expected 10 but found 8
(indentation)
[warning] 375-375: wrong indentation: expected 10 but found 8
(indentation)
[warning] 408-408: wrong indentation: expected 10 but found 8
(indentation)
[warning] 444-444: wrong indentation: expected 6 but found 4
(indentation)
[warning] 482-482: wrong indentation: expected 6 but found 4
(indentation)
[warning] 530-530: wrong indentation: expected 6 but found 4
(indentation)
[warning] 563-563: wrong indentation: expected 6 but found 4
(indentation)
[warning] 616-616: wrong indentation: expected 6 but found 4
(indentation)
[warning] 667-667: wrong indentation: expected 6 but found 4
(indentation)
[warning] 733-733: wrong indentation: expected 6 but found 4
(indentation)
[warning] 805-805: wrong indentation: expected 6 but found 4
(indentation)
Additional comments not posted (15)
devops/linux/docker/deployment/reboot/d-stack-start-backup-restore-api-services.sh (6)
1-1: Good use of shebang for portability.Using
#!/bin/bashensures that the script is executed using Bash, which supports the features used in this script.
3-4: Proper use of error handling flags.The use of
set -eandset -uis a best practice for bash scripts as it helps in handling errors and undefined variables effectively.
6-8: Good practice to use pushd for directory management.Using
pushdandpopdhelps in handling directory changes more cleanly compared tocd, especially in scripts that change directories multiple times.
10-15: Proper file existence check before sourcing.The script correctly checks for the existence of
conf.envbefore attempting to source it, which prevents potential errors from occurring if the file is missing.
19-19: Docker service scaling command is correct.The command
docker service scale jempi_backup-restore-api=1is syntactically correct and aligns with Docker best practices for managing service scale.
17-17: Ensure environment variables are used after sourcingconf.env.It's important to verify that any environment variables defined in
conf.envare actually utilized in the script after sourcing. This is crucial for the script's functionality.Verification successful
Ensure environment variables are used after sourcing
conf.env.The environment variables sourced from
conf.envare utilized in the scriptdevops/linux/docker/deployment/reboot/d-stack-start-backup-restore-api-services.shas expected.
devops/linux/docker/deployment/reboot/d-stack-start-backup-restore-api-services.sh
docker service scale jempi_backup-restore-api=1Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the usage of environment variables defined in `conf.env`. # Test: Search for the usage of environment variables. Expect: At least one usage of an environment variable. rg --type bash -A 5 $'source ./conf.env'Length of output: 70
Script:
#!/bin/bash # Description: Verify the usage of environment variables defined in `conf.env`. # Test: Search for the usage of environment variables after sourcing `conf.env`. rg --type sh -A 5 $'source ./conf.env'Length of output: 46341
JeMPI_Apps/JeMPI_LibAPI/src/main/java/org/jembi/jempi/libapi/PsqlClient.java (1)
41-41: Improved error logging:The change to log the error message with the URL only (without user and password) enhances security by not exposing sensitive credentials in logs. This is a best practice for secure logging.
JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/PsqlClient.java (1)
41-41: Good practice in error handling:The implementation correctly logs the URL without exposing sensitive details such as username or password. This is consistent with security best practices.
devops/linux/docker/deployment/reboot/d-stack-start-services.sh (2)
10-15: Good implementation of configuration file check.The script now correctly checks for the existence of
conf.envbefore attempting to source it, which is a good practice to avoid runtime errors.
17-21: Ensure services are scaled correctly.The script scales services
jempi_apiandjempi_async-receiverback to 1. Ensure that this is the intended scale and that it does not disrupt ongoing operations.This script checks the logs of the scaled services to ensure that there are no error messages indicating disruption.
devops/linux/docker/backup_restore/dgraph-backup-api.sh (2)
18-56: Structured and modular script with good error handling.The script is well-structured into functions for stopping services, starting backup services, performing the backup, and then restarting the services. Each function includes proper directory management and error handling.
15-16: Check the availability of the Python command.The script dynamically finds the Python command. Ensure that this command is available and points to the correct Python version needed for
dgraph-backup.py.This script checks the Python version to ensure that the correct Python interpreter is used for the backup script.
devops/linux/docker/backup_restore/dgraph-backup.py (3)
8-11: Proper use of environment variables.The script correctly loads environment variables from
conf.envand uses them to configure the backup process.
23-33: Enhanced error handling in data fetching.The function
fetch_data_for_idnow includes improved error handling, logging errors when the API request fails. This is crucial for debugging and monitoring.
35-56: Robust backup process with error handling.The function
backup_dgraph_datahandles errors effectively, ensuring that the backup process is robust. It logs errors and continues processing, which is essential for resilience.
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/Routes.java (1 hunks)
Files skipped from review as they are similar to previous changes (2)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/Routes.java
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- JeMPI_Apps/JeMPI_Bootstrapper/src/main/resources/data/kafka/kafkaBootStrapConfig.json (1 hunks)
Additional comments not posted (2)
JeMPI_Apps/JeMPI_Bootstrapper/src/main/resources/data/kafka/kafkaBootStrapConfig.json (2)
38-43: Review of new Kafka topic:JeMPI-interaction-validate.The configuration for the
JeMPI-interaction-validatetopic has been added with the following settings:
- Partition: 30
- Replications: 1
- Retention Period: 86400000 ms (1 day)
- Segment Size: 4194304 bytes
These settings are consistent with the other high-traffic topics like
JeMPI-interaction-linker, suggesting that this topic is expected to handle a similar volume of messages.
45-50: Review of new Kafka topic:JeMPI-interaction-match.The configuration for the
JeMPI-interaction-matchtopic has been added with identical settings to theJeMPI-interaction-validatetopic:
- Partition: 30
- Replications: 1
- Retention Period: 86400000 ms (1 day)
- Segment Size: 4194304 bytes
This configuration also aligns well with other high-traffic topics, ensuring consistency across the board.
There was a problem hiding this comment.
Actionable comments posted: 2
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- devops/linux/docker/backup_restore/dgraph-restore-api.py (1 hunks)
- devops/linux/docker/backup_restore/dgraph-restore-api.sh (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- devops/linux/docker/backup_restore/dgraph-restore-api.py
Additional context used
Shellcheck
devops/linux/docker/backup_restore/dgraph-restore-api.sh
[warning] 5-5: BACKUP_DATE_TIME appears unused. Verify use (or export if used externally).
(SC2034)
[warning] 7-7: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
[warning] 18-18: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 21-21: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 25-25: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 28-28: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 32-32: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 35-35: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 40-40: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 43-43: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 50-50: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 53-53: Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.
(SC2010)
[warning] 55-55: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 70-70: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 80-80: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (5)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/Ask.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java (1 hunks)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/Routes.java (1 hunks)
- devops/linux/docker/backup_restore/dgraph-backup.py (1 hunks)
- devops/linux/docker/backup_restore/dgraph-restore-api.py (1 hunks)
Files skipped from review as they are similar to previous changes (5)
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/Ask.java
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/BackEnd.java
- JeMPI_Apps/JeMPI_BackupRestoreAPI/src/main/java/org/jembi/jempi/backuprestoreapi/Routes.java
- devops/linux/docker/backup_restore/dgraph-backup.py
- devops/linux/docker/backup_restore/dgraph-restore-api.py
There was a problem hiding this comment.
Actionable comments posted: 6
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- devops/linux/docker/backup_restore/dgraph-backup-api.py (1 hunks)
- devops/linux/docker/backup_restore/dgraph-backup-api.sh (1 hunks)
- devops/linux/docker/backup_restore/dgraph-restore-api.sh (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- devops/linux/docker/backup_restore/dgraph-backup-api.sh
Additional context used
Shellcheck
devops/linux/docker/backup_restore/dgraph-restore-api.sh
[warning] 5-5: BACKUP_DATE_TIME appears unused. Verify use (or export if used externally).
(SC2034)
[warning] 7-7: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
[warning] 18-18: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 21-21: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 25-25: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 28-28: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 32-32: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 35-35: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 40-40: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 43-43: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 50-50: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 53-53: Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.
(SC2010)
[warning] 55-55: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 70-70: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 79-79: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
Additional comments not posted (2)
devops/linux/docker/backup_restore/dgraph-backup-api.py (2)
18-21: Good use of directory checking and creation.The function
create_folder_if_not_existscorrectly checks if a directory exists before attempting to create it. This is a good practice to avoid unnecessary errors.
68-68: Verify the main execution flow.The script ends with a call to
backup_dgraph_data(). Ensure this is intended and that there are no conditions under which this should not be called directly.Verification successful
backup_dgraph_data()is called directly without any conditions.The script does not contain any conditional checks or configurations that would prevent the direct execution of the
backup_dgraph_data()function. The main execution flow directly calls this function at the end of the script.
backup_dgraph_data()is called directly at the end of the script without any conditions.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for conditions or configurations that might affect direct execution. # Test: Search for conditional checks or configurations related to direct execution. rg --type python -A 5 $'backup_dgraph_data()'Length of output: 77
Script:
#!/bin/bash # Description: Check for conditions or configurations that might affect direct execution. # Test: Search for conditional checks or configurations related to direct execution. rg -A 5 'backup_dgraph_data()'Length of output: 705
Script:
#!/bin/bash # Description: Check for the main execution flow in the script. # Test: Search for the main execution block in the script. rg -A 10 'if __name__ == "__main__"' devops/linux/docker/backup_restore/dgraph-backup-api.pyLength of output: 94
Script:
#!/bin/bash # Description: List all top-level statements in the script to confirm direct execution of `backup_dgraph_data()`. # Test: List all top-level statements in the script. rg '^[^ \t]' devops/linux/docker/backup_restore/dgraph-backup-api.pyLength of output: 611
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- devops/linux/docker/backup_restore/dgraph-backup-api.py (1 hunks)
- devops/linux/docker/backup_restore/dgraph-restore-api.sh (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- devops/linux/docker/backup_restore/dgraph-backup-api.py
Additional context used
Shellcheck
devops/linux/docker/backup_restore/dgraph-restore-api.sh
[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
[warning] 17-17: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 20-20: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 24-24: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 27-27: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 31-31: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 34-34: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 39-39: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 42-42: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 52-52: Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.
(SC2010)
[warning] 54-54: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 69-69: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 78-78: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
Additional comments not posted (7)
devops/linux/docker/backup_restore/dgraph-restore-api.sh (7)
6-6: Add error handling forcd.Ensure the script exits if the
cdcommand fails to prevent unintended script execution in the wrong directory.Tools
Shellcheck
[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
17-20: Add error handling forpushdandpopdinstop_services.Ensure the script exits if these commands fail. This prevents the script from continuing in an incorrect state.
Tools
Shellcheck
[warning] 17-17: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 20-20: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
24-27: Add error handling forpushdandpopdinstart_backup_restore_service.Incorporate error handling to ensure the script exits if these commands fail.
Tools
Shellcheck
[warning] 24-24: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 27-27: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
31-34: Add error handling forpushdandpopdinstart_services.It's crucial to add error handling here to prevent script execution in an incorrect directory if navigation fails.
Tools
Shellcheck
[warning] 31-31: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 34-34: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
39-42: Add error handling forpushdandpopdinstop_backup_restore_service.Adding error handling here ensures that the script exits if navigation commands fail, maintaining the script's integrity.
Tools
Shellcheck
[warning] 39-39: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 42-42: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
52-54: Replacels | grepwith a safer pattern.Use a more robust and safe pattern to list directories to avoid issues with non-standard filenames.
Tools
Shellcheck
[warning] 52-52: Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.
(SC2010)
[warning] 54-54: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
69-78: Add error handling forpushdandpopdinrestore_data.Ensure robust error handling by exiting the script if these directory navigation commands fail.
[duplicate_lcomment]Tools
Shellcheck
[warning] 69-69: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 78-78: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
…n one step, including process enhancements
There was a problem hiding this comment.
Actionable comments posted: 3
Outside diff range, codebase verification and nitpick comments (2)
devops/linux/docker/backup_restore/postgres-restore.sh (2)
Line range hint
9-29: Review of database restoration logic.The script iterates over a list of databases and checks if each exists before attempting to restore them. This is a good practice as it ensures that the restoration process only happens for existing databases. However, consider adding more robust error handling around the database operations to ensure that any failures in the commands are caught and handled appropriately.
17a18,20 > if ! PGPASSWORD="$PGPASSWORD" psql -U "$PGUSER" -h "$PGHOST" -p $PGPORT -lqt | cut -d \| -f 1 | grep -qw "$db"; then > echo "Error checking database existence: $db" > exit 1 > fi
Line range hint
20-29: Enhance error handling in the restoration loop.The script attempts to restore databases from SQL files without checking if the pg_restore command succeeds. Adding error handling here would improve the robustness of the script.
25a26,28 > if ! PGPASSWORD="$PGPASSWORD" pg_restore -U "$PGUSER" -h "$PGHOST" -d "$DB_NAME" -F c --clean --if-exists "$backup_file"; then > echo "Failed to restore database: $DB_NAME" > exit 1 > fi
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (4)
- devops/linux/docker/backup_restore/dgraph-restore-api.sh (1 hunks)
- devops/linux/docker/backup_restore/postgres-restore.sh (1 hunks)
- devops/linux/docker/backup_restore/restore-dgraph-postgres.sh (1 hunks)
- devops/linux/docker/deployment/deploy-local.sh (5 hunks)
Additional context used
Shellcheck
devops/linux/docker/backup_restore/dgraph-restore-api.sh
[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
[warning] 18-18: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 21-21: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 26-26: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 29-29: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 34-34: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 43-43: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
devops/linux/docker/backup_restore/restore-dgraph-postgres.sh
[warning] 5-5: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
[warning] 10-10: backup_restore_dir appears unused. Verify use (or export if used externally).
(SC2034)
[warning] 19-19: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 22-22: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
devops/linux/docker/deployment/deploy-local.sh
[warning] 145-145: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 153-153: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 157-157: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
Additional comments not posted (8)
devops/linux/docker/backup_restore/postgres-restore.sh (1)
3-3: Command-line argument handling implemented.The script now correctly sets the
BACKUP_FOLDER_NAMEvariable using the first command-line argument. This change enhances automation by removing the need for interactive input, aligning with best practices for script usability in automated environments.devops/linux/docker/backup_restore/dgraph-restore-api.sh (4)
6-6: Add error handling forcd.Ensure the script exits if the
cdcommand fails to prevent the script from continuing in an incorrect directory.Tools
Shellcheck
[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
18-21: Add error handling forpushdandpopd.Ensure the script exits if
pushdorpopdfails to prevent the script from continuing in an incorrect directory.Tools
Shellcheck
[warning] 18-18: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 21-21: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
26-29: Add error handling forpushdandpopd.Ensure the script exits if
pushdorpopdfails to prevent the script from continuing in an incorrect directory.Tools
Shellcheck
[warning] 26-26: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 29-29: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
34-43: Add error handling forpushdandpopd.Ensure the script exits if
pushdorpopdfails to prevent the script from continuing in an incorrect directory.Tools
Shellcheck
[warning] 34-34: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 43-43: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
devops/linux/docker/backup_restore/restore-dgraph-postgres.sh (2)
60-69: Add error handling forpushdandpopd.Ensure the script exits if
pushdorpopdfails to prevent the script from continuing in an incorrect directory.- pushd "$JEMPI_DOCKER_HOME/backup_restore" || exit + pushd "$JEMPI_DOCKER_HOME/backup_restore" || exit - popd || exit + popd || exitLikely invalid or redundant comment.
35-41: Add error handling forpushdandpopd.Ensure the script exits if
pushdorpopdfails to prevent the script from continuing in an incorrect directory.- pushd ${DGRAPH_BACKUP_DIRECTORY} || exit + pushd ${DGRAPH_BACKUP_DIRECTORY} || exit - popd || exit + popd || exitLikely invalid or redundant comment.
devops/linux/docker/deployment/deploy-local.sh (1)
143-157: Ensure robustness in backup operations and handle potential failures.Add error handling for the
pushdandpopdcommands and the backup scripts.Tools
Shellcheck
[warning] 145-145: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 153-153: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 157-157: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
There was a problem hiding this comment.
Actionable comments posted: 10
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (5)
- devops/linux/docker/backup_restore/dgraph-backup-api.sh (1 hunks)
- devops/linux/docker/backup_restore/dgraph-restore-api.sh (1 hunks)
- devops/linux/docker/backup_restore/postgres-backup.sh (4 hunks)
- devops/linux/docker/backup_restore/postgres-restore.sh (1 hunks)
- devops/linux/docker/backup_restore/restore-dgraph-postgres.sh (1 hunks)
Files skipped from review as they are similar to previous changes (2)
- devops/linux/docker/backup_restore/dgraph-backup-api.sh
- devops/linux/docker/backup_restore/postgres-restore.sh
Additional context used
Shellcheck
devops/linux/docker/backup_restore/dgraph-restore-api.sh
[warning] 11-11: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
[warning] 23-23: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 26-26: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 31-31: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 34-34: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
[warning] 39-39: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
[warning] 48-48: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
devops/linux/docker/backup_restore/postgres-backup.sh
[warning] 16-16: DB_NAME appears unused. Verify use (or export if used externally).
(SC2034)
[warning] 42-42: backup_file appears unused. Verify use (or export if used externally).
(SC2034)
devops/linux/docker/backup_restore/restore-dgraph-postgres.sh
[warning] 10-10: backup_restore_dir appears unused. Verify use (or export if used externally).
(SC2034)
Additional comments not posted (16)
devops/linux/docker/backup_restore/dgraph-restore-api.sh (11)
1-1: Shebang line is correct.The shebang line correctly specifies
bashas the interpreter.
2-2: Configuration file sourced correctly.The script sources
../conf.envcorrectly relative to its location.
4-8: Proper error handling for missing backup folder name.The script checks if the backup folder name is provided and exits with an error message if not. This is a good practice to ensure the script does not proceed with incomplete parameters.
9-9: Backup folder name correctly assigned.The backup folder name is correctly assigned from the first script argument.
10-13: Correct calculation of script directory and setting environment variables.The script calculates its directory and sets the
JEMPI_DOCKER_HOMEenvironment variable correctly. This is crucial for relative path operations within the script.Tools
Shellcheck
[warning] 11-11: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
14-16: Directory variables are set correctly.The script sets directory paths using environment variables, which is a good practice for flexibility and maintainability.
51-59: Start and stop backup restore service correctly.The script correctly starts and stops the backup restore service by sourcing the appropriate scripts. This is essential for managing the service lifecycle during the backup process.
53-57: Loop through backup files correctly.The script loops through backup files and calls the
restore_datafunction for each file. This ensures that all backup files are processed.
59-59: Stop backup restore service correctly.The script correctly stops the backup restore service, ensuring that services are properly shut down after the backup process.
22-23: Add error handling forpopd.The script should handle errors for
popdto ensure it exits if the directory change fails.- popd + popd || exitLikely invalid or redundant comment.
Tools
Shellcheck
[warning] 23-23: Use 'pushd ... || exit' or 'pushd ... || return' in case pushd fails.
(SC2164)
25-26: Add error handling forpushd.The script should handle errors for
pushdto ensure it exits if the directory change fails.- pushd "$reboot_dir" + pushd "$reboot_dir" || exitLikely invalid or redundant comment.
Tools
Shellcheck
[warning] 26-26: Use 'popd ... || exit' or 'popd ... || return' in case popd fails.
(SC2164)
devops/linux/docker/backup_restore/postgres-backup.sh (3)
4-8: Add error handling for missing backup folder name argument.The script exits if no backup folder name is provided, which is good practice to avoid unintended behavior. However, the error message could be more user-friendly by suggesting the use of a default name based on the current date and time.
Line range hint
20-29: Ensure directory creation before proceeding with backups.The script correctly checks and creates the necessary directories before proceeding. This is a good practice to avoid errors during the backup process.
Tools
Shellcheck
[warning] 16-16: DB_NAME appears unused. Verify use (or export if used externally).
(SC2034)
56-56: Secure file transfer usingscp.The script uses
scpto transfer backup files to a remote server. Ensure that the remote server is secured and that appropriate SSH keys are configured for authentication to prevent security risks.devops/linux/docker/backup_restore/restore-dgraph-postgres.sh (2)
5-5: Add error handling forcd.The script exits if the
cdcommand fails, which is crucial to prevent the script from continuing in an incorrect directory. This is a good practice and aligns with previous comments.
19-22: Add error handling forpushdandpopd.The script correctly handles potential failures of
pushdandpopdcommands by exiting if they fail. This ensures that the script does not continue in an incorrect directory.
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- devops/linux/docker/backup_restore/dgraph-restore-api.sh (1 hunks)
- devops/linux/docker/backup_restore/restore-dgraph-postgres.sh (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- devops/linux/docker/backup_restore/dgraph-restore-api.sh
Additional context used
Shellcheck
devops/linux/docker/backup_restore/restore-dgraph-postgres.sh
[warning] 10-10: backup_restore_dir appears unused. Verify use (or export if used externally).
(SC2034)
Additional comments not posted (7)
devops/linux/docker/backup_restore/restore-dgraph-postgres.sh (7)
1-6: Initial setup and configuration look good.The script correctly handles potential failures for
cd, ensuring the script exits if the directory change fails, which is crucial for maintaining the correct execution context.
18-22: Functionstop_servicesis implemented correctly.The use of
pushdandpopdwith error handling ensures that the script exits if directory changes fail, which is essential for maintaining the correct execution context.
24-28: Functionstart_servicesis implemented correctly.The consistency in error handling with
pushdandpopdacross service control functions is commendable, ensuring robustness in script execution.
31-52: User interaction for backup folder selection is well-implemented.The enhancements to list available backups before asking for user input improve the user experience and help prevent errors in folder selection.
57-72: Backup restoration process is robust and well-handled.The script ensures a clean environment by stopping services before restoration and handles potential directory change failures effectively.
73-76: Handling of invalid backup directory input is effective.The use of colored error messages enhances user feedback and guides them towards providing valid input.
10-10: Verify the usage ofbackup_restore_dir.The variable appears unused in this script. If it's intended for use in external scripts, consider exporting it; otherwise, it might be cleaner to remove it.
Tools
Shellcheck
[warning] 10-10: backup_restore_dir appears unused. Verify use (or export if used externally).
(SC2034)
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- devops/linux/docker/backup_restore/dgraph-restore-api.py (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- devops/linux/docker/backup_restore/dgraph-restore-api.py
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (4)
- JeMPI_Apps/JeMPI_AsyncReceiver/src/main/java/org/jembi/jempi/async_receiver/Main.java (1 hunks)
- JeMPI_Apps/JeMPI_LibShared/src/main/java/org/jembi/jempi/shared/config/linker/Programs.java (4 hunks)
- JeMPI_Apps/JeMPI_LibShared/src/main/java/org/jembi/jempi/shared/models/AuxGoldenRecordData.java (2 hunks)
- devops/linux/docker/data-config/config-reference-link-dp.json (2 hunks)
Additional comments not posted (4)
JeMPI_Apps/JeMPI_LibShared/src/main/java/org/jembi/jempi/shared/models/AuxGoldenRecordData.java (1)
Line range hint
14-40: Simplified Constructor Logic inAuxGoldenRecordDataThe constructor now uses a streamlined ternary operation. This change should improve readability and potentially performance if the previous operation was more complex or computationally expensive.
However, ensure that this change does not alter the expected behavior, particularly in edge cases where the previous logic might have handled specific conditions differently.
devops/linux/docker/data-config/config-reference-link-dp.json (1)
Line range hint
1-100: Removal of Fields in JSON ConfigurationThe fields "aux_iid" and "aux_gid" have been removed from the configuration. This change simplifies the data model and might be part of a larger effort to streamline data handling.
However, verify that the removal of these fields does not break existing functionalities or integrations that might depend on these fields.
JeMPI_Apps/JeMPI_AsyncReceiver/src/main/java/org/jembi/jempi/async_receiver/Main.java (1)
79-79: Simplified Data Mapping inauxInteractionDataThe removal of a conditional check simplifies the data mapping process, potentially improving performance by reducing conditional logic.
However, ensure that this simplification does not omit necessary validations or error handling that might have been performed by the removed conditional logic.
JeMPI_Apps/JeMPI_LibShared/src/main/java/org/jembi/jempi/shared/config/linker/Programs.java (1)
Line range hint
1-1000: Removal ofGoldenRecord.aux_gidfrom Query LogicThe removal of
GoldenRecord.aux_gidfrom several query construction methods could indicate a change in the data model or a move towards simplifying the data structure.However, verify that this change does not impact the functionality or performance of the queries, especially in complex data retrieval scenarios.
Verification successful
Removal of
GoldenRecord.aux_gidfrom Query LogicThe removal of
GoldenRecord.aux_gidfrom the query logic inPrograms.javadoes not impact the functionality or performance of the queries, as there are no references toaux_gidin the Java codebase.
- The only reference to
aux_gidis in a configuration file (devops/linux/docker/data-config/config-reference-link-dp-minimal.json).Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check the impact of removing `GoldenRecord.aux_gid` on query performance and correctness. # Test: Run performance benchmarks and correctness tests on the modified queries. # Expect: Queries should perform at least as well as before and return correct results. echo "Run performance and correctness tests for modified queries"Length of output: 126
Script:
#!/bin/bash # Search for references to `aux_gid` in the codebase to understand its usage and potential impact. rg --context 5 'aux_gid'Length of output: 1024
Script:
#!/bin/bash # Search for references to `aux_gid` in Java files to understand its usage and potential impact. rg --glob '*.java' --context 5 'aux_gid'Length of output: 40
Backup Functionality using API
JeMPI_BackupRestoreAPI
A separate app for backup and restore operations.
Script Steps:
Backup Steps:
Dgraph
Backup:
JeMPI/devops/linux/docker/backup_restore/dgraph-backup-api.sh./dgraph-backup-api.sh./local-deployment.shJeMPI/devops/linux/docker/docker_data/data/backups/dgraphPostgres
Backup:
JeMPI/devops/linux/docker/docker_data/data/backups/postgresTask: CU-86bymxdnb_Create-backup-api
Summary by CodeRabbit
New Features
Improvements
Bug Fixes
Chores