-
Notifications
You must be signed in to change notification settings - Fork 2
feat(neo4j): Neo4j Submodule Integration Enhancement #898
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -640,60 +640,6 @@ supabase-clean: ## Stop + remove Supabase containers and volumes | |
| @$(DC) --profile supabase-local down -v --remove-orphans >/dev/null 2>&1 || true | ||
| @echo "โ Supabase stack cleaned" | ||
|
|
||
| .PHONY: neo4j-up neo4j-down neo4j-restart neo4j-logs neo4j-migrate neo4j-seed neo4j-bootstrap neo4j-status | ||
|
|
||
| neo4j-up: ## Start Neo4j submodule stack | ||
| @echo "โ Starting Neo4j from submodule..." | ||
| @$(MAKE) --no-print-directory -C pmoves/integrations/neo4j up | ||
|
|
||
| neo4j-down: ## Stop Neo4j submodule stack | ||
| @echo "โ Stopping Neo4j from submodule..." | ||
| @$(MAKE) --no-print-directory -C pmoves/integrations/neo4j down | ||
|
|
||
| neo4j-restart: ## Restart Neo4j submodule stack | ||
| @echo "โ Restarting Neo4j from submodule..." | ||
| @$(MAKE) --no-print-directory -C pmoves/integrations/neo4j restart | ||
|
|
||
| neo4j-logs: ## View Neo4j logs | ||
| @$(MAKE) --no-print-directory -C pmoves/integrations/neo4j logs | ||
|
|
||
| neo4j-migrate: ## Run Neo4j migration (usage: make neo4j-migrate VERSION=003) | ||
| @if [ -z "$(VERSION)" ]; then \ | ||
| echo "Usage: make neo4j-migrate VERSION=003"; \ | ||
| echo ""; \ | ||
| echo "Available migrations:"; \ | ||
| ls -1 pmoves/integrations/neo4j/db/migrations/*.cypher 2>/dev/null | while read -r f; do \ | ||
| echo " โข $$(basename "$f")"; \ | ||
| done; \ | ||
| exit 1; \ | ||
| fi | ||
| @$(MAKE) --no-print-directory -C pmoves/integrations/neo4j migrate VERSION=$(VERSION) | ||
|
|
||
| neo4j-seed: ## Load Neo4j seed data (usage: make neo4j-seed SEED=001_person_aliases.csv) | ||
| @if [ -z "$(SEED)" ]; then \ | ||
| echo "Usage: make neo4j-seed SEED=001_person_aliases.csv"; \ | ||
| echo ""; \ | ||
| echo "Available seeds:"; \ | ||
| ls -1 pmoves/integrations/neo4j/db/seeds/*.csv 2>/dev/null | while read -r f; do \ | ||
| echo " โข $$(basename "$f")"; \ | ||
| done; \ | ||
| exit 1; \ | ||
| fi | ||
| @$(MAKE) --no-print-directory -C pmoves/integrations/neo4j seed SEED=$(SEED) | ||
|
|
||
| neo4j-bootstrap: ## Initialize Neo4j (run migrations and seeds) | ||
| @echo "โ Bootstrapping Neo4j from submodule..." | ||
| @$(MAKE) --no-print-directory -C pmoves/integrations/neo4j bootstrap | ||
|
|
||
| neo4j-status: ## Check Neo4j status from submodule | ||
| @$(MAKE) --no-print-directory -C pmoves/integrations/neo4j status | ||
|
|
||
| # Legacy compatibility (delegates to submodule) | ||
| .PHONY: neo4j-bootstrap-legacy | ||
| neo4j-bootstrap-legacy: ## Legacy bootstrap script (deprecated: use neo4j-bootstrap) | ||
| @echo "โ ๏ธ This target is deprecated. Use: make neo4j-bootstrap" | ||
| @$(MAKE) --no-print-directory neo4j-bootstrap | ||
|
|
||
| .PHONY: seed-data | ||
| seed-data: ## Seed Qdrant/Meilisearch with demo corpus via Hi-RAG v2 | ||
| @echo "๐ Seeding Qdrant/Meili demo corpus..." | ||
|
|
@@ -1917,10 +1863,6 @@ neo4j-reset: ## DANGEROUS: wipe Neo4j volume and recreate with current NEO4J_AUT | |
| "/var/lib/neo4j/bin/cypher-shell -u \"$$user\" -p \"$$pass\" \"RETURN 1\""; \ | ||
| ' | ||
|
|
||
| neo4j-status: ## Show Neo4j logs and health line | ||
| @$(DC) ps neo4j || true | ||
| @$(DC) logs --tail 60 neo4j || true | ||
|
|
||
| brand-verify: ## Verify key branded endpoints respond | ||
| @echo "Presign:" && curl -fsS http://localhost:8088/healthz && echo | ||
| @echo "Supabase REST:" && curl -fsS -o /dev/null -w '%{http_code}\n' http://host.docker.internal:65421/rest/v1 || true | ||
|
|
@@ -2820,49 +2762,49 @@ github-app-test: ## Test GitHub App token minting (requires services running) | |
|
|
||
| neo4j-up: ## Start Neo4j submodule stack | ||
| @echo "โ Starting Neo4j from submodule..." | ||
| @$(MAKE) --no-print-directory -C PMOVES-Neo4j up | ||
| @$(MAKE) --no-print-directory -C ../PMOVES-Neo4j up | ||
|
|
||
| neo4j-down: ## Stop Neo4j submodule stack | ||
| @echo "โ Stopping Neo4j from submodule..." | ||
| @$(MAKE) --no-print-directory -C PMOVES-Neo4j down | ||
| @$(MAKE) --no-print-directory -C ../PMOVES-Neo4j down | ||
|
|
||
| neo4j-restart: ## Restart Neo4j submodule stack | ||
| @echo "โ Restarting Neo4j from submodule..." | ||
| @$(MAKE) --no-print-directory -C PMOVES-Neo4j restart | ||
| @$(MAKE) --no-print-directory -C ../PMOVES-Neo4j restart | ||
|
|
||
| neo4j-logs: ## View Neo4j logs | ||
| @$(MAKE) --no-print-directory -C PMOVES-Neo4j logs | ||
| @$(MAKE) --no-print-directory -C ../PMOVES-Neo4j logs | ||
|
|
||
| neo4j-migrate: ## Run Neo4j migration (usage: make neo4j-migrate VERSION=003) | ||
| @if [ -z "$(VERSION)" ]; then \ | ||
| echo "Usage: make neo4j-migrate VERSION=003"; \ | ||
| echo ""; \ | ||
| echo "Available migrations:"; \ | ||
| ls -1 PMOVES-Neo4j/db/migrations/*.cypher 2>/dev/null | while read -r f; do \ | ||
| ls -1 ../PMOVES-Neo4j/db/migrations/*.cypher 2>/dev/null | while read -r f; do \ | ||
| echo " โข $$(basename "$$f")"; \ | ||
| done; \ | ||
| exit 1; \ | ||
| fi | ||
| @$(MAKE) --no-print-directory -C PMOVES-Neo4j migrate VERSION=$(VERSION) | ||
| @$(MAKE) --no-print-directory -C ../PMOVES-Neo4j migrate VERSION=$(VERSION) | ||
|
|
||
| neo4j-seed: ## Load Neo4j seed data (usage: make neo4j-seed SEED=001_person_aliases.csv) | ||
| @if [ -z "$(SEED)" ]; then \ | ||
| echo "Usage: make neo4j-seed SEED=001_person_aliases.csv"; \ | ||
| echo ""; \ | ||
| echo "Available seeds:"; \ | ||
| ls -1 PMOVES-Neo4j/db/seeds/*.csv 2>/dev/null | while read -r f; do \ | ||
| ls -1 ../PMOVES-Neo4j/db/seeds/*.csv 2>/dev/null | while read -r f; do \ | ||
| echo " โข $$(basename "$$f")"; \ | ||
| done; \ | ||
| exit 1; \ | ||
| fi | ||
| @$(MAKE) --no-print-directory -C PMOVES-Neo4j seed SEED=$(SEED) | ||
| @$(MAKE) --no-print-directory -C ../PMOVES-Neo4j seed SEED=$(SEED) | ||
|
|
||
| neo4j-bootstrap: ## Initialize Neo4j (run migrations and seeds) | ||
| @echo "โ Bootstrapping Neo4j from submodule..." | ||
| @$(MAKE) --no-print-directory -C PMOVES-Neo4j bootstrap | ||
| @$(MAKE) --no-print-directory -C ../PMOVES-Neo4j bootstrap | ||
|
|
||
| neo4j-status: ## Check Neo4j status from submodule | ||
| @$(MAKE) --no-print-directory -C PMOVES-Neo4j status | ||
| @$(MAKE) --no-print-directory -C ../PMOVES-Neo4j status | ||
|
|
||
| # Legacy compatibility | ||
| .PHONY: neo4j-bootstrap-legacy | ||
|
|
@@ -2881,7 +2823,8 @@ neo4j-local-up: ## Start Neo4j with profile (make neo4j-local-up) | |
|
|
||
| neo4j-local-down: ## Stop Neo4j with profile (make neo4j-local-down) | ||
| @echo "โ Stopping Neo4j with neo4j-local profile..." | ||
| @$(DC) --profile neo4j-local down -v --remove-orphans | ||
| @$(DC) --profile neo4j-local stop neo4j >/dev/null 2>&1 || true | ||
| @$(DC) --profile neo4j-local rm -f neo4j >/dev/null 2>&1 || true | ||
| @echo "โ Neo4j stopped" | ||
|
|
||
| neo4j-local-status: ## Check Neo4j status (make neo4j-local-status) | ||
|
|
@@ -2890,10 +2833,10 @@ neo4j-local-status: ## Check Neo4j status (make neo4j-local-status) | |
|
|
||
| .PHONY: neo4j-backup neo4j-restore | ||
|
|
||
| neo4j-backup: ## Backup Neo4j database to pmoves/backups/ with timestamp | ||
| neo4j-backup: ## Backup Neo4j database to backups/ with timestamp | ||
| @echo "โ Creating Neo4j backup..." | ||
| @timestamp=$$(date +%Y%m%d_%H%M%S); \ | ||
| backup_dir="pmoves/backups"; \ | ||
| backup_dir="backups"; \ | ||
| mkdir -p "$$backup_dir"; \ | ||
| container_name=$$($(DC) ps -q neo4j 2>/dev/null | head -1); \ | ||
| if [ -z "$$container_name" ]; then \ | ||
|
|
@@ -2903,35 +2846,36 @@ neo4j-backup: ## Backup Neo4j database to pmoves/backups/ with timestamp | |
| container_name=$$($(DC) ps -q neo4j 2>/dev/null | head -1); \ | ||
| fi; \ | ||
| if [ -n "$$container_name" ]; then \ | ||
| docker exec $$container_name neo4j-admin database dump \ | ||
| if docker exec $$container_name neo4j-admin database dump \ | ||
| --to-path=/backups \ | ||
| --overwrite-destination=true \ | ||
| --username=neo4j \ | ||
| --password="$${NEO4J_PASSWORD:-changeme}" 2>/dev/null || \ | ||
| docker exec $$container_name cypher-shell -u neo4j -p "$${NEO4J_PASSWORD:-changeme}" \ | ||
| "CALL apoc.export.cypher.all(null, 'file:///backups/neo4j_$${timestamp}.cypher', {})"; \ | ||
| docker cp $$container_name:/backups/neo4j.dump "$$backup_dir/neo4j_$${timestamp}.dump" 2>/dev/null || \ | ||
| echo "โ ๏ธ Backup file not found at /backups/neo4j.dump"; \ | ||
| if [ -f "$$backup_dir/neo4j_$${timestamp}.dump" ]; then \ | ||
| echo "โ Backup created: $$backup_dir/neo4j_$${timestamp}.dump"; \ | ||
| ls -lh "$$backup_dir/neo4j_$${timestamp}.dump"; \ | ||
| --password="$${NEO4J_PASSWORD:-changeme}" 2>/dev/null; then \ | ||
| docker cp $$container_name:/backups/neo4j.dump "$$backup_dir/neo4j_$${timestamp}.dump" 2>/dev/null && \ | ||
| echo "โ Backup created: $$backup_dir/neo4j_$${timestamp}.dump" && \ | ||
| ls -lh "$$backup_dir/neo4j_$${timestamp}.dump"; \ | ||
| elif docker exec $$container_name cypher-shell -u neo4j -p "$${NEO4J_PASSWORD:-changeme}" \ | ||
| "CALL apoc.export.cypher.all(null, 'file:///backups/neo4j_$${timestamp}.cypher', {})"; then \ | ||
| docker cp $$container_name:/backups/neo4j_$${timestamp}.cypher "$$backup_dir/neo4j_$${timestamp}.cypher" 2>/dev/null && \ | ||
|
Comment on lines
+2849
to
+2859
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ๐งฉ Analysis chain๐ Script executed: # First, let's look at the Makefile at the specified lines
head -2787 pmoves/Makefile | tail -20Repository: POWERFULMOVES/PMOVES.AI Length of output: 1187 ๐ Script executed: # Also check the second location mentioned (2824-2829)
head -2829 pmoves/Makefile | tail -10Repository: POWERFULMOVES/PMOVES.AI Length of output: 484 ๐ Web query:
๐ก Result:
Syntax (Neo4j Ops Manual): neo4j-admin database dump [-h] [--expand-commands] [--verbose]
[--overwrite-destination[=true|false]]
[--additional-config=<file>]
[--to-path=<path> | --to-stdout]
<database>[1] Example: neo4j-admin database dump neo4j --to-path=/var/backups/neo4j --overwrite-destination=true[1] If you need an authenticated, online-safe backup, thatโs a different command flow (e.g., Sources: Neo4j Operations Manual (offline dump syntax/options) [1], Neo4j ๐ Web query:
๐ก Result:
What can require โauthenticationโ:
Sources: Fix neo4j-admin backup/restore syntax โ missing database argument and invalid authentication flags. The
Current code# Lines ~2777-2787 (backup)
docker exec $$container_name neo4j-admin database dump \
--to-path=/backups \
--overwrite-destination=true \
--username=neo4j \
--password="$${NEO4J_PASSWORD:-changeme}"
# Lines ~2824-2829 (restore)
docker exec $$container_name neo4j-admin database load \
--from=/backups/neo4j.dump \
--overwrite-destination=true \
--username=neo4j \
--password="$${NEO4J_PASSWORD:-changeme}"These commands will fail immediately with syntax errors. Correct syntax requires removing the auth flags and adding the database name to dump, and using ๐ค Prompt for AI Agents |
||
| echo "โ Backup created: $$backup_dir/neo4j_$${timestamp}.cypher" && \ | ||
| ls -lh "$$backup_dir/neo4j_$${timestamp}.cypher"; \ | ||
| else \ | ||
| echo "โ Backup failed - no file created"; \ | ||
| echo "โ Backup failed - neither dump nor cypher export succeeded"; \ | ||
| exit 1; \ | ||
| fi; \ | ||
| else \ | ||
| echo "โ Neo4j container not available"; \ | ||
| exit 1; \ | ||
| fi | ||
|
|
||
| neo4j-restore: ## Restore Neo4j from backup (usage: make neo4j-restore BACKUP=pmoves/backups/neo4j_YYYYMMDD_HHMMSS.dump) | ||
| neo4j-restore: ## Restore Neo4j from backup (usage: make neo4j-restore BACKUP=backups/neo4j_YYYYMMDD_HHMMSS.dump) | ||
| @if [ -z "$(BACKUP)" ]; then \ | ||
| echo "โ Error: BACKUP parameter required"; \ | ||
| echo ""; \ | ||
| echo "Usage: make neo4j-restore BACKUP=pmoves/backups/neo4j_20260313_120000.dump"; \ | ||
| echo "Usage: make neo4j-restore BACKUP=backups/neo4j_20260313_120000.dump"; \ | ||
| echo ""; \ | ||
| echo "Available backups:"; \ | ||
| ls -lh pmoves/backups/neo4j_*.dump 2>/dev/null || echo " No backups found in pmoves/backups/"; \ | ||
| ls -lh backups/neo4j_*.dump 2>/dev/null || echo " No backups found in backups/"; \ | ||
| exit 1; \ | ||
| fi | ||
| @if [ ! -f "$(BACKUP)" ]; then \ | ||
|
|
@@ -2943,7 +2887,7 @@ neo4j-restore: ## Restore Neo4j from backup (usage: make neo4j-restore BACKUP=pm | |
| @echo "Press Ctrl+C to cancel, or wait 5 seconds to continue..." | ||
| @sleep 5 | ||
| @$(MAKE) --no-print-directory neo4j-local-down | ||
| @docker volume rm pmoves_neo4jdata 2>/dev/null || true | ||
| @docker volume rm pmoves_neo4j-data 2>/dev/null || true | ||
| @echo "โ Starting fresh Neo4j..." | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| @$(MAKE) --no-print-directory neo4j-local-up | ||
| @sleep 15 | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,7 +2,7 @@ | |
| # Neo4j Database Backup Script | ||
| # ============================================================================= | ||
| # Automated backup script for Neo4j graph database. | ||
| # Creates timestamped dumps to pmoves/backups/ with rotation. | ||
| # Creates timestamped dumps to backups/ with rotation. | ||
| # | ||
| # Usage: ./pmoves/scripts/backup-neo4j.sh [retention_days] | ||
| # Default retention: 7 days | ||
|
|
@@ -13,12 +13,22 @@ | |
|
|
||
| set -euo pipefail | ||
|
|
||
| # Script directory (for loading env.shared) | ||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| PROJECT_ROOT="$(dirname "$SCRIPT_DIR")" | ||
|
|
||
| # Load credentials if env.shared exists | ||
| if [ -f "$PROJECT_ROOT/env.shared" ]; then | ||
| source "$PROJECT_ROOT/env.shared" | ||
| fi | ||
|
|
||
| # Configuration | ||
| BACKUP_DIR="pmoves/backups" | ||
| BACKUP_DIR="backups" | ||
| CONTAINER_NAME="pmoves-neo4j-1" | ||
| RETENTION_DAYS=${1:-7} | ||
| TIMESTAMP=$(date +%Y%m%d_%H%M%S) | ||
| BACKUP_FILE="neo4j_${TIMESTAMP}.dump" | ||
| CONTAINER_BACKUP_DIR="/data/backups" | ||
|
|
||
| # Colors for output | ||
| RED='\033[0;31m' | ||
|
|
@@ -55,21 +65,25 @@ if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then | |
| exit 1 | ||
| fi | ||
|
|
||
| # Get Neo4j password from environment | ||
| # Get Neo4j password from environment (with fallback) | ||
| NEO4J_PASSWORD=${NEO4J_PASSWORD:-changeme} | ||
|
|
||
| # Create backup directory inside container | ||
| log_info "Creating backup directory inside container..." | ||
| docker exec "$CONTAINER_NAME" mkdir -p "$CONTAINER_BACKUP_DIR" | ||
|
|
||
| # Create backup using neo4j-admin | ||
| log_info "Creating Neo4j backup: ${BACKUP_FILE}" | ||
|
|
||
| if docker exec "$CONTAINER_NAME" neo4j-admin database dump \ | ||
| --to-path=/backups \ | ||
| --to-path="$CONTAINER_BACKUP_DIR" \ | ||
| --overwrite-destination=true \ | ||
| --username=neo4j \ | ||
| --password="$NEO4J_PASSWORD" 2>/dev/null; then | ||
|
Comment on lines
+78
to
+82
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ๐งฉ Analysis chain๐ Script executed: # First, check if the file exists and read the full script context
find . -name "backup-neo4j.sh" -type fRepository: POWERFULMOVES/PMOVES.AI Length of output: 99 ๐ Script executed: # Also check the Neo4j version or any version pinning in the codebase
fd -e "Dockerfile\|docker-compose\|env\|requirements" | head -20Repository: POWERFULMOVES/PMOVES.AI Length of output: 49 ๐ Script executed: # Search for neo4j references in the codebase
rg "neo4j" --type sh -A 2 -B 2Repository: POWERFULMOVES/PMOVES.AI Length of output: 12758 ๐ Script executed: cat -n pmoves/scripts/backup-neo4j.shRepository: POWERFULMOVES/PMOVES.AI Length of output: 4063 ๐ Web query:
๐ก Result:
|
||
|
|
||
| # Copy backup from container | ||
| docker cp "$CONTAINER_NAME:/backups/neo4j.dump" "$BACKUP_DIR/$BACKUP_FILE" | ||
| docker cp "$CONTAINER_NAME:$CONTAINER_BACKUP_DIR/neo4j.dump" "$BACKUP_DIR/$BACKUP_FILE" | ||
|
|
||
| # Verify backup file exists and is not empty | ||
| if [ -f "$BACKUP_DIR/$BACKUP_FILE" ] && [ -s "$BACKUP_DIR/$BACKUP_FILE" ]; then | ||
| BACKUP_SIZE=$(du -h "$BACKUP_DIR/$BACKUP_FILE" | cut -f1) | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.