Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .coderabbit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@ reviews:
Verify eventId and tenantId are always set.
chat:
auto_reply: true

103 changes: 103 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
.PHONY: up down restart logs build seed test lint clean help
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Targets defined in Makefile =="
rg -n '^[a-zA-Z_-]+:.*$' Makefile

echo
echo "== .PHONY declaration =="
rg -n '^\.(PHONY):' Makefile

echo
echo "== Check whether required default target exists =="
rg -n '^all:' Makefile || true

Repository: pahuldeepp/GrainGuard-

Length of output: 1021


Add a default all target and complete the .PHONY list.

The .PHONY declaration on line 1 is incomplete, missing 12 targets that are actually defined later in the Makefile: logs-gateway, logs-bff, logs-kafka, migrate, test-go, test-react, test-load, lint-go, lint-ts, clean-cache, ps, and health. Additionally, the all target is missing entirely, which is a standard Makefile convention. These omissions can cause target/file name collisions and potential CI lint failures.

Proposed fix
-.PHONY: up down restart logs build seed test lint clean help
+.PHONY: all up down restart logs logs-gateway logs-bff logs-kafka build seed migrate test test-go test-react test-load lint lint-go lint-ts clean clean-cache ps health help
+
+all: up ## Default target
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
.PHONY: up down restart logs build seed test lint clean help
.PHONY: all up down restart logs logs-gateway logs-bff logs-kafka build seed migrate test test-go test-react test-load lint lint-go lint-ts clean clean-cache ps health help
all: up ## Default target
🧰 Tools
🪛 checkmake (0.2.2)

[warning] 1-1: Missing required phony target "all"

(minphony)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Makefile` at line 1, Update the .PHONY declaration to include the missing
targets and add a default all target: extend the existing .PHONY list to include
all, logs-gateway, logs-bff, logs-kafka, migrate, test-go, test-react,
test-load, lint-go, lint-ts, clean-cache, ps, and health; then add a top-level
all target (e.g., all: up or all: build up — choose the appropriate default
behavior for this Makefile) so running make with no args uses the standard
default; ensure the new all name doesn't collide with any file and that the
added phony names match the actual targets (logs-gateway, logs-bff, logs-kafka,
migrate, test-go, test-react, test-load, lint-go, lint-ts, clean-cache, ps,
health).


# ============================================
# GrainGuard — Developer Makefile
# ============================================

help: ## Show this help
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

# ============================================
# Docker Compose
# ============================================

up: ## Start all services
docker compose -f infra/docker/docker-compose.yml up -d

down: ## Stop all services
docker compose -f infra/docker/docker-compose.yml down

restart: ## Restart all services
docker compose -f infra/docker/docker-compose.yml restart

logs: ## Tail logs for all services
docker compose -f infra/docker/docker-compose.yml logs -f

logs-gateway: ## Tail gateway logs
docker compose -f infra/docker/docker-compose.yml logs -f gateway

logs-bff: ## Tail BFF logs
docker compose -f infra/docker/docker-compose.yml logs -f bff

logs-kafka: ## Tail Kafka logs
docker compose -f infra/docker/docker-compose.yml logs -f kafka

build: ## Build all Docker images
docker compose -f infra/docker/docker-compose.yml build

# ============================================
# Database
# ============================================

seed: ## Seed dev data (Postgres + Elasticsearch)
cd scripts/seed && bash seed-postgres.sh
cd scripts/seed && bash seed-elasticsearch.sh

migrate: ## Run database migrations
go run libs/migrate/migrate.go

# ============================================
# Testing
# ============================================

test: ## Run all tests
$(MAKE) test-go
$(MAKE) test-react

test-go: ## Run Go tests
go test ./...

test-react: ## Run React/Node tests
cd apps/dashboard && npm test -- --run
cd apps/bff && npm test -- --run

test-load: ## Run k6 load tests
k6 run scripts/load-tests/gateway-load-test.js
k6 run scripts/load-tests/bff-load-test.js

# ============================================
# Linting
# ============================================

lint: ## Lint all services
$(MAKE) lint-go
$(MAKE) lint-ts

lint-go: ## Lint Go services
golangci-lint run ./...

lint-ts: ## Lint TypeScript services
cd apps/gateway && npm run lint
cd apps/bff && npm run lint
cd apps/dashboard && npm run lint

# ============================================
# Cleanup
# ============================================

clean: ## Remove all containers and volumes
docker compose -f infra/docker/docker-compose.yml down -v --remove-orphans

clean-cache: ## Clear Redis cache
docker compose -f infra/docker/docker-compose.yml exec redis redis-cli FLUSHALL

Comment on lines +91 to +93
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# First, find and verify the Makefile exists
fd -type f -name "Makefile" | head -20

Repository: pahuldeepp/GrainGuard-

Length of output: 236


🏁 Script executed:

# Once located, read the specific lines mentioned (91-93)
cat -n Makefile | sed -n '85,100p'

Repository: pahuldeepp/GrainGuard-

Length of output: 660


🏁 Script executed:

# Check the docker-compose configuration for Redis setup
fd -type f -name "docker-compose.yml" | head -10

Repository: pahuldeepp/GrainGuard-

Length of output: 236


🏁 Script executed:

# Find docker-compose.yml with correct syntax
find . -name "docker-compose.yml" -type f

Repository: pahuldeepp/GrainGuard-

Length of output: 99


🏁 Script executed:

# Look at the docker-compose configuration
cat infra/docker/docker-compose.yml

Repository: pahuldeepp/GrainGuard-

Length of output: 14710


🏁 Script executed:

# Check if there are any references to Redis DB selection in the codebase
rg "redis|Redis|REDIS" --max-count=20 -i

Repository: pahuldeepp/GrainGuard-

Length of output: 28172


Use FLUSHDB instead of FLUSHALL in the clean-cache target.

Line 92 executes FLUSHALL, which clears all Redis databases. Multiple services (bff, gateway, read-model-builder) depend on this Redis instance for caching, rate-limiting, and idempotency locks. Use FLUSHDB to clear only the current database, limiting the scope of cache cleanup.

Fix
 clean-cache: ## Clear Redis cache
-	docker compose -f infra/docker/docker-compose.yml exec redis redis-cli FLUSHALL
+	docker compose -f infra/docker/docker-compose.yml exec redis redis-cli FLUSHDB
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
clean-cache: ## Clear Redis cache
docker compose -f infra/docker/docker-compose.yml exec redis redis-cli FLUSHALL
clean-cache: ## Clear Redis cache
docker compose -f infra/docker/docker-compose.yml exec redis redis-cli FLUSHDB
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Makefile` around lines 91 - 93, The Makefile target clean-cache currently
runs redis-cli FLUSHALL which wipes all Redis databases; update the target
(clean-cache) to use redis-cli FLUSHDB so only the current DB is cleared, i.e.,
replace the FLUSHALL argument with FLUSHDB in the docker compose exec command
that invokes redis-cli.

# ============================================
# Status
# ============================================

ps: ## Show running services
docker compose -f infra/docker/docker-compose.yml ps

health: ## Check health of all services
curl -s http://localhost:8086/health | jq
curl -s http://localhost:4000/health | jq
Comment on lines +101 to +103
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

# First, locate and examine the Makefile
find . -name "Makefile" -type f | head -5

Repository: pahuldeepp/GrainGuard-

Length of output: 76


🏁 Script executed:

# Once we have the Makefile, check the health target
sed -n '95,110p' Makefile

Repository: pahuldeepp/GrainGuard-

Length of output: 332


🌐 Web query:

curl -s flag behavior HTTP error codes 4xx 5xx exit status

💡 Result:

  • -s / --silent only affects output: it suppresses the progress meter and curl’s error messages; it does not change how HTTP status codes map to curl’s exit status. Use -S / --show-error with -s if you want “silent except show errors”. [1][2]
  • For HTTP 4xx/5xx: by default, curl can still exit 0 even if the server responds 404, 500, etc., because the transfer itself succeeded at the HTTP layer. If you want HTTP >= 400 to be treated as a failure (non-zero exit), use -f / --fail (or --fail-with-body). [3]
  • With -f/--fail, curl uses exit code 22 for “HTTP error returned” (4xx/5xx). [4]

Common script pattern:

curl -fsS https://example.com/resource   # silent, show errors, fail (exit!=0) on 4xx/5xx

Sources: [1] [2] [3] [4]


Add -f flag to curl commands so the health target fails on HTTP errors.

Lines 102–103 use curl -s which exits 0 even on HTTP 4xx/5xx responses (the transfer itself succeeds). Without the -f flag, make health will report success when services are unhealthy. Use curl -fsS instead to fail fast on HTTP errors; optionally add --max-time 5 to prevent hangs on stalled connections.

Proposed fix
 health: ## Check health of all services
-	curl -s http://localhost:8086/health | jq
-	curl -s http://localhost:4000/health | jq
+	curl -fsS --max-time 5 http://localhost:8086/health | jq
+	curl -fsS --max-time 5 http://localhost:4000/health | jq
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Makefile` around lines 101 - 103, The health Makefile target uses silent curl
commands that don't fail on HTTP errors; update the two curl invocations in the
"health" target to use curl -fsS (and optionally add --max-time 5) so the
command will exit non‑zero on 4xx/5xx responses and time out reliably; modify
the curl lines under the health target (the two curl commands calling
http://localhost:8086/health and http://localhost:4000/health) to include -fsS
(and --max-time 5 if desired).

180 changes: 180 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# GrainGuard

> Production-grade, multi-tenant agri-operations SaaS platform.
> Built as a Staff+/Principal Engineer portfolio reference system.

[![CI](https://github.com/pahuldeepp/GrainGuard-/actions/workflows/ci.yml/badge.svg)](https://github.com/pahuldeepp/GrainGuard-/actions)

---

## What is GrainGuard?

GrainGuard ingests telemetry from grain bins and field devices (temperature,
humidity, CO2), computes spoilage risk scores, and automates alert workflows.

It is deliberately architected to demonstrate end-to-end distributed systems
patterns from DDIA and Staff-level engineering practices.

---

## Architecture
```
React/Next.js Dashboard
GraphQL Gateway (Node) — auth, rate limiting, versioning
BFF (Node/Apollo) — resolvers, Redis cache, circuit breaker
┌─────────────────────────────────────────┐
│ Telemetry Ingest (Go) │
│ Asset Registry (Go) │
│ Risk Engine (Python) │
│ Workflow Alerts (Node) │
│ Search Indexer (Python) │
│ Jobs Worker (Node) │
└─────────────────────────────────────────┘
Kafka (domain + CDC) — RabbitMQ (tasks)
┌─────────────────────────────────────────┐
│ Postgres (OLTP + Outbox) │
│ Cassandra (Time-series telemetry) │
│ Elasticsearch (Search) │
│ Redis (Cache + Locks + Rate limiting) │
│ Memcached (Volatile dashboard cache) │
└─────────────────────────────────────────┘
```

---

## Key Engineering Patterns

| Pattern | Implementation |
|---|---|
| CQRS | Separate write (Postgres) and read (projections) models |
| Transactional Outbox | Reliable event publishing without dual-write |
| Saga Orchestration | Device provisioning with compensation |
| CDC | Debezium + Kafka Connect for Postgres WAL streaming |
| Exactly-once-ish | `processed_events` unique insert + projection UPSERT |
| Multi-tenancy | Postgres RLS + tenant-scoped queries |
| Rate limiting | Redis sliding window — 7-layer architecture |
| Cache stampede | Redis distributed lock on cache miss |
| Circuit breaker | BFF → Postgres, CLOSED/OPEN/HALF_OPEN states |
| gRPC + mTLS | Internal service communication with zero-trust mesh |

---

## Tech Stack

**Languages:** Go, TypeScript/Node, Python
**Frontend:** React, Apollo Client, Tailwind, Storybook
**Databases:** Postgres, Cassandra, Elasticsearch, Redis, Memcached
**Messaging:** Kafka (KRaft), RabbitMQ, Debezium
**Infra:** Docker, Kubernetes, Helm
**Observability:** OpenTelemetry, Prometheus, Grafana, Loki, Tempo
**Auth:** Auth0, JWT RS256, RBAC
**CI/CD:** GitHub Actions

---

## Getting Started

### Prerequisites
- Docker + Docker Compose
- Node.js 20+
- Go 1.25+
- Python 3.12+

### One-command bootstrap
```bash
bash scripts/bootstrap.sh
```

### Manual setup
```bash
# Start all infrastructure
make up

# Seed dev data
make seed

# Run tests
make test
```

### Available commands
```bash
make help
```

---

## Services

| Service | Language | Port | Description |
|---|---|---|---|
| gateway | Node | 8086 | GraphQL gateway, auth, rate limiting |
| bff | Node | 4000 | Apollo GraphQL BFF |
| dashboard | React | 5173 | Frontend UI |
| telemetry-service | Go | 50051 | gRPC telemetry ingest |
| read-model-builder | Go | - | Kafka → Postgres projections |
| saga-orchestrator | Go | - | Device provisioning saga |
| cdc-transformer | Go | - | Debezium CDC transformer |
| search-indexer | Python | - | Elasticsearch indexer |
| jobs-worker | Node | - | RabbitMQ async jobs |

---

## Observability

| Tool | URL | Purpose |
|---|---|---|
| Grafana | http://localhost:3000 | Dashboards + alerts |
| Prometheus | http://localhost:9090 | Metrics |
| Kibana | http://localhost:5601 | Elasticsearch UI |
| RabbitMQ UI | http://localhost:15672 | Queue management |
| Schema Registry | http://localhost:8082 | Protobuf schemas |

---

## Architecture Decision Records

10 ADRs documented in [`/docs/adr`](/docs/adr):

| ADR | Decision |
|---|---|
| ADR-001 | gRPC for internal service communication |
| ADR-002 | Transactional outbox over dual writes |
| ADR-003 | SAGA orchestration for device provisioning |
| ADR-004 | Cassandra TWCS for telemetry time-series |
| ADR-005 | Postgres RLS for standard-tier tenant isolation |
| ADR-006 | Redis lock with fencing tokens |
| ADR-007 | Reject cross-datastore 2PC |
| ADR-008 | Kafka KRaft over ZooKeeper |
| ADR-009 | Serializable isolation for write-skew ops |
| ADR-010 | etcd leases for singleton jobs |

---

## Load Testing Results

- **Gateway:** 1,700 events/sec, p95 5.89ms
- **Test tool:** k6 spike/soak/stress
- **Results:** [`/docs/load`](/docs/load)

---

## Project Status

| Milestone | Status |
|---|---|
| R1 — Core correctness loop | 🟡 In Progress |
| R2 — CDC + Search | ✅ Done |
| R3 — Massive scale | 🔴 Pending |
| R4 — 1B hardening | 🔴 Pending |

---

## License

Private — portfolio project.
1 change: 1 addition & 0 deletions apps/bff/src/datasources/cassandra.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,4 @@ export async function closeCassandraClient(): Promise<void> {
client = null;
}
}

1 change: 1 addition & 0 deletions apps/bff/src/datasources/elasticsearch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,4 @@ export const search = {
}
},
};

Loading
Loading