Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion go/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
ARG BASE_IMAGE=gcr.io/distroless/static-debian12:latest

FROM golang:1.25 AS builder

WORKDIR /go/src/github.com/unkeyed/unkey/go
Expand All @@ -9,7 +11,8 @@ ARG VERSION
ENV CGO_ENABLED=0
RUN go build -o bin/unkey -ldflags="-X 'github.com/unkeyed/unkey/go/pkg/version.Version=${VERSION}'" ./main.go

FROM gcr.io/distroless/static-debian12
FROM ${BASE_IMAGE}

COPY --from=builder /go/src/github.com/unkeyed/unkey/go/bin/unkey /

LABEL org.opencontainers.image.source=https://github.com/unkeyed/unkey/go
Expand Down
9 changes: 9 additions & 0 deletions go/Dockerfile.tilt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM busybox:latest

# Copy the pre-built binary
COPY bin/unkey /unkey

LABEL org.opencontainers.image.source=https://github.com/unkeyed/unkey/go
LABEL org.opencontainers.image.description="Unkey API"

ENTRYPOINT ["/unkey"]
65 changes: 65 additions & 0 deletions go/K8S_DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Kubernetes Development Setup

Run Unkey services locally using Kubernetes instead of Docker Compose.

## Prerequisites

- Docker Desktop with Kubernetes enabled OR OrbStack with Kubernetes enabled
- kubectl

Check requirements:
```bash
make k8s-check
```

## Quick Start

Start everything:
```bash
make k8s-up
```

Start with hot reloading (requires Tilt):
```bash
make dev
```

## Individual Services

```bash
make start-mysql
make start-clickhouse
make start-redis
make start-s3
make start-api
make start-gw
make start-ctrl
```

## Management

```bash
# Stop everything
make k8s-down

# Reset environment
make k8s-reset

# View services
kubectl get pods -n unkey
kubectl get services -n unkey
```
Comment on lines +39 to +51
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Document ‘k8s-status’ target and warn about destructive reset.

Expose status helper and add a data-loss warning for reset.

 ## Management
 
 ```bash
 # Stop everything
 make k8s-down
 
 # Reset environment
 make k8s-reset
 
-# View services
+# Status and services
+make k8s-status
 kubectl get pods -n unkey
 kubectl get services -n unkey

+Warning: make k8s-reset may delete persistent volumes and data (mysql/clickhouse/minio). Back up data or use ephemeral test DBs before running.


<details>
<summary>🤖 Prompt for AI Agents</summary>

In go/K8S_DEVELOPMENT.md around lines 39 to 51, the Make targets section should
expose a status helper and warn about data loss from reset: add a new "make
k8s-status" entry under a renamed subsection (e.g., "Status and services") and
insert a clear warning that "make k8s-reset" may delete persistent volumes and
data (mysql/clickhouse/minio), advising to back up data or use ephemeral test
DBs before running; ensure the new lines appear before the kubectl commands and
format the warning as a separate paragraph or block for visibility.


</details>

<!-- fingerprinting:phantom:triton:chinchilla -->

<!-- This is an auto-generated comment by CodeRabbit -->


## Tilt (Optional)

Start specific services:
```bash
tilt up -- --services=mysql --services=clickhouse
tilt up -- --services=api --services=gw --services=ctrl
tilt up -- --services=all
```

Stop Tilt:
```bash
tilt down
```
105 changes: 104 additions & 1 deletion go/Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: install tools fmt test-unit test-integration test-integration-long test-stress test build generate pull up clean
.PHONY: install tools fmt test-unit test-integration test-integration-long test-stress test build generate pull up clean k8s-check k8s-up k8s-down k8s-reset k8s-status start-mysql start-ctrl start-all dev

# Detect OS and set GOMAXPROCS accordingly
UNAME_S := $(shell uname -s)
Expand Down Expand Up @@ -68,3 +68,106 @@ test-integration: export SIMULATION_TEST=false
test-integration: up
@echo "Running integration tests w/$(PARALLEL_PROCS) parallel test processes"
@go test -tags=integration -json -failfast -timeout=15m -parallel=$(PARALLEL_PROCS) ./... | tparse -all -progress -smallscreen

# ============================================================================
# Kubernetes Development Commands
# ============================================================================

k8s-check: ## Check if kubectl is available and cluster is running
@command -v kubectl >/dev/null 2>&1 || { echo "ERROR: kubectl not found. Install from: https://kubernetes.io/docs/tasks/tools/"; exit 1; }
@kubectl cluster-info >/dev/null 2>&1 || { echo "ERROR: Kubernetes cluster not available. Enable Kubernetes in Docker Desktop/OrbStack"; exit 1; }
@echo "Kubernetes cluster is available"

k8s-up: k8s-check ## Deploy all services to current Kubernetes cluster
@echo "Building Docker images..."
@docker build -t unkey/mysql:latest -f ../deployment/Dockerfile.mysql ../
@docker build -t unkey/clickhouse:latest -f ../deployment/Dockerfile.clickhouse ../
@docker build -t unkey:latest .
@echo "Creating namespace..."
@kubectl apply -f k8s/manifests/namespace.yaml
@echo "Applying Kubernetes manifests..."
@kubectl apply -f k8s/manifests/
@echo "Waiting for services to be ready..."
@kubectl wait --for=condition=ready pod -l app=mysql -n unkey --timeout=180s
@kubectl wait --for=condition=ready pod -l app=clickhouse -n unkey --timeout=180s
@kubectl wait --for=condition=ready pod -l app=s3 -n unkey --timeout=180s
@kubectl wait --for=condition=ready pod -l app=api -n unkey --timeout=180s
@kubectl wait --for=condition=ready pod -l app=gw -n unkey --timeout=180s
@kubectl wait --for=condition=ready pod -l app=ctrl -n unkey --timeout=180s
@kubectl wait --for=condition=ready pod -l app=dashboard -n unkey --timeout=180s
@echo "Kubernetes environment is ready!"
@echo ""
@echo "Services accessible via NodePort on localhost - check actual ports with:"
@echo ""
@echo "Check NodePort assignments: make k8s-ports"
@make k8s-status

Comment on lines +81 to +104
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add waits for all dependencies, configurable timeout, and implement referenced status/ports targets.

  • api/gw/ctrl can flap if redis/planetscale/observability aren’t ready.
  • Replace hardcoded 180s with K8S_WAIT_TIMEOUT?=.
  • You echo “make k8s-ports” and “k8s-status” but only the latter is missing; implement both.

Apply:

+k8s_wait_timeout ?= 300
 k8s-up: k8s-check ## Deploy all services to current Kubernetes cluster
 	@echo "Building Docker images..."
 	@docker build -t unkey/mysql:latest -f ../deployment/Dockerfile.mysql ../
 	@docker build -t unkey/clickhouse:latest -f ../deployment/Dockerfile.clickhouse ../
 	@docker build -t unkey:latest .
 	@echo "Creating namespace..."
 	@kubectl apply -f k8s/manifests/namespace.yaml
 	@echo "Applying Kubernetes manifests..."
 	@kubectl apply -f k8s/manifests/
 	@echo "Waiting for services to be ready..."
-	@kubectl wait --for=condition=ready pod -l app=mysql -n unkey --timeout=180s
-	@kubectl wait --for=condition=ready pod -l app=clickhouse -n unkey --timeout=180s
-	@kubectl wait --for=condition=ready pod -l app=s3 -n unkey --timeout=180s
-	@kubectl wait --for=condition=ready pod -l app=api -n unkey --timeout=180s
-	@kubectl wait --for=condition=ready pod -l app=gw -n unkey --timeout=180s
-	@kubectl wait --for=condition=ready pod -l app=ctrl -n unkey --timeout=180s
-	@kubectl wait --for=condition=ready pod -l app=dashboard -n unkey --timeout=180s
+	@kubectl wait --for=condition=ready pod -l app=mysql -n unkey --timeout=$(k8s_wait_timeout)s
+	@kubectl wait --for=condition=ready pod -l app=clickhouse -n unkey --timeout=$(k8s_wait_timeout)s
+	@kubectl wait --for=condition=ready pod -l app=redis -n unkey --timeout=$(k8s_wait_timeout)s
+	@kubectl wait --for=condition=ready pod -l app=planetscale -n unkey --timeout=$(k8s_wait_timeout)s || true
+	@kubectl wait --for=condition=ready pod -l app=s3 -n unkey --timeout=$(k8s_wait_timeout)s
+	@kubectl wait --for=condition=ready pod -l app=otel-collector -n unkey --timeout=$(k8s_wait_timeout)s || true
+	@kubectl wait --for=condition=ready pod -l app=api -n unkey --timeout=$(k8s_wait_timeout)s
+	@kubectl wait --for=condition=ready pod -l app=gw -n unkey --timeout=$(k8s_wait_timeout)s
+	@kubectl wait --for=condition=ready pod -l app=ctrl -n unkey --timeout=$(k8s_wait_timeout)s
+	@kubectl wait --for=condition=ready pod -l app=dashboard -n unkey --timeout=$(k8s_wait_timeout)s
 	@echo "Kubernetes environment is ready!"
 	@echo ""
 	@echo "Services accessible via NodePort on localhost - check actual ports with:"
 	@echo ""
 	@echo "Check NodePort assignments: make k8s-ports"
-	@make k8s-status
+	@$(MAKE) k8s-status

Add these targets:

k8s-status:
	@kubectl get pods -n unkey -o wide
	@kubectl get svc -n unkey

k8s-ports:
	@kubectl get svc -n unkey -o custom-columns=NAME:.metadata.name,TYPE:.spec.type,PORTS:.spec.ports[*].nodePort,CLUSTER-IP:.spec.clusterIP
🤖 Prompt for AI Agents
In go/Makefile around lines 81 to 104, the k8s-up target uses a hardcoded 180s
timeout and doesn't wait for all dependent services or provide the referenced
helper targets; change the timeout to a configurable variable
K8S_WAIT_TIMEOUT?=180s and replace each kubectl wait --timeout=180s with
--timeout=$(K8S_WAIT_TIMEOUT), add additional kubectl wait lines for redis,
planetscale and observability (or whatever service labels your manifests use) to
ensure api/gw/ctrl dependencies are ready, and implement the missing k8s-status
and k8s-ports phony targets as described so the echoed instructions actually
work.

k8s-stop: ## Stop all pods (scale deployments to 0)
@echo "Stopping all pods..."
@kubectl scale deployment --all --replicas=0 -n unkey
@echo "All pods stopped"

Comment on lines +105 to +109
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Stop also scales StatefulSets (MySQL/ClickHouse likely use them).

Current rule only scales Deployments.

 k8s-stop: ## Stop all pods (scale deployments to 0)
 	@echo "Stopping all pods..."
 	@kubectl scale deployment --all --replicas=0 -n unkey
+	@kubectl scale statefulset --all --replicas=0 -n unkey
 	@echo "All pods stopped"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
k8s-stop: ## Stop all pods (scale deployments to 0)
@echo "Stopping all pods..."
@kubectl scale deployment --all --replicas=0 -n unkey
@echo "All pods stopped"
k8s-stop: ## Stop all pods (scale deployments to 0)
@echo "Stopping all pods..."
@kubectl scale deployment --all --replicas=0 -n unkey
@kubectl scale statefulset --all --replicas=0 -n unkey
@echo "All pods stopped"
🤖 Prompt for AI Agents
In go/Makefile around lines 105 to 109, the k8s-stop target only scales
Deployments and therefore misses StatefulSets (used by MySQL/ClickHouse); update
the target to also scale StatefulSets to 0 by running kubectl scale statefulset
--all --replicas=0 -n unkey in addition to the existing deployment scale command
(preserve the echoes and error behavior).

k8s-down: ## Delete all services from current Kubernetes cluster
@echo "Deleting all services..."
@kubectl delete namespace unkey --ignore-not-found=true
@echo "Services deleted"

k8s-reset: k8s-down k8s-up ## Reset the entire Kubernetes environment

# Helper function to deploy a service
define deploy-service
@echo "Starting $(2)..."
$(if $(3),@docker build -t $(3) $(4))
@kubectl apply -f k8s/manifests/namespace.yaml
@kubectl apply -f k8s/manifests/$(1).yaml
@kubectl wait --for=condition=ready pod -l app=$(1) -n unkey --timeout=180s
$(if $(5),@kubectl wait --for=condition=ready pod -l app=$(5) -n unkey --timeout=180s)
@echo "$(2) is ready!"
endef

start-mysql: k8s-check ## Deploy only MySQL
$(call deploy-service,mysql,MySQL,unkey/mysql:latest,-f ../deployment/Dockerfile.mysql ../)

start-clickhouse: k8s-check ## Deploy only ClickHouse
@docker build -t unkey/clickhouse:latest -f ../deployment/Dockerfile.clickhouse ../
$(call deploy-service,clickhouse,ClickHouse)

start-redis: k8s-check ## Deploy only Redis
$(call deploy-service,redis,Redis)

start-s3: k8s-check ## Deploy only S3 (MinIO)
$(call deploy-service,s3,S3 (MinIO))

start-planetscale: k8s-check ## Deploy only PlanetScale HTTP driver
$(call deploy-service,planetscale,PlanetScale HTTP driver)

start-observability: k8s-check ## Deploy only Observability stack
$(call deploy-service,observability,Observability stack,,,otel-collector)

start-api: k8s-check ## Deploy only API service (3 replicas)
$(call deploy-service,api,API,unkey:latest,.)

start-gw: k8s-check ## Deploy only Gateway service
$(call deploy-service,gw,Gateway,unkey:latest,.)

start-ctrl: k8s-check ## Deploy only ctrl service
$(call deploy-service,ctrl,Control Plane,unkey:latest,.)

Comment on lines +147 to +155
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Ensure manifests use the locally built image and sane pull policy.

You build unkey:latest; make sure api/gw/ctrl manifests reference image: unkey:latest with imagePullPolicy: IfNotPresent (or Never for local).


🏁 Script executed:

#!/bin/bash
# Verify images/pullPolicies for api/gw/ctrl
rg -nC2 -g 'k8s/manifests/*.yaml' -e 'name:\s*(api|gw|ctrl)' -e 'image:\s*unkey:latest' -e 'imagePullPolicy:'

Length of output: 148


🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "PWD: $(pwd)"
echo
echo "Searching for 'deploy-service' usages..."
rg -n --hidden -S "deploy-service" || true
echo
echo "Listing k8s/manifests directories if they exist..."
[ -d k8s ] && ls -la k8s || echo "no k8s dir"
[ -d k8s/manifests ] && ls -la k8s/manifests || echo "no k8s/manifests dir"
echo
echo "Searching repo for image references to 'unkey' and for imagePullPolicy (with context)..."
rg -n --hidden -S -C2 -E 'image:\s*.*unkey(:latest)?|imagePullPolicy:|name:\s*(api|gw|ctrl)' || true
echo
echo "Listing all YAML files found (first 200 lines) for manual inspection:"
rg --hidden --files -g '*.yml' -g '*.yaml' -S | sed -n '1,200p' || true

Length of output: 15270


🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "=== Show deploy-service macro from go/Makefile ==="
awk 'BEGIN{p=0} /define deploy-service/{p=1} p{print NR": "$0} /endef/{if(p==1){print NR": "$0; exit}}' go/Makefile || true
echo
echo "=== Show Makefile lines around start-api/start-gw/start-ctrl ==="
sed -n '110,170p' go/Makefile || true
echo
for f in go/k8s/manifests/api.yaml go/k8s/manifests/gw.yaml go/k8s/manifests/ctrl.yaml; do
  echo "=== Grep relevant keys in $f ==="
  rg -n -C1 -E '^\s*(image:|imagePullPolicy:|name:|containers:)' "$f" || true
  echo
  echo "=== Full file ($f) head (first 200 lines) ==="
  sed -n '1,200p' "$f" || true
  echo
done
echo "=== Search repo for any 'image: .*unkey' or 'imagePullPolicy' occurrences ==="
rg -n --hidden -S -E 'image:\s*.*unkey(:latest)?|imagePullPolicy:' || true

Length of output: 14386


Fix image-tag mismatch: make built image tags match k8s manifests (or update manifests to match builds).

Makefile start-api/start-gw/start-ctrl build/tag unkey:latest, but go/k8s/manifests/api.yaml, gw.yaml, ctrl.yaml reference unkey-api:latest / unkey-gw:latest / unkey-ctrl:latest (they already set imagePullPolicy: Never). Action: either change go/Makefile to build/tag the per-service images (recommended — e.g., build unkey-api:latest, unkey-gw:latest, unkey-ctrl:latest) or update the manifests to use unkey:latest and set imagePullPolicy to IfNotPresent (or keep Never for local). Files: go/Makefile (start-api/start-gw/start-ctrl), go/k8s/manifests/{api,gw,ctrl}.yaml.

🤖 Prompt for AI Agents
In go/Makefile around lines 147-155 the start-api/start-gw/start-ctrl targets
currently reference images tagged as unkey:latest but the k8s manifests in
go/k8s/manifests/api.yaml, gw.yaml, ctrl.yaml expect unkey-api:latest,
unkey-gw:latest, unkey-ctrl:latest; update the Makefile so the start-* targets
build and tag the service-specific images (e.g., unkey-api:latest,
unkey-gw:latest, unkey-ctrl:latest) instead of unkey:latest, or alternatively
update the three manifest files to reference unkey:latest (and set
imagePullPolicy appropriately); preferred fix: change the Makefile to produce
per-service tags that match the manifests and keep imagePullPolicy as currently
configured in the manifests.

start-dashboard: k8s-check ## Deploy only dashboard service
$(call deploy-service,dashboard,Dashboard,unkey/dashboard:latest,-f ../apps/dashboard/Dockerfile ../)

start-unkey-services: start-api start-gw start-ctrl start-dashboard ## Deploy all Unkey services

start-dependencies: start-mysql start-clickhouse start-redis start-s3 start-planetscale start-observability ## Deploy all dependency services

start-all: start-dependencies start-unkey-services ## Deploy all services individually

dev: ## Start with Tilt (if available) or fallback to k8s-up
@if command -v tilt >/dev/null 2>&1; then \
echo "Starting with Tilt..."; \
tilt up; \
else \
echo "Tilt not found, using k8s-up instead"; \
echo "Install Tilt from: https://docs.tilt.dev/install.html"; \
make k8s-up; \
fi
Loading