Docs: improve approachability, discoverability, and content depth

Cookiecutter.md

@@ -1,13 +1,16 @@
 ---
 layout: default
-title: "Cookiecutter Setup"
-nav_order: 3
-description: "Detailed cookiecutter template setup for ColdBrew"
-permalink: /getting-started
+title: "Cookiecutter Reference"
+nav_order: 10
+description: "Detailed reference for the ColdBrew cookiecutter project template"
+permalink: /cookiecutter-reference
 ---
-# Cookiecutter Setup
+# Cookiecutter Reference
 {: .no_toc }
 
+{: .note }
+Looking to create your first ColdBrew service? See the **[Getting Started](/getting-started)** guide instead. This page is a detailed reference for the cookiecutter template.
+
 ## Table of contents
 {: .no_toc .text-delta }
 

FAQ.md

@@ -2,7 +2,7 @@
 layout: default
 title: "FAQ"
 nav_order: 8
-description: "Frequently asked questions about ColdBrew"
+description: "Frequently asked questions about ColdBrew: gRPC framework configuration, interceptors, tracing, and troubleshooting"
 permalink: /faq
 ---
 # Frequently Asked Questions

Packages.md

@@ -1,7 +1,7 @@
 ---
 layout: default
 title: Packages
-description: "ColdBrew packages documentation"
+description: "ColdBrew Go packages: core, interceptors, errors, log, tracing, options, grpcpool, and data-builder API reference"
 permalink: /packages
 nav_order: 9
 ---

USING.md

@@ -101,11 +101,13 @@ ColdBrew uses environment variables for configuration. Common settings:
 | `HTTP_PORT` | `9091` | HTTP gateway port |
 | `LOG_LEVEL` | `info` | Log level (debug, info, warn, error) |
 | `JSON_LOGS` | `true` | JSON formatted logs |
-| `ENVIRONMENT` | `development` | Environment name |
-| `TRACE_HEADER_NAME` | `X-Trace-Id` | Header name for trace propagation |
-| `NEW_RELIC_APP_NAME` | | New Relic application name |
-| `NEW_RELIC_LICENSE_KEY` | | New Relic license key |
-| `SENTRY_DSN` | | Sentry DSN for error tracking |
+| `ENVIRONMENT` | `""` | Environment name |
+| `TRACE_HEADER_NAME` | `x-trace-id` | Header name for trace propagation |
+| `NEW_RELIC_APPNAME` | `""` | New Relic application name |
+| `NEW_RELIC_LICENSE_KEY` | `""` | New Relic license key |
+| `SENTRY_DSN` | `""` | Sentry DSN for error tracking |
+
+See the **[Configuration Reference](/config-reference)** for the complete list of 40+ environment variables including gRPC keepalive, TLS, OpenTelemetry OTLP, Prometheus histogram buckets, and graceful shutdown tuning.
 
 ## Adding Interceptors
 

config-reference.md

@@ -0,0 +1,169 @@
+---
+layout: default
+title: "Configuration Reference"
+nav_order: 5
+description: "Complete environment variable reference for ColdBrew Go microservice framework configuration"
+permalink: /config-reference
+---
+# Configuration Reference
+{: .no_toc }
+
+## Table of contents
+{: .no_toc .text-delta }
+
+1. TOC
+{:toc}
+
+---
+
+ColdBrew is configured entirely through environment variables using [envconfig](https://github.com/kelseyhightower/envconfig). All fields have sensible defaults — you can run a service with zero configuration.
+
+Access the config in code via:
+
+```go
+cfg := config.GetColdBrewConfig()
+```
+
+## Server
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `LISTEN_HOST` | string | `0.0.0.0` | Host address to listen on |
+| `GRPC_PORT` | int | `9090` | gRPC server port |
+| `HTTP_PORT` | int | `9091` | HTTP gateway port |
+| `APP_NAME` | string | `""` | Application name (used in logs, metrics, New Relic) |
+| `ENVIRONMENT` | string | `""` | Environment name (e.g., production, staging, development) |
+| `RELEASE_NAME` | string | `""` | Release/version name |
+
+## Logging
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `LOG_LEVEL` | string | `info` | Log level: debug, info, warn, error |
+| `JSON_LOGS` | bool | `true` | Emit logs in JSON format |
+
+## gRPC Server
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `DISABLE_GRPC_REFLECTION` | bool | `false` | Disable gRPC server reflection (used by tools like grpcurl) |
+| `DO_NOT_LOG_GRPC_REFLECTION` | bool | `true` | Suppress logging of gRPC reflection API calls |
+| `GRPC_MAX_SEND_MSG_SIZE` | int | `2147483647` | Maximum send message size in bytes (default: ~2GB, unlimited) |
+| `GRPC_MAX_RECV_MSG_SIZE` | int | `4194304` | Maximum receive message size in bytes (default: 4MB) |
+| `DISABLE_VT_PROTOBUF` | bool | `false` | Disable [vtprotobuf](https://github.com/planetscale/vtprotobuf) marshaller for gRPC |
+
+## gRPC TLS
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `GRPC_TLS_KEY_FILE` | string | `""` | Path to TLS private key file. Both key and cert must be set to enable TLS |
+| `GRPC_TLS_CERT_FILE` | string | `""` | Path to TLS certificate file. Both key and cert must be set to enable TLS |
+| `GRPC_TLS_INSECURE_SKIP_VERIFY` | bool | `false` | Skip TLS certificate verification (development only) |
+
+## gRPC Keepalive
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `GRPC_SERVER_MAX_CONNECTION_IDLE_IN_SECONDS` | int | `0` | Close idle connections after this duration (0 = disabled) |
+| `GRPC_SERVER_MAX_CONNECTION_AGE_IN_SECONDS` | int | `0` | Maximum connection lifetime with ±10% jitter (0 = disabled) |
+| `GRPC_SERVER_MAX_CONNECTION_AGE_GRACE_IN_SECONDS` | int | `0` | Grace period after max connection age before force-closing |
+
+## HTTP Gateway
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `DISABLE_SWAGGER` | bool | `false` | Disable Swagger UI at the swagger URL |
+| `SWAGGER_URL` | string | `/swagger/` | URL path for Swagger UI |
+| `DISABLE_DEBUG` | bool | `false` | Disable pprof debug endpoints at `/debug/` |
+| `USE_JSON_BUILTIN_MARSHALLER` | bool | `false` | Use `encoding/json` instead of the default protojson marshaller for `application/json` |
+| `JSON_BUILTIN_MARSHALLER_MIME` | string | `application/json` | Content-Type for the JSON builtin marshaller |
+| `HTTP_HEADER_PREFIXES` | []string | `""` | HTTP header prefixes to forward as gRPC metadata (comma-separated) |
+| `TRACE_HEADER_NAME` | string | `x-trace-id` | HTTP header name for trace ID propagation to log/trace contexts |
+
+## Prometheus Metrics
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `DISABLE_PROMETHEUS` | bool | `false` | Disable Prometheus metrics endpoint at `/metrics` |
+| `ENABLE_PROMETHEUS_GRPC_HISTOGRAM` | bool | `true` | Enable gRPC request latency histograms |
+| `PROMETHEUS_GRPC_HISTOGRAM_BUCKETS` | []float64 | `""` | Custom histogram buckets (comma-separated seconds, e.g., `0.005,0.01,0.025,0.05,0.1,0.25,0.5,1,2.5,5,10`) |
+
+## New Relic
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `NEW_RELIC_LICENSE_KEY` | string | `""` | New Relic license key (required to enable New Relic) |
+| `NEW_RELIC_APPNAME` | string | `""` | Application name in New Relic |
+| `DISABLE_NEW_RELIC` | bool | `false` | Disable all New Relic reporting |
+| `NEW_RELIC_DISTRIBUTED_TRACING` | bool | `true` | Enable New Relic distributed tracing |
+| `NEW_RELIC_OPENTELEMETRY` | bool | `true` | Enable New Relic via OpenTelemetry |
+| `NEW_RELIC_OPENTELEMETRY_SAMPLE` | float64 | `0.2` | Trace sampling ratio for New Relic OpenTelemetry (0.0–1.0) |
+
+## OpenTelemetry (OTLP)
+
+{: .note }
+When `OTLP_ENDPOINT` is set, it takes precedence over New Relic OpenTelemetry configuration.
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `OTLP_ENDPOINT` | string | `""` | OTLP gRPC endpoint (e.g., `localhost:4317`, `api.honeycomb.io:443`) |
+| `OTLP_HEADERS` | string | `""` | Custom headers as `key=value` pairs (comma-separated, e.g., `x-honeycomb-team=your-key`) |
+| `OTLP_COMPRESSION` | string | `gzip` | Compression type: `gzip` or `none` |
+| `OTLP_INSECURE` | bool | `false` | Disable TLS for OTLP connection (development only) |
+| `OTLP_SAMPLING_RATIO` | float64 | `0.2` | Trace sampling ratio (0.0–1.0, where 1.0 = sample all) |
+| `OTLP_USE_OPENTRACING_BRIDGE` | bool | `true` | Enable OpenTracing compatibility bridge for existing instrumentation |
+
+## Error Tracking
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `SENTRY_DSN` | string | `""` | Sentry DSN for error notification |
+
+## Graceful Shutdown
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `DISABLE_SIGNAL_HANDLER` | bool | `false` | Disable ColdBrew's SIGINT/SIGTERM handler |
+| `SHUTDOWN_DURATION_IN_SECONDS` | int | `15` | Time to wait for in-flight requests to complete before forced shutdown |
+| `GRPC_GRACEFUL_DURATION_IN_SECONDS` | int | `7` | Time to wait for healthcheck failure to propagate before initiating shutdown. Should be less than `SHUTDOWN_DURATION_IN_SECONDS` |
+
+## Runtime
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `DISABLE_AUTO_MAX_PROCS` | bool | `false` | Disable automatic GOMAXPROCS tuning (useful if your container runtime already sets it) |
+
+## Deprecated
+
+| Variable | Replacement | Notes |
+|----------|------------|-------|
+| `HTTP_HEADER_PREFIX` | `HTTP_HEADER_PREFIXES` | Single prefix replaced by comma-separated list |
+
+---
+
+## Example: Minimal Production Configuration
+
+```bash
+export APP_NAME=myservice
+export ENVIRONMENT=production
+export LOG_LEVEL=info
+export NEW_RELIC_LICENSE_KEY=your-key
+export NEW_RELIC_APPNAME=myservice
+export SENTRY_DSN=https://your-dsn@sentry.io/123
+```
+
+## Example: Local Development with Jaeger
+
+```bash
+export APP_NAME=myservice
+export ENVIRONMENT=development
+export LOG_LEVEL=debug
+export OTLP_ENDPOINT=localhost:4317
+export OTLP_INSECURE=true
+export OTLP_SAMPLING_RATIO=1.0
+export DISABLE_NEW_RELIC=true
+```
+
+---
+
+Source: [`core/config/config.go`](https://github.com/go-coldbrew/core/blob/main/config/config.go)

howto/APIs.md

@@ -2,6 +2,7 @@
 layout: default
 title: "Building and Configuring APIs"
 parent: "How To"
+description: "Build gRPC and REST APIs with ColdBrew using protobuf definitions and grpc-gateway HTTP annotations"
 ---
 ## Table of contents
 {: .no_toc .text-delta }
@@ -543,5 +544,5 @@ For more advanced customization options, refer to the [grpc-gateway customizatio
 [grpc-gateway]: https://grpc-ecosystem.github.io/grpc-gateway/
 [gRPC Gateway mapping]: https://grpc-ecosystem.github.io/grpc-gateway/docs/mapping/examples/
 [grpc-gateway plugin]: https://grpc-ecosystem.github.io/grpc-gateway/docs/tutorials/generating_stubs/
-[ColdBrew cookiecutter]: /getting-started#using-the-coldbrew-cookiecutter-template
+[ColdBrew cookiecutter]: /cookiecutter-reference#using-the-coldbrew-cookiecutter-template
 [grpc-gateway customization guide]: https://grpc-ecosystem.github.io/grpc-gateway/docs/mapping/customizing_your_gateway/

howto/Debugging.md

@@ -58,6 +58,92 @@ ColdBrew provides a way to override the log level of a request based on the requ
 
 For information on this feature, please refer to the [Overriding log level at request time] page.
 
+## Debugging with Delve
+
+[Delve](https://github.com/go-delve/delve) is the standard Go debugger. To debug a ColdBrew service:
+
+```bash
+# Install delve
+go install github.com/go-delve/delve/cmd/dlv@latest
+
+# Run your service under delve
+dlv debug . -- [flags]
+
+# Or attach to a running process
+dlv attach $(pgrep myservice)
+```
+
+### Useful breakpoint locations
+
+When debugging ColdBrew services, these are good places to set breakpoints:
+
+- **Your handler**: `break service/service.go:42` — your gRPC method implementation
+- **Interceptor chain entry**: `break github.com/go-coldbrew/interceptors.UnaryServerInterceptor` — see what interceptors fire
+- **Error notification**: `break github.com/go-coldbrew/errors/notifier.Notify` — catch when errors are sent to Sentry/Rollbar
+
+### VS Code / GoLand
+
+Both IDEs support Delve natively. Set `"host": "0.0.0.0", "port": 9090` in your launch.json to match ColdBrew's default gRPC port, or use a custom port to avoid conflicts.
+
+## gRPC debugging environment variables
+
+Go's gRPC library has built-in debug logging. These environment variables are useful when troubleshooting connectivity or protocol issues:
+
+```bash
+# Enable gRPC internal logging (WARNING: very verbose)
+export GRPC_GO_LOG_VERBOSITY_LEVEL=99
+export GRPC_GO_LOG_SEVERITY_LEVEL=info
+```
+
+This will print detailed gRPC transport and connection state information to stderr. Useful for diagnosing:
+- Connection establishment failures
+- TLS handshake issues
+- Load balancer resolution problems
+- Keepalive/ping timeouts
+
+{: .warning }
+Do not enable verbose gRPC logging in production — it generates enormous log volume and may impact performance.
+
+## Inspecting the interceptor chain
+
+ColdBrew chains interceptors in a specific order. If you're not sure what's running, you can inspect the chain at startup by setting `LOG_LEVEL=debug`:
+
+```bash
+LOG_LEVEL=debug make run
+```
+
+The server interceptor chain runs in this order:
+1. Response time logging
+2. Trace ID injection
+3. Context tags
+4. OpenTracing/OpenTelemetry
+5. Prometheus metrics
+6. Error notification
+7. NewRelic
+8. Panic recovery
+
+If a request is failing or behaving unexpectedly, check whether an interceptor is modifying the context or returning early. The response time logging interceptor logs every request with method name and duration — check these logs first.
+
+## Common error patterns
+
+### "transport is closing"
+Usually means the client connection was closed before the response arrived. Check:
+- `SHUTDOWN_DURATION_IN_SECONDS` is long enough for your slowest requests
+- Client-side timeouts match server-side processing time
+- Load balancer idle timeout isn't shorter than your keepalive settings
+
+### "context deadline exceeded"
+The request's context expired. This propagates through the interceptor chain. Check:
+- Client-side deadline/timeout settings
+- Whether a downstream dependency (database, external API) is slow
+- Circuit breaker state via Prometheus metrics
+
+### Metrics endpoint returns 404
+Prometheus is disabled. Check `DISABLE_PROMETHEUS` environment variable (should be `false` or unset).
+
+### Health check returns error
+The service hasn't called `SetReady()` yet. This typically happens during startup while dependencies are initializing. Check your service's `InitGRPC` method.
+
 ---
 [configuration option]: https://pkg.go.dev/github.com/go-coldbrew/core/config#Config
 [Overriding log level at request time]: /howto/Log/#overriding-log-level-at-request-time

howto/Log.md

@@ -139,7 +139,7 @@ Will output the debug log messages even when the global log level is set to info
 ---
 [TraceId interceptor]: https://pkg.go.dev/github.com/go-coldbrew/interceptors#TraceIdInterceptor
 [go-coldbrew/tracing]: https://pkg.go.dev/github.com/go-coldbrew/tracing
-[ColdBrew cookiecutter]: /getting-started
+[ColdBrew cookiecutter]: /cookiecutter-reference
 [interceptors]: https://pkg.go.dev/github.com/go-coldbrew/interceptors
 [UseColdBrewServcerInterceptors]: https://pkg.go.dev/github.com/go-coldbrew/interceptors#UseColdBrewServerInterceptors
 [OverrideLogLevel]: https://github.com/go-coldbrew/log#func-overrideloglevel

howto/Tracing.md

@@ -2,6 +2,7 @@
 layout: default
 title: "Tracing"
 parent: "How To"
+description: "Set up distributed tracing in ColdBrew with OpenTelemetry, Jaeger, and New Relic for gRPC services"
 ---
 ## Table of contents
 {: .no_toc .text-delta }
@@ -151,7 +152,7 @@ The functions `CloneContextValues` and `MergeParentContext` are deprecated. Use
 
 [TraceId interceptor]: https://pkg.go.dev/github.com/go-coldbrew/interceptors#TraceIdInterceptor
 [go-coldbrew/tracing]: https://pkg.go.dev/github.com/go-coldbrew/tracing
-[ColdBrew cookiecutter]: /getting-started
+[ColdBrew cookiecutter]: /cookiecutter-reference
 [interceptors]: https://pkg.go.dev/github.com/go-coldbrew/interceptors
 [UseColdBrewServcerInterceptors]: https://pkg.go.dev/github.com/go-coldbrew/interceptors#UseColdBrewServerInterceptors
 [Default Client Interceptors]: https://pkg.go.dev/github.com/go-coldbrew/interceptors#DefaultClientInterceptors