Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,9 +121,9 @@ See [Custom HTTP Routes](/howto/APIs/#custom-http-routes) for full examples incl

## Is hystrixprometheus still maintained?

**No.** The `hystrixprometheus` package depends on `afex/hystrix-go`, which is unmaintained. Do not invest in this package for new projects.
**No.** The `hystrixprometheus` package depends on `afex/hystrix-go`, which is unmaintained and will be removed in v1.
Comment thread
ankurs marked this conversation as resolved.
Outdated

For circuit breaking, consider [failsafe-go](https://github.com/failsafe-go/failsafe-go) as an alternative. The client-side interceptors in the `interceptors` package provide retry and circuit breaking functionality that covers most use cases.
ColdBrew now provides a pluggable executor hook (`interceptors.SetDefaultExecutor`) for circuit breaking. Use [failsafe-go](https://github.com/failsafe-go/failsafe-go) as the recommended resilience library. See the [integrations page](/integrations/#circuit-breaker--resilience) for setup examples.
Comment thread
ankurs marked this conversation as resolved.
Outdated

## How do I do cross-package development?

Expand Down
2 changes: 1 addition & 1 deletion Index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ A Kubernetes-native Go microservice framework for building production-grade gRPC
| **gRPC + REST Gateway** | Define your API once in protobuf — get gRPC, REST, and [Swagger docs](/architecture#self-documenting-apis) automatically via [grpc-gateway]. HTTP gateway supports JSON, `application/proto`, and `application/protobuf` [content types](/howto/APIs/#http-content-type) out of the box |
| **Structured Logging** | Native [slog] with custom Handler — per-request context fields, trace ID propagation, and typed attrs for zero-boxing performance |
| **Distributed Tracing** | [OpenTelemetry] and [New Relic] support with automatic span creation via gRPC stats handlers — traces can be sent to any OTLP-compatible backend including [Jaeger] |
| **Prometheus Metrics** | Built-in request latency, error rate, and circuit breaker metrics at `/metrics` |
| **Prometheus Metrics** | Built-in request latency, error rate metrics at `/metrics`, plus pluggable [resilience executor hook](/integrations/#circuit-breaker--resilience) for circuit breaking |
| **Error Tracking** | Stack traces, gRPC status codes, and async notification to [Sentry], Rollbar, or Airbrake |
| **Rate Limiting** | Per-pod token bucket rate limiter — disabled by default, pluggable via custom [`ratelimit.Limiter`](https://pkg.go.dev/github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/ratelimit#Limiter) interface for distributed or per-tenant rate limiting. Config: `RATE_LIMIT_PER_SECOND`. See [interceptors howto](/howto/interceptors#rate-limiting) |
| **Auth Examples** | JWT and API key authentication interceptor examples in the [cookiecutter template][ColdBrew cookiecutter], built on [go-grpc-middleware auth](https://github.com/grpc-ecosystem/go-grpc-middleware/tree/main/interceptors/auth). See [auth howto](/howto/auth/) |
Expand Down
4 changes: 2 additions & 2 deletions Packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ Tracing is a library that provides distributed tracing to Go applications. It of

Documentation can be found at [tracing-docs]

## [Hystrix Prometheus] (Deprecated)
hystrixprometheus provides a Prometheus metrics collector for Hystrix (https://github.com/afex/hystrix-go). This package is deprecated as hystrix-go is unmaintained. Consider migrating to [failsafe-go](https://github.com/failsafe-go/failsafe-go).
## [Hystrix Prometheus] (Deprecated — will be archived in v1)
hystrixprometheus provides a Prometheus metrics collector for Hystrix. This package is deprecated as hystrix-go is unmaintained. Use `interceptors.SetDefaultExecutor` with [failsafe-go](https://github.com/failsafe-go/failsafe-go) instead. See [integrations](/integrations/#circuit-breaker--resilience).
Comment thread
ankurs marked this conversation as resolved.

Documentation can be found at [hystrixprometheus-docs]

Expand Down
36 changes: 14 additions & 22 deletions howto/Metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ layout: default
title: "Metrics"
parent: "How To"
nav_order: 6
description: "Prometheus and OpenTelemetry metrics in ColdBrew: default runtime metrics, OTLP export, custom counters and histograms, and Hystrix circuit breaker monitoring"
description: "Prometheus and OpenTelemetry metrics in ColdBrew: default runtime metrics, OTLP export, custom counters and histograms, and circuit breaker monitoring"
---
## Table of contents
{: .no_toc .text-delta }
Expand Down Expand Up @@ -136,32 +136,24 @@ Health check, readiness, and server reflection RPCs are bucketed under a generic

Both export pipelines use independent metric names and registries, so there is no conflict or double-counting.

## How to use Hystrix Metrics in Prometheus
## Circuit Breaker Metrics

{: .warning }
Hystrix-Go is unmaintained (last updated 2018). Consider migrating to [failsafe-go](https://github.com/failsafe-go/failsafe-go) for circuit breaker functionality.

[Hystrix Prometheus] is a library that provides a Prometheus metrics collector for [Hystrix-go]. To use it, you can register the collector with the default Prometheus registry:
Circuit breaker metrics are the responsibility of the resilience library you plug into ColdBrew's executor hook. For example, with [failsafe-go](https://github.com/failsafe-go/failsafe-go) you can attach event listeners to track circuit state:

Comment thread
ankurs marked this conversation as resolved.
```go

import (
metricCollector "github.com/afex/hystrix-go/hystrix/metric_collector"
"github.com/go-coldbrew/hystrixprometheus"
"github.com/prometheus/client_golang/prometheus"
)

// setupHystrix sets up the hystrix metrics
// This is a workaround for hystrix-go not supporting the prometheus registry
func setupHystrix() {
promC := hystrixprometheus.NewPrometheusCollector("hystrix", nil, prometheus.DefBuckets)
metricCollector.Registry.Register(promC.Collector)
}
cb := circuitbreaker.NewBuilder[any]().
WithFailureThreshold(5).
WithDelay(5 * time.Second).
OnStateChanged(func(e circuitbreaker.StateChangedEvent) {
circuitStateGauge.WithLabelValues(commandName).Set(float64(e.NewState))
}).
Build()
Comment thread
ankurs marked this conversation as resolved.
```

{: .note .note-info }
If you are using the `go-coldbrew/core` package, you can skip the above step as it will automatically register the collector for you.
See [Hystrix Prometheus] for more details.
ColdBrew's built-in `grpc_client_*` Prometheus metrics (request count, duration, status codes) cover request-level observability automatically. See the [integrations page](/integrations/#circuit-breaker--resilience) for full executor setup examples.

{: .warning }
**Legacy:** The previous `hystrixprometheus` package and `core.SetupHystrixPrometheus()` are deprecated. Circuit breaker metrics are now managed by your chosen resilience library, not ColdBrew.

---
[HTTP port]: https://pkg.go.dev/github.com/go-coldbrew/core/config#readme-type-config
Expand Down
85 changes: 73 additions & 12 deletions integrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,28 +231,89 @@ func main() {
}
```

## Hystrix-Go
## Circuit Breaker / Resilience

{: .warning }
Hystrix-Go (`afex/hystrix-go`) is unmaintained (last updated 2018). Consider migrating to [failsafe-go] for circuit breaker functionality.
ColdBrew provides a pluggable executor hook for client-side resilience (circuit breaking, retries, bulkheading, etc.). You bring your own resilience library — [failsafe-go] is recommended.

[Hystrix-Go] is a Go implementation of the circuit breaker pattern. ColdBrew provides Prometheus metrics integration for Hystrix.
### How it works

### Initialising
Call `interceptors.SetDefaultExecutor` during init to wrap all outbound gRPC calls with your resilience logic. The executor receives the gRPC method name, enabling per-method circuit breakers.

If your app is using [ColdBrew cookiecutter] template, initialisation is done automatically.
```go
import (
"github.com/failsafe-go/failsafe-go"
"github.com/failsafe-go/failsafe-go/circuitbreaker"
"github.com/go-coldbrew/interceptors"
)

func init() {
cb := circuitbreaker.NewBuilder[any]().
WithFailureThreshold(5).
WithDelay(5 * time.Second).
WithSuccessThreshold(2).
Comment thread
ankurs marked this conversation as resolved.
Build()

interceptors.SetDefaultExecutor(func(ctx context.Context, method string, fn func(ctx context.Context) error) error {
return failsafe.With[any](cb).WithContext(ctx).Run(func() error {
return fn(ctx)
})
})
Comment thread
ankurs marked this conversation as resolved.
}
Comment thread
ankurs marked this conversation as resolved.
```

If you are using ColdBrew packages in your app, you need to initialise Hystrix Prometheus manually:
### Per-method circuit breakers

```go
import "github.com/go-coldbrew/core"
For independent circuit breakers per gRPC method (so failures in one method don't trip another):

func main() {
// SetupHystrixPrometheus registers Hystrix metrics with Prometheus
core.SetupHystrixPrometheus()
```go
func init() {
var (
mu sync.Mutex
breakers = make(map[string]circuitbreaker.CircuitBreaker[any])
)
Comment thread
ankurs marked this conversation as resolved.

interceptors.SetDefaultExecutor(func(ctx context.Context, method string, fn func(ctx context.Context) error) error {
mu.Lock()
cb, ok := breakers[method]
if !ok {
cb = circuitbreaker.NewBuilder[any]().
WithFailureThreshold(5).
WithDelay(5 * time.Second).
Build()
breakers[method] = cb
}
mu.Unlock()

return failsafe.With[any](cb).WithContext(ctx).Run(func() error {
return fn(ctx)
})
Comment thread
ankurs marked this conversation as resolved.
})
}
```

### Disabling for specific connections

Use `WithoutExecutor()` to skip resilience for internal or loopback connections:

```go
interceptors.DefaultClientInterceptor(
interceptors.WithoutExecutor(),
)
Comment thread
ankurs marked this conversation as resolved.
```

### Excluded errors

Use `WithExcludedErrors` or `WithExcludedCodes` to prevent expected errors from tripping the circuit breaker:

```go
Comment thread
ankurs marked this conversation as resolved.
interceptors.WithExcludedCodes(codes.NotFound, codes.InvalidArgument)
```
Comment thread
ankurs marked this conversation as resolved.

See the [interceptors examples](https://github.com/go-coldbrew/interceptors/tree/main/examples) for more patterns including circuit breaker + bulkhead composition.

{: .warning }
**Legacy:** The previous `HystrixClientInterceptor` (wrapping `afex/hystrix-go`) is deprecated and will be removed in v1. Migrate to `SetDefaultExecutor` with [failsafe-go].

Comment thread
ankurs marked this conversation as resolved.
Comment thread
ankurs marked this conversation as resolved.
## Environment Configuration

ColdBrew provides functions to configure the environment and release information used by monitoring tools.
Expand Down
Loading