-
Notifications
You must be signed in to change notification settings - Fork 240
feat: expression support for retry condition #2167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
SkArchon
merged 21 commits into
main
from
dustin/eng-7974-allow-to-configure-retry-condition
Aug 28, 2025
Merged
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
17ca80d
feat: expression support for retry condition
StarpTech 7cb34a3
Merge branch 'main' into dustin/eng-7974-allow-to-configure-retry-con…
StarpTech c61b167
chore: update modules
StarpTech 7d70025
chore: mutation tests
StarpTech 36c93f0
chore: fix tests
StarpTech a394639
chore: dont retry on 429 by default
StarpTech a496eff
fix: staticchecks
SkArchon d39bcee
fix: retry integration test
SkArchon d639aec
fix: review comments
SkArchon 259f539
fix: review comments
SkArchon c5162ac
fix: review comments
SkArchon a7ddd62
fix: return noop instead of nil to be safer
SkArchon 95de8ad
Merge branch 'main' into dustin/eng-7974-allow-to-configure-retry-con…
SkArchon 4a34c7e
fix: tests
SkArchon 6614c42
fix: review comments
SkArchon e7d8ed0
fix: review comments
SkArchon 9a2c9d7
fix: validate algorithm
SkArchon f53e424
fix: tests
SkArchon 80c7cab
Merge branch 'main' into dustin/eng-7974-allow-to-configure-retry-con…
SkArchon 23ac555
fix: tests
SkArchon 8f5d283
fix: cleanup the provided 5xx helper
SkArchon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| package core | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "net/http" | ||
|
|
||
| "github.com/wundergraph/cosmo/router/internal/expr" | ||
| "github.com/wundergraph/cosmo/router/internal/retrytransport" | ||
| "go.uber.org/zap" | ||
| ) | ||
|
SkArchon marked this conversation as resolved.
|
||
|
|
||
| const DefaultRetryExpression = "IsRetryableStatusCode() || IsConnectionError() || IsTimeout()" | ||
|
Noroth marked this conversation as resolved.
Outdated
|
||
|
|
||
| // BuildRetryFunction creates a ShouldRetry function based on the provided expression | ||
| func BuildRetryFunction(expression string, logger *zap.Logger) (retrytransport.ShouldRetryFunc, error) { | ||
| // Use default expression if empty string is passed | ||
| if expression == "" { | ||
| expression = DefaultRetryExpression | ||
| } | ||
|
|
||
| // Create the retry expression manager | ||
| manager, err := expr.NewRetryExpressionManager(expression) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("failed to compile retry expression: %w", err) | ||
|
Noroth marked this conversation as resolved.
Outdated
|
||
| } | ||
|
|
||
| // Return expression-based retry function | ||
| return func(err error, req *http.Request, resp *http.Response) bool { | ||
| // Never retry mutations, regardless of expression result | ||
| if isMutationRequest(req.Context()) { | ||
| return false | ||
| } | ||
|
|
||
| // Create retry context | ||
| ctx := expr.LoadRetryContext(err, resp) | ||
|
|
||
| // Evaluate the expression | ||
| shouldRetry, evalErr := manager.ShouldRetry(ctx) | ||
| if evalErr != nil { | ||
| logger.Error("Failed to evaluate retry expression", | ||
| zap.Error(evalErr), | ||
| zap.String("expression", expression), | ||
| ) | ||
| // Disable retries on evaluation error | ||
| return false | ||
| } | ||
|
|
||
| return shouldRetry | ||
| }, nil | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,154 @@ | ||
| package core | ||
|
|
||
| import ( | ||
| "errors" | ||
| "net/http" | ||
| "syscall" | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/assert" | ||
| "go.uber.org/zap" | ||
| ) | ||
|
|
||
| func TestBuildRetryFunction(t *testing.T) { | ||
| logger := zap.NewNop() | ||
|
|
||
| t.Run("default expression behavior", func(t *testing.T) { | ||
| // Use the default expression that would be in the config | ||
| fn, err := BuildRetryFunction(DefaultRetryExpression, logger) | ||
| assert.NoError(t, err) | ||
| assert.NotNil(t, fn) | ||
|
|
||
| // Test default behavior - should retry on 500 | ||
| req, _ := http.NewRequest("GET", "http://example.com", nil) | ||
| resp := &http.Response{StatusCode: 500} | ||
| assert.True(t, fn(nil, req, resp)) | ||
|
|
||
| // Should not retry on 200 | ||
| resp.StatusCode = 200 | ||
| assert.False(t, fn(nil, req, resp)) | ||
|
|
||
| // Note: Testing mutation behavior would require setting up a proper request context | ||
| // which is beyond the scope of this unit test. The mutation check is tested | ||
| // in integration tests. | ||
|
|
||
| // Test with errors - only expression-defined errors are handled here | ||
| req, _ = http.NewRequest("GET", "http://example.com", nil) | ||
| assert.True(t, fn(syscall.ETIMEDOUT, req, nil)) | ||
| assert.True(t, fn(errors.New("connection refused"), req, nil)) | ||
| assert.False(t, fn(errors.New("unexpected EOF"), req, nil)) // EOF is now handled at transport layer, not expression | ||
| assert.False(t, fn(errors.New("some other error"), req, nil)) | ||
| }) | ||
|
|
||
| t.Run("expression-based retry", func(t *testing.T) { | ||
| expression := "statusCode == 500 || statusCode == 503" | ||
| fn, err := BuildRetryFunction(expression, logger) | ||
| assert.NoError(t, err) | ||
| assert.NotNil(t, fn) | ||
|
|
||
| req, _ := http.NewRequest("GET", "http://example.com", nil) | ||
|
|
||
| // Should retry on 500 | ||
| resp := &http.Response{StatusCode: 500} | ||
| assert.True(t, fn(nil, req, resp)) | ||
|
|
||
| // Should retry on 503 | ||
| resp.StatusCode = 503 | ||
| assert.True(t, fn(nil, req, resp)) | ||
|
|
||
| // Should not retry on 502 | ||
| resp.StatusCode = 502 | ||
| assert.False(t, fn(nil, req, resp)) | ||
| }) | ||
|
|
||
| t.Run("expression with error conditions", func(t *testing.T) { | ||
| expression := "IsTimeout() || statusCode == 503" | ||
| fn, err := BuildRetryFunction(expression, logger) | ||
| assert.NoError(t, err) | ||
| assert.NotNil(t, fn) | ||
|
|
||
| req, _ := http.NewRequest("GET", "http://example.com", nil) | ||
|
|
||
| // Should retry on timeout error | ||
| err = syscall.ETIMEDOUT | ||
| assert.True(t, fn(err, req, nil)) | ||
|
|
||
| // Should retry on 503 | ||
| resp := &http.Response{StatusCode: 503} | ||
| assert.True(t, fn(nil, req, resp)) | ||
|
|
||
| // Should not retry on other errors | ||
| err = errors.New("some other error") | ||
| assert.False(t, fn(err, req, nil)) | ||
| }) | ||
|
|
||
| t.Run("invalid expression returns error", func(t *testing.T) { | ||
| expression := "invalid syntax +++" | ||
| fn, err := BuildRetryFunction(expression, logger) | ||
| assert.Error(t, err) | ||
| assert.Nil(t, fn) | ||
| assert.Contains(t, err.Error(), "failed to compile retry expression") | ||
| }) | ||
|
|
||
| t.Run("empty expression uses default", func(t *testing.T) { | ||
| fn, err := BuildRetryFunction("", logger) | ||
| assert.NoError(t, err) | ||
| assert.NotNil(t, fn) | ||
|
|
||
| // Test with retryable status code | ||
| req, _ := http.NewRequest("GET", "http://example.com", nil) | ||
| resp := &http.Response{StatusCode: 502} | ||
| assert.True(t, fn(nil, req, resp)) | ||
|
|
||
| // Test with connection error | ||
| err = errors.New("connection refused") | ||
| assert.True(t, fn(err, req, nil)) | ||
|
|
||
| // Test with timeout error | ||
| err = syscall.ETIMEDOUT | ||
| assert.True(t, fn(err, req, nil)) | ||
|
|
||
| // Test with non-retryable error | ||
| err = errors.New("some other error") | ||
| assert.False(t, fn(err, req, nil)) | ||
| }) | ||
|
|
||
| t.Run("expression that always returns true", func(t *testing.T) { | ||
| expression := "true" // Always retry | ||
| fn, err := BuildRetryFunction(expression, logger) | ||
| assert.NoError(t, err) | ||
| assert.NotNil(t, fn) | ||
|
|
||
| req, _ := http.NewRequest("GET", "http://example.com", nil) | ||
| resp := &http.Response{StatusCode: 500} | ||
|
|
||
| // Should retry when expression is true | ||
| assert.True(t, fn(nil, req, resp)) | ||
|
|
||
| // Even for status codes that wouldn't normally retry | ||
| resp.StatusCode = 200 | ||
| assert.True(t, fn(nil, req, resp)) | ||
| }) | ||
|
|
||
| t.Run("complex expression", func(t *testing.T) { | ||
| expression := "(statusCode >= 500 && statusCode < 600) || IsConnectionError()" | ||
| fn, err := BuildRetryFunction(expression, logger) | ||
| assert.NoError(t, err) | ||
| assert.NotNil(t, fn) | ||
|
|
||
| req, _ := http.NewRequest("GET", "http://example.com", nil) | ||
|
|
||
| // Test 5xx errors | ||
| resp := &http.Response{StatusCode: 503} | ||
| assert.True(t, fn(nil, req, resp)) | ||
|
|
||
| // Test connection error | ||
| err = errors.New("connection refused") | ||
| assert.True(t, fn(err, req, nil)) | ||
|
|
||
| // Test non-matching conditions | ||
| resp.StatusCode = 404 | ||
| err = errors.New("some other error") | ||
| assert.False(t, fn(err, req, resp)) | ||
| }) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.