Skip to content

feat: routes v2 ratelimit#2891

Merged
chronark merged 21 commits intomainfrom
routes_v2_ratelimit
Feb 18, 2025
Merged

feat: routes v2 ratelimit#2891
chronark merged 21 commits intomainfrom
routes_v2_ratelimit

Conversation

@chronark
Copy link
Collaborator

@chronark chronark commented Feb 13, 2025

  • feat: tracing middleware
  • wip

Summary by CodeRabbit

  • New Features

    • Introduced a local API testing job that streamlines automated testing.
    • Extended the rate limiting API with namespace support and enhanced error details.
    • Added refined role-based access control options to improve permission management.
    • Added a new document outlining the specifications and configuration details for the Unkey API.
    • Implemented a Redis-based discovery mechanism for improved node state management.
  • Improvements

    • Enhanced error responses for bad requests, providing clearer feedback.
    • Upgraded API server initialization and authentication handling for better performance.
    • Improved validation logic for OpenAPI documents to ensure compliance.
    • Modified database connection handling for improved configurability.
    • Simplified component usage in several UI pages by removing unnecessary class attributes.
    • Updated configuration management to replace AWS Cloud Map with Redis for cluster discovery.
    • Adjusted database operations to use parameterized arguments for timestamps.
  • Tests

    • Expanded test coverage for rate limit override functionality to ensure robustness.
    • Added new tests for validating the creation and error handling of rate limit overrides.
    • Introduced simulation tests for a caching mechanism to validate behavior under various conditions.

@changeset-bot
Copy link

changeset-bot bot commented Feb 13, 2025

⚠️ No Changeset found

Latest commit: f3a1665

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 13, 2025

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

This pull request introduces extensive changes across multiple components. Updates include modifications to CI workflows, API configurations and routes, database queries, caching libraries, key and identity services, and error handling. New functionality is added for rate-limiting overrides, RBAC, and simulation testing, while legacy and unused test files are removed. Additionally, dependencies have been updated and several service interfaces and data transforms have been refactored to improve clarity, consistency, and error management.

Changes

File(s) Change Summary
.github/workflows/job_test_go_api_local.yaml, .github/workflows/pr.yaml Modified CI workflows: simplified test command from go test to task test and activated the local Go API testing job by uncommenting previously inactive sections.
go/.golangci.yaml, go/Taskfile.yml Adjusted linter configurations (formatting, enabled/disabled linters) and added a command to pull the latest MySQL Docker image in the test task.
go/api/config.yaml, go/api/gen.go, go/api/openapi.json Removed output-options; added Namespace field in rate limit request; updated OpenAPI schemas with enhanced error responses and validation rules.
go/cmd/api/main.go, go/cmd/api/routes/register.go, go/cmd/api/routes/services.go, route handlers (e.g. v2_ratelimit_limit/handler.go, v2_ratelimit_set_override/handler.go) Refactored API server initialization (added clock service and setupCluster), updated route registration with tracing and new rate limit endpoints, and extended service structs to include new dependencies (e.g. Ratelimit service).
go/cmd/api/routes/v2_ratelimit_set_override/200_test.go, .../400_test.go, (removed happy_test.go) Added tests for successful creation and error handling in rate limit override endpoints; removed deprecated test file.
go/go.mod Added new dependencies: github.com/panjf2000/ants v1.3.0 for improved goroutine pool management and github.com/redis/go-redis/v9 v9.6.1 for Redis integration.
go/internal/services/keys/*.go Updated key service: added VerifyRootKey method to the interface and service; removed caching functionality; modified Verify signature to use SHA-256 on the raw key.
go/pkg/cache/* Refactored caching: introduced generic type parameters <K, V> in cache types, interfaces, middleware, and tests; removed unused cache utility.
go/pkg/database/*, go/pkg/database/queries/* Enhanced DSN validation; added new methods and queries for keys, keyrings, permissions, and rate limit overrides; updated transformation functions for key and identity models.
go/pkg/entities/key.go, go/pkg/entities/keyring.go, go/pkg/fault/tag.go Modified Key entity (renamed and removed fields, added RemainingRequests); introduced new Keyring entity and a new fault tag BAD_REQUEST.
go/pkg/rbac/* Introduced RBAC package: defined action types, tuple structures, a query builder, and an evaluation mechanism for permission checks; added corresponding tests.
go/pkg/sim/*, go/pkg/testutil/* Added a simulation framework (events, RNG, seed, and simulation orchestration) and updated test utilities for MySQL and HTTP testing with resource seeding.
go/pkg/zen/* Improved authorization by enhancing token parsing in Bearer; added middleware for error handling and tracing; refactored session initialization and validation with new context handling.

Suggested reviewers

  • MichaelUnkey
  • mcstepp
  • perkinsjr

Possibly related PRs

  • feat: routes v2 ratelimit #2891: The changes in the main PR, which involve modifications to the GitHub Actions workflow for testing a Go API, are directly related to the changes in the retrieved PR, as both PRs modify the same workflow file (.github/workflows/job_test_go_api_local.yaml) to alter the test execution command.
  • ci: disable go test #2865: The changes in the main PR, which modify the testing command in the GitHub Actions workflow for a Go API, are related to the retrieved PR that disables the Go test functionality in the CI setup, as both involve alterations to the same workflow files for testing the Go API.
✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 13, 2025

Thank you for following the naming conventions for pull request titles! 🙏

@vercel
Copy link

vercel bot commented Feb 13, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
engineering ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 18, 2025 5:57pm
play ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 18, 2025 5:57pm
www ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 18, 2025 5:57pm
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
dashboard ⬜️ Ignored (Inspect) Visit Preview Feb 18, 2025 5:57pm

@chronark chronark marked this pull request as ready for review February 17, 2025 15:29
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
go/pkg/database/key_insert.go (1)

19-23: ⚠️ Potential issue

Potential data loss in int32 conversion.

The conversion of RemainingRequests to int32 could lead to data loss for large values.

🧹 Nitpick comments (14)
go/pkg/database/workspace_insert.go (1)

18-21: LGTM! Good use of sql.NullTime and clock interface.

The implementation correctly uses sql.NullTime for database compatibility and db.clock for time operations, which is great for testing. The Valid flag is appropriately set.

Consider adding a comment explaining why CreatedAt is using NullTime, as it might not be immediately obvious to other developers why this field needs to be nullable.

go/pkg/database/queries/workspace_insert.sql (1)

18-22: Consider parameterizing default values.

The default values for plan, beta_features, features, enabled, and delete_protection are hardcoded in the SQL query. Consider making these configurable parameters for better flexibility.

Example modification:

-    'free',
-    '{}',
-    '{}',
-    true,
-    true
+    sqlc.arg(plan),
+    sqlc.arg(beta_features),
+    sqlc.arg(features),
+    sqlc.arg(enabled),
+    sqlc.arg(delete_protection)
go/pkg/testutil/containers.go (2)

51-58: Consider using a conditional logger for debugging.

While using NopLogger is fine for production, consider making the logger configurable to help with debugging during development.

-	cfg.Logger = &mysql.NopLogger{}
+	if testing.Verbose() {
+		cfg.Logger = &mysql.StdLogger{}
+	} else {
+		cfg.Logger = &mysql.NopLogger{}
+	}

63-72: Improve error variable naming.

While using err2 and err3 avoids shadowing, consider using more descriptive names that indicate the error's context.

-		connector, err2 := mysql.NewConnector(cfg)
-		if err2 != nil {
-			return fmt.Errorf("unable to create mysql connector: %w", err2)
+		connector, connErr := mysql.NewConnector(cfg)
+		if connErr != nil {
+			return fmt.Errorf("unable to create mysql connector: %w", connErr)
		}

		db = sql.OpenDB(connector)
-		err3 := db.Ping()
-		if err3 != nil {
-			return fmt.Errorf("unable to ping mysql: %w", err3)
+		pingErr := db.Ping()
+		if pingErr != nil {
+			return fmt.Errorf("unable to ping mysql: %w", pingErr)
		}
go/pkg/testutil/http.go (3)

93-99: Consider documenting the commented metrics middleware.

The middleware stack includes a commented-out metrics middleware. Consider adding a comment explaining why it's disabled or when it should be enabled.

 middleware: []zen.Middleware{
   zen.WithTracing(),
-  //	zen.WithMetrics(svc.EventBuffer)
+  // TODO: Enable metrics middleware once EventBuffer is properly initialized
   zen.WithLogging(logger),
   zen.WithErrorHandling(),
   zen.WithValidation(validator),
 },

121-183: Use h.Clock for deterministic testing.

Replace time.Now() calls with h.Clock.Now() to ensure deterministic test behavior. This makes tests more reliable and easier to debug.

 rootWorkspace := entities.Workspace{
   ID:                   uid.New("test_ws"),
   TenantID:             "unkey",
   Name:                 "unkey",
-  CreatedAt:            time.Now(),
+  CreatedAt:            h.Clock.Now(),
   // ... other fields ...
 }

 rootKeyring := entities.Keyring{
   // ... other fields ...
-  CreatedAt:          time.Now(),
+  CreatedAt:          h.Clock.Now(),
   // ... other fields ...
 }

 userWorkspace := entities.Workspace{
   // ... other fields ...
-  CreatedAt:            time.Now(),
+  CreatedAt:            h.Clock.Now(),
   // ... other fields ...
 }

185-212: Improve test key creation for better reliability.

  1. Use h.Clock.Now() for deterministic testing.
  2. Use different IDs for the key's ID and the key itself to better reflect real-world scenarios.
 func (h *Harness) CreateRootKey() string {
-  key := uid.New("test_root_key")
+  key := uid.New("test_key_value")
 
   err := h.DB.InsertKey(context.Background(), entities.Key{
     ID:                uid.New("test_root_key"),
     Hash:              hash.Sha256(key),
     // ... other fields ...
-    CreatedAt:         time.Now(),
+    CreatedAt:         h.Clock.Now(),
     // ... other fields ...
   })
   require.NoError(h.t, err)
 
   return key
 }
go/pkg/database/keyring_insert.go (2)

34-36: Improve error description.

The second parameter in WithDesc is empty. Consider adding more detailed error context.

-		return fault.Wrap(err, fault.WithDesc("failed to insert key ring", ""))
+		return fault.Wrap(err, fault.WithDesc("failed to insert key ring", "error occurred while executing InsertKeyring query"))

12-39: Consider adding input validation.

The function could benefit from validating critical fields before proceeding with the database operation.

Consider adding validation like this at the start of the function:

func (db *database) InsertKeyring(ctx context.Context, keyring entities.Keyring) error {
	if keyring.ID == "" {
		return fault.New("invalid keyring ID")
	}
	if keyring.WorkspaceID == "" {
		return fault.New("invalid workspace ID")
	}
	// ... rest of the function

Additionally, consider adding a reasonable context timeout if not already handled by the caller:

	ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
	defer cancel()
go/pkg/database/transform/key.go (1)

14-32: Consider adding validation for required fields.

Consider adding validation for required fields like ID, KeyringID, and WorkspaceID to ensure they are not empty before proceeding with the transformation.

 func KeyModelToEntity(m gen.Key) (entities.Key, error) {
+	if m.ID == "" {
+		return entities.Key{}, fmt.Errorf("id is required")
+	}
+	if m.KeyAuthID == "" {
+		return entities.Key{}, fmt.Errorf("keyring_id is required")
+	}
+	if m.WorkspaceID == "" {
+		return entities.Key{}, fmt.Errorf("workspace_id is required")
+	}
+
 	key := entities.Key{
go/pkg/database/key_insert.go (1)

31-79: Consider validating required fields before database insertion.

Add validation for required fields to ensure data integrity:

 func (db *database) InsertKey(ctx context.Context, key entities.Key) error {
+	if key.ID == "" {
+		return fault.New("id is required")
+	}
+	if key.KeyringID == "" {
+		return fault.New("keyring_id is required")
+	}
+	if key.WorkspaceID == "" {
+		return fault.New("workspace_id is required")
+	}
+	if key.Hash == "" {
+		return fault.New("hash is required")
+	}
+
 	meta, err := json.Marshal(key.Meta)
go/cmd/api/main.go (3)

109-114: Reuse the existing clock instance.

A new clock instance is created for the database configuration while one already exists (line 68). Consider reusing the existing instance for consistency.

 db, err := database.New(database.Config{
   PrimaryDSN:  cfg.Database.Primary,
   ReadOnlyDSN: cfg.Database.ReadonlyReplica,
   Logger:      logger,
-  Clock:       clock.New(),
+  Clock:       clk,
 }, dbCache.WithCaching(logger))

164-171: Add shutdown handling for the rate limit service.

Consider adding proper cleanup for the rate limit service during shutdown, similar to how the cluster shutdown is handled.

 rlSvc, err := ratelimit.New(ratelimit.Config{
   Logger:  logger,
   Cluster: c,
   Clock:   clk,
 })
 if err != nil {
   return fmt.Errorf("unable to create ratelimit service: %w", err)
 }
+defer func() {
+  if err := rlSvc.Shutdown(ctx); err != nil {
+    logger.Error(ctx, "error shutting down rate limit service", slog.String("error", err.Error()))
+  }
+}()

246-247: Implement AWS Cloudmap discovery method.

The AWS Cloudmap discovery method is marked as not implemented. Consider creating a tracking issue for this implementation.

Would you like me to create an issue to track the implementation of AWS Cloudmap discovery?

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1580c14 and c79542e.

⛔ Files ignored due to path filters (10)
  • go/pkg/database/gen/key_insert.sql.go is excluded by !**/gen/**
  • go/pkg/database/gen/keyring_insert.sql.go is excluded by !**/gen/**
  • go/pkg/database/gen/querier.go is excluded by !**/gen/**
  • go/pkg/database/gen/ratelimit_namespace_delete.sql.go is excluded by !**/gen/**
  • go/pkg/database/gen/ratelimit_namespace_insert.sql.go is excluded by !**/gen/**
  • go/pkg/database/gen/ratelimit_override_delete.sql.go is excluded by !**/gen/**
  • go/pkg/database/gen/ratelimit_override_insert.sql.go is excluded by !**/gen/**
  • go/pkg/database/gen/ratelimit_override_update.sql.go is excluded by !**/gen/**
  • go/pkg/database/gen/workspace_insert.sql.go is excluded by !**/gen/**
  • go/pkg/database/gen/workspace_soft_delete.sql.go is excluded by !**/gen/**
📒 Files selected for processing (28)
  • go/api/gen.go (2 hunks)
  • go/api/openapi.json (4 hunks)
  • go/cmd/api/main.go (6 hunks)
  • go/cmd/api/routes/v2_ratelimit_set_override/400_test.go (1 hunks)
  • go/pkg/database/database.go (5 hunks)
  • go/pkg/database/key_insert.go (1 hunks)
  • go/pkg/database/keyring_insert.go (1 hunks)
  • go/pkg/database/queries/key_insert.sql (1 hunks)
  • go/pkg/database/queries/keyring_insert.sql (1 hunks)
  • go/pkg/database/queries/ratelimit_namespace_delete.sql (1 hunks)
  • go/pkg/database/queries/ratelimit_namespace_insert.sql (1 hunks)
  • go/pkg/database/queries/ratelimit_override_delete.sql (1 hunks)
  • go/pkg/database/queries/ratelimit_override_insert.sql (2 hunks)
  • go/pkg/database/queries/ratelimit_override_update.sql (1 hunks)
  • go/pkg/database/queries/workspace_insert.sql (1 hunks)
  • go/pkg/database/queries/workspace_soft_delete.sql (1 hunks)
  • go/pkg/database/ratelimit_namespace_delete.go (1 hunks)
  • go/pkg/database/ratelimit_override_delete.go (1 hunks)
  • go/pkg/database/ratelimit_override_insert.go (1 hunks)
  • go/pkg/database/ratelimit_override_update.go (1 hunks)
  • go/pkg/database/transform/identity.go (1 hunks)
  • go/pkg/database/transform/key.go (1 hunks)
  • go/pkg/database/transform/ratelimit_namespace.go (1 hunks)
  • go/pkg/database/workspace_delete.go (2 hunks)
  • go/pkg/database/workspace_insert.go (2 hunks)
  • go/pkg/fault/tag.go (1 hunks)
  • go/pkg/testutil/containers.go (3 hunks)
  • go/pkg/testutil/http.go (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (10)
  • go/pkg/database/ratelimit_override_insert.go
  • go/pkg/fault/tag.go
  • go/pkg/database/queries/ratelimit_override_insert.sql
  • go/pkg/database/queries/keyring_insert.sql
  • go/pkg/database/queries/key_insert.sql
  • go/api/gen.go
  • go/pkg/database/database.go
  • go/pkg/database/queries/ratelimit_namespace_insert.sql
  • go/api/openapi.json
  • go/cmd/api/routes/v2_ratelimit_set_override/400_test.go
⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: Test Packages / Test ./internal/clickhouse
  • GitHub Check: Test Go API Local / test_agent_local
  • GitHub Check: Test API / API Test Local
  • GitHub Check: Test Agent Local / test_agent_local
  • GitHub Check: Build / Build
🔇 Additional comments (27)
go/pkg/database/queries/workspace_insert.sql (1)

17-17: LGTM! Consistent timestamp handling.

The change to use sqlc.arg(created_at) aligns well with the Go code changes and provides better control over timestamp values.

go/pkg/testutil/containers.go (3)

9-9: LGTM!

The explicit import of the MySQL driver with an alias enhances code clarity.


77-79: LGTM!

The cleanup function is well-implemented, ensuring proper resource cleanup after tests.


95-95: LGTM!

Using FormatDSN() is the correct approach for generating a properly formatted DSN string.

go/pkg/testutil/http.go (4)

25-29: LGTM! Well-structured Resources type.

The new Resources struct provides a clean way to organize test resources, making them easily accessible throughout tests.


31-46: LGTM! Enhanced Harness structure.

The updated Harness struct provides a comprehensive test environment with all necessary dependencies and services. The addition of middleware, validator, and resources fields enables more thorough testing scenarios.


106-119: LGTM! Flexible middleware configuration.

The updated Register method provides a clean way to override default middleware while maintaining backward compatibility.


223-272: LGTM! Improved response handling.

The updated TestResponse struct and CallRoute function provide better error handling and response inspection capabilities, making tests more robust and easier to debug.

go/pkg/database/keyring_insert.go (2)

1-11: LGTM! Well-organized imports.

The package declaration and import organization follow good practices, with standard library imports separated from project imports.


22-24:

Details

✅ Verification successful

Verify int32 conversion safety.

Converting DefaultBytes to int32 could lead to data truncation for large values. Consider using int64 or adding range validation.


🏁 Script executed:

#!/bin/bash
# Search for DefaultBytes usage to understand the expected range
rg "DefaultBytes" -A 5 -B 5

Length of output: 11137


Int32 Conversion Verification Complete

The conversion from keyring.DefaultBytes to int32 is consistent with the overall design. Our investigation shows that:

  • The keyring.DefaultBytes field in the entity is defined as an int, and typical usage (e.g., in tests it’s set to 16) indicates that the value will be small.
  • The corresponding database model expects a value of type sql.NullInt32, making the cast appropriate.
  • The existing // nolint:gosec suppression appears to acknowledge the potential issue, but given the practical range of values, the risk of truncation is minimal.

If there is a future possibility of handling larger integers, consider adding explicit range validation before conversion. For now, the conversion is safe and aligned with the expected schema.

go/pkg/database/transform/identity.go (2)

25-28: LGTM!

The meta data unmarshalling is well-implemented with proper error handling and initialization.


30-32: LGTM!

The UpdatedAt handling correctly checks for null values and performs proper time conversion.

go/pkg/database/ratelimit_override_delete.go (1)

12-18: LGTM! Enhanced timestamp handling for soft deletes.

The addition of the Now field in DeleteRatelimitOverrideParams improves the soft deletion mechanism by using a parameterized timestamp instead of relying on database-level NOW() function. This change provides better control and testability of the deletion timestamp.

go/pkg/database/ratelimit_namespace_delete.go (1)

12-18: LGTM! Consistent timestamp handling across rate limit operations.

The changes maintain consistency with the rate limit override deletion by using the same parameterized timestamp pattern. This uniformity in handling soft deletes across different rate limit entities is a good practice.

go/pkg/database/transform/ratelimit_namespace.go (1)

36-36: LGTM! Preserving creation timestamps during entity transformation.

The addition of the CreatedAt field ensures that the original creation timestamp is preserved when transforming the entity for database operations. This is crucial for maintaining data integrity and audit trails.

go/pkg/database/ratelimit_override_update.go (2)

16-19: LGTM! Consistent timestamp handling for updates.

The addition of the Now field maintains consistency with other rate limit operations and provides better control over update timestamps.


21-21:

Details

❓ Verification inconclusive

Verify the security implications of the nolint directive.

The line is marked with nolint:gosec to suppress security warnings for the duration conversion. Please verify that this is intentional and document why it's safe to ignore the security warning.

Run the following script to check for similar patterns and any documentation:


🏁 Script executed:

#!/bin/bash
# Search for similar nolint:gosec directives and any related comments
rg -A 3 "nolint:gosec.*Duration" 

# Search for documentation about gosec warnings
rg -i "gosec.*duration|duration.*conversion"

Length of output: 81


Action Required: Confirm and Document Security Rationale for nolint Directive

The // nolint:gosec suppression on the duration conversion (in file go/pkg/database/ratelimit_override_update.go at line 21) did not reveal any similar patterns or accompanying documentation elsewhere in the repository. Please manually verify that converting e.Duration.Milliseconds() to an int32 does not introduce any security risks—such as potential overflow issues—and that suppressing the gosec warning is indeed safe in this context. If the use of nolint:gosec is intentional and safe, add an inline comment explaining the reasoning for future maintainers.

go/pkg/database/transform/key.go (1)

12-32: LGTM! Clean transformation implementation.

The function properly initializes all fields with appropriate default values and handles the transformation from model to entity cleanly.

go/pkg/database/workspace_delete.go (1)

46-52: LGTM! Well-structured soft delete implementation.

Good use of structured parameters and proper timestamp handling with sql.NullTime. The implementation maintains transaction safety and delete protection checks.

go/pkg/database/queries/ratelimit_namespace_delete.sql (1)

1-4: LGTM! Good use of parameterized timestamp.

The change from NOW() to sqlc.arg(now) improves testability and ensures timestamp consistency across the application.

go/pkg/database/queries/ratelimit_override_delete.sql (1)

1-6: Consistent Parameterized Timestamp for Deletion Queries
Switching from the direct SQL call NOW() to the parameterized sqlc.arg(now) improves consistency and testability by deferring the current timestamp production to your application logic (typically sourced via db.clock.Now()). Please confirm that the calling code provides the intended timestamp value.

go/pkg/database/queries/workspace_soft_delete.sql (1)

1-6: Unified Soft Deletion Timestamp Handling
The update replaces NOW() with sqlc.arg(now) for setting the deleted_at column, which aligns with similar changes elsewhere in the project. This approach facilitates better control over timestamps across different environments and easier unit testing. Just ensure that the supplied now parameter is consistent with your application’s current time logic.

go/pkg/database/queries/ratelimit_override_update.sql (1)

1-9: Standardized Update Query with Parameterized Timestamps
By using sqlc.arg(now) for the updated_at field instead of the direct call NOW(), this update ensures that timestamp values are controlled externally and remain consistent across your SQL queries. Please verify that all relevant update operations pass the correct current time value to the query parameters.

go/cmd/api/main.go (4)

17-19: LGTM! Import changes align with new functionality.

The new imports support the rate limiting feature and database caching middleware.

Also applies to: 23-23


79-84: LGTM! Improved node ID handling logic.

The changes provide better control over node ID assignment while maintaining backward compatibility.


121-130: LGTM! Improved cluster management with proper cleanup.

The changes modularize cluster setup and ensure proper cleanup through deferred shutdown.


211-211:

Details

❓ Verification inconclusive

Verify empty IP address in membership configuration.

Using an empty IP address (net.ParseIP("")) in the membership configuration might cause issues. Consider using a loopback address or the actual interface IP.


🏁 Script executed:

#!/bin/bash
# Description: Check if empty IP is handled correctly in the membership package

# Search for IP address validation in the membership package
rg -A 5 'func.*New.*membership.Config' pkg/membership/
rg -A 5 'ParseIP|net.IP' pkg/membership/

Length of output: 211


Empty IP Address in Membership Configuration Requires Manual Verification

The use of net.ParseIP("") for setting the membership configuration’s address is concerning as it may lead to subtle networking issues if a valid IP isn’t provided. Our automated search for the membership package (using pkg/membership/) didn’t yield any results, which suggests that the relevant validation logic isn’t located in that directory or may be elsewhere. Please double-check that:

  • The membership initialization code correctly handles an empty IP address.
  • The intended behavior (whether a loopback address or an actual interface IP should be used) aligns with the configuration and deployment requirements.
  • Any implicit reliance on the empty IP isn’t masking a missing or incomplete configuration elsewhere in the codebase.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (8)
apps/engineering/app/architecture/[[...slug]]/page.tsx (1)

39-41: Consider documenting the styling strategy.

The systematic removal of className props from both DocsDescription and DocsBody suggests a move towards centralized styling. Consider:

  1. Documenting the new styling approach in the PR description
  2. Updating any style guides or component documentation

Since this appears to be part of a larger styling refactor:

#!/bin/bash
# Find other instances of these components to ensure consistent styling
rg "DocsDescription|DocsBody" --type tsx
go/pkg/discovery/redis.go (2)

66-83: Ticker-driven heartbeat approach is fine, but consider a context-based shutdown.
Using shutdownC is effective, though adopting a dedicated context.Context could simplify the signaling mechanism. No critical issues, just a future improvement suggestion.


92-100: Use of KEYS may impact performance on large keyspaces.
Consider replacing Keys with SCAN for better scalability if the Redis instance contains many unrelated keys.

go/cmd/api/config.go (1)

23-25: Redis configuration addition looks good.
Optional improvement: offer more flexibility (e.g., consider fields for TTL or intervals) if needed. The minimal struct is otherwise fine.

go/cmd/api/main.go (2)

186-187: Consider confirming HTTP server shutdown.
gracefulShutdown cleans up cluster resources, but ensure srv.Shutdown() is either implicitly or explicitly called if needed.


189-210: Graceful shutdown with error aggregation.
Collecting shutdown errors in a slice is good. You might consider wrapping them in a multi-error struct, but the current approach is acceptable.

apps/engineering/content/architecture/services/api.mdx (2)

9-11: Review Callout Text Tone.
The callout effectively highlights that the document covers v2 of the Unkey API and notes the deprecation of v1. However, the phrase “It was too hard to selfhost anyways” may come off as unprofessional. Consider rephrasing to maintain a more neutral tone (e.g., “due to significant self-hosting challenges”).


61-66: Improve Configuration Description Clarity.
In the sentence:

"You can use ${SOME_NAME} as placeholder in your config file and it will be replaced by the value of the environment variable SOME_NAME."
Consider revising to:
"You can use ${SOME_NAME} as a placeholder in your config file, and it will be replaced by the value of the environment variable SOME_NAME."
This minor grammatical improvement enhances clarity and readability.

🧰 Tools
🪛 LanguageTool

[uncategorized] ~64-~64: You might be missing the article “a” here.
Context: ... binary. You can use ${SOME_NAME} as placeholder in your config file and it will be repl...

(AI_EN_LECTOR_MISSING_DETERMINER_A)


[uncategorized] ~64-~64: Use a comma before ‘and’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...AME}` as placeholder in your config file and it will be replaced by the value of the...

(COMMA_COMPOUND_SENTENCE)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c79542e and b67a47d.

⛔ Files ignored due to path filters (1)
  • go/go.sum is excluded by !**/*.sum
📒 Files selected for processing (11)
  • apps/engineering/app/architecture/[[...slug]]/page.tsx (1 hunks)
  • apps/engineering/app/architecture/layout.tsx (1 hunks)
  • apps/engineering/app/company/[[...slug]]/page.tsx (1 hunks)
  • apps/engineering/app/contributing/[[...slug]]/page.tsx (1 hunks)
  • apps/engineering/app/rfcs/layout.tsx (1 hunks)
  • apps/engineering/content/architecture/services/api.mdx (1 hunks)
  • deployment/docker-compose.yaml (1 hunks)
  • go/cmd/api/config.go (1 hunks)
  • go/cmd/api/main.go (7 hunks)
  • go/go.mod (2 hunks)
  • go/pkg/discovery/redis.go (1 hunks)
✅ Files skipped from review due to trivial changes (3)
  • apps/engineering/app/rfcs/layout.tsx
  • apps/engineering/app/architecture/layout.tsx
  • apps/engineering/app/contributing/[[...slug]]/page.tsx
🧰 Additional context used
🪛 LanguageTool
apps/engineering/content/architecture/services/api.mdx

[uncategorized] ~64-~64: You might be missing the article “a” here.
Context: ... binary. You can use ${SOME_NAME} as placeholder in your config file and it will be repl...

(AI_EN_LECTOR_MISSING_DETERMINER_A)


[uncategorized] ~64-~64: Use a comma before ‘and’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...AME}` as placeholder in your config file and it will be replaced by the value of the...

(COMMA_COMPOUND_SENTENCE)

⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: Test Packages / Test ./internal/clickhouse
  • GitHub Check: Test Go API Local / test_agent_local
  • GitHub Check: Test Agent Local / test_agent_local
  • GitHub Check: Test API / API Test Local
  • GitHub Check: Build / Build
🔇 Additional comments (26)
apps/engineering/app/company/[[...slug]]/page.tsx (1)

40-40:

Details

✅ Verification successful

Verify styling after className removal.

The removal of className="text-sm" from DocsDescription might affect text readability. Please ensure that:

  1. The component has appropriate default styling from fumadocs-ui
  2. The change is consistent with the design system

Let's check the fumadocs-ui documentation and other usages:


🌐 Web query:

What is the default styling for DocsDescription component in fumadocs-ui?

💡 Result:

The default styling for the DocsDescription component in fumadocs-ui inherits from Fumadocs' built-in typography system and theme configurations[1][5][7]. Key aspects include:

Typography & Layout

  • Uses Tailwind CSS classes for text formatting
  • Applies Fumadocs' default sans font family via CSS variables:
    --font-sans: 'Mona Sans', 'Hubot Sans', sans-serif;
  • Inherits text sizing from Fumadocs' type scale (base font size 16px with responsive line heights)[7]

Visual Style

  • Text color uses theme's --fg-muted CSS variable (default: hsl(240 3.8% 46.1%))
  • Margin spacing controlled through layout grid system
  • Responsive width constraints based on container layout[4][6]

To customize styling:

  1. Override CSS variables in your global CSS[7]
  2. Apply custom Tailwind classes directly to the component[1]
  3. Modify the theme preset in tailwind.config.js[3][7]

Example default implementation:

<DocsDescription className="text-[var(--fg-muted)] text-base/relaxed">
  {description}
</DocsDescription>

The component works within Fumadocs' layout system to maintain visual consistency with other documentation elements like DocsTitle and DocsBody[1][6][9].

Citations:


Action: Confirm that removal of className="text-sm" maintains the intended default styling.

After verifying that the DocsDescription component in fumadocs-ui leverages the built-in typography system—which applies a default text style (typically a text-base/relaxed configuration) consistent with Fumadocs’ design—and seeing that similar elements adhere to this default, the removal appears to be in line with our broader styling standard. Please ensure:

  • The text on the company documentation pages remains legible and consistent with other components.
  • The default styling (inherited from Fumadocs' theme and Tailwind CSS configurations) adequately replaces the previously applied text-sm.

If these conditions are met, the update is acceptable.

go/pkg/discovery/redis.go (7)

1-2: Module initialization approved.
No issues found in these package declaration lines.


3-12: Imports look appropriate.
All necessary packages are imported correctly, aligning with the Redis and logging functionalities.


14-24: Struct definition appears consistent.
The Redis struct fields (client, logger, config, etc.) and channel approach for shutdown are straightforward. Consider ensuring any resources are fully released on shutdown, though the current design is acceptable.


26-31: Configuration struct is clear.
Optional enhancement: allow users to configure ttl and heartbeatInterval via this struct, if custom intervals become necessary.


85-87: Key generation looks correct.
Generates a Redis key using the node ID without issues.


88-90: Advertise method looks good.
Setting the Node's address with TTL is straightforward.


102-105: Shutdown logic is handled properly.
Signal the heartbeat goroutine to stop, then remove the node entry. Overall design is sound.

go/cmd/api/main.go (9)

17-17: Imports approved.
All new imports (ratelimit, clock, and dbCache) appear aligned with the new features.

Also applies to: 19-19, 23-23


57-57: Initialization of the shutdowns slice is fine.
Collecting cleanup functions in a slice encourages a clear shutdown sequence.


70-70: Clock usage is appropriate.
Creating a fresh clock instance for time-based services fits well with your design.


82-86: Node ID logic is well-handled.
If cfg.Cluster.NodeID is empty, the system uses a generated ID; otherwise, it respects the configured value.


111-119: Database setup.
The instantiation of the database with caching middleware is coherent, and error handling is correct.


121-121: Database closure.
Deferring db.Close() is a good approach to ensure resource cleanup.


123-128: Cluster setup function.
Invoking setupCluster(...) here simplifies the main run logic.


161-168: Rate limit service initialization.
Proper approach passing logger, cluster, and clock. Error handling on creation is consistent.


212-293: New cluster setup logic looks comprehensive.

  • Lines 243:

    RpcAddr: "TO DO",

    This line has already been flagged in previous comments.

  • net.ParseIP("") is used for membership; verify that it does not cause any unexpected behavior. Using 0.0.0.0 might be more explicit if you aim to bind on all interfaces.

Overall, the function properly supports static or Redis-based discovery, handles membership and returns appropriate cleanup hooks.

deployment/docker-compose.yaml (1)

58-58: Database DSN Update: Verify MySQL Time Parsing Behavior.
The addition of ?parseTime=true to the DATABASE_PRIMARY_DSN ensures that time values are correctly parsed by the MySQL driver. Please double-check that this aligns with the expected behavior for any time-valued columns and that downstream components (e.g., in Go code using this DSN) properly utilize this setting.

apps/engineering/content/architecture/services/api.mdx (4)

1-3: New Frontmatter for API Documentation.
The frontmatter correctly sets the title to "API" and delineates the document boundaries. This provides clear metadata for the document.


4-6: Clear Import Statements for UI Components.
The new imports for Step, Steps, TypeTable, and Property are clear and consistent with the expected component usage in the documentation.


67-71: Well-Structured JSON Example.
The provided JSON configuration example is clear and helps users understand how to set dynamic values using environment variable placeholders.


90-212: Consistent Documentation Structure.
The document’s sections—including Quickstart, Configuration, and various configuration properties—are well-organized and detailed. The usage of custom components (e.g., <Property>) aids clarity. No further issues are noted in these segments.

go/go.mod (4)

21-22: New Dependency: ants Library.
The addition of github.com/panjf2000/ants v1.3.0 introduces a high-performance goroutine pool that can help optimize concurrency patterns. Please ensure that this dependency is compatible with existing concurrency logic and that its usage is documented where applicable.


24-25: New Dependency: go-redis Client.
Adding github.com/redis/go-redis/v9 v9.6.1 supports the new Redis-based discovery mechanism. Verify that the version change is consistent with other Redis integrations in the codebase.


50-51: Indirect Dependency: xxhash Library.
The inclusion of github.com/cespare/xxhash/v2 v2.3.0 as an indirect dependency seems standard. Confirm that this version is required by one of the direct dependencies for optimal performance.


56-57: Indirect Dependency: go-rendezvous Library.
The new indirect dependency github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f is now part of the module. Ensure that any module using it is compatible with this revision and that its functionality aligns with the overall service-discovery enhancements.

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
go/pkg/discovery/redis.go (2)

33-64: Use a more consistent context approach across initialization.

Currently, the ping and initial advertisement use context.Background() directly. Consider passing a caller-provided context to NewRedis for uniform cancellation behavior, especially in cases where the service might need to abort initialization cleanly.


102-105: Recheck address deletion if using multiple contexts within shutdown.

Currently, the Del call is the only operation that uses ctx here; heartbeat and advertise calls are still using background contexts. Ensure that all calls align with the same shutdown context if you want a truly graceful teardown.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b67a47d and e14083d.

📒 Files selected for processing (1)
  • go/pkg/discovery/redis.go (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: Test Packages / Test ./internal/clickhouse
  • GitHub Check: Test Go API Local / test_agent_local
  • GitHub Check: Test Agent Local / test_agent_local
  • GitHub Check: Build / Build
  • GitHub Check: Test API / API Test Local
🔇 Additional comments (2)
go/pkg/discovery/redis.go (2)

66-84: Ensure graceful termination of the ticker goroutine.

The heartbeat goroutine stops when it receives from shutdownC. However, if there’s a delay in reading from the channel during shutdown, the goroutine may continue. Consider using a context-based approach to ensure timely cancellation or verifying that the ticker does not continue beyond shutdown.


85-90: Approved usage of wrapped errors in advertise().

This function correctly wraps the Redis command error with %w, following best practices for error propagation. Great job!

@chronark chronark merged commit 1e49090 into main Feb 18, 2025
23 of 27 checks passed
@chronark chronark deleted the routes_v2_ratelimit branch February 18, 2025 17:54
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
go/pkg/discovery/redis.go (2)

92-119: 🛠️ Refactor suggestion

Avoid using KEYS for production-grade discovery.
Using r.rdb.Keys can degrade performance for large datasets, as it scans the entire keyspace. Switching to SCAN would provide a more efficient and scalable approach.


121-124: 🛠️ Refactor suggestion

Guard against multiple writes to shutdownC.

Repeated calls to Shutdown could attempt multiple writes to shutdownC, causing a panic. Consider closing the channel or using a sync.Once to ensure idempotent shutdown logic.

 func (r *Redis) Shutdown(ctx context.Context) error {
-   r.shutdownC <- struct{}{}
+   select {
+   case <-r.shutdownC:
+       // Already shutting down
+   default:
+       close(r.shutdownC)
+   }
    return r.rdb.Del(ctx, r.key()).Err()
 }
🧹 Nitpick comments (4)
go/pkg/discovery/redis.go (2)

33-64: Recommend passing a shared context.

Within NewRedis, each method call uses a fresh context.Background(). Consider passing in the caller's context to maintain traceability and handle cancellations more gracefully.


66-83: Heartbeat interval may deserve external configuration.

The hardcoded 60-second heartbeat interval might not be optimal for all deployments. Exposing this as a configurable parameter could improve flexibility in different environments.

deployment/docker-compose.yaml (2)

53-55: New Redis Dependency Added to apiv2 Service
The apiv2 service now explicitly depends on the redis service (line 54). This change aligns the service startup order with the new Redis-based discovery mechanism. In a production scenario, consider adding a healthcheck for Redis to ensure it’s ready before apiv2 initiates.


63-67: New Redis Service Block Added
A new Redis service block has been introduced (lines 63–67). While using redis:latest is acceptable during development, it is recommended to pin a specific version for production to avoid unexpected changes with future image updates. Additionally, assess whether a persistent volume is needed based on your data retention requirements.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e14083d and a8ba276.

📒 Files selected for processing (4)
  • deployment/docker-compose.yaml (1 hunks)
  • go/config.docker.json (1 hunks)
  • go/pkg/discovery/redis.go (1 hunks)
  • go/schema.json (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: Test Go API Local / test_agent_local
  • GitHub Check: Test Packages / Test ./internal/clickhouse
  • GitHub Check: Test Agent Local / test_agent_local
  • GitHub Check: Test API / API Test Local
  • GitHub Check: Build / Build
🔇 Additional comments (5)
go/config.docker.json (2)

13-13: Verify HOSTNAME environment variable.

The advertiseAddr field depends on ${HOSTNAME} being set. Ensure that the runtime environment guarantees this variable’s availability.


15-16: Validate REDIS_URL.

The new redis.url must reference a valid Redis endpoint. Confirm that ${REDIS_URL} is defined and contains a valid scheme (e.g., redis://).

go/schema.json (1)

31-43: Redis property aligns with updated discovery approach.

The newly introduced redis property with a required url field is consistent with the Redis-based discovery changes. This update provides a cleaner schema compared to the previous AWS Cloud Map configuration.

deployment/docker-compose.yaml (2)

59-59: Updated MySQL DSN Configuration
The modification of DATABASE_PRIMARY_DSN to include ?parseTime=true (line 59) should help correctly parse time-related fields from MySQL. Please verify that the database driver in use supports this parameter and that existing functionalities depending on the DSN remain unaffected.


61-61: Added Redis Connection URL Environment Variable
The introduction of the REDIS_URL environment variable (line 61) correctly points to the new Redis service. Ensure that all services interacting with Redis are configured to use this URL consistently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant