Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CapMan visibility] Separates rejecting policy and throttling policy in Sentry tags and spans #74733

Merged
merged 1 commit into from
Jul 23, 2024

Conversation

xurui-c
Copy link
Member

@xurui-c xurui-c commented Jul 23, 2024

It's possible for a query to be in the throttling zone for an allocation policy in Snuba, and also in another allocation policy's rejection zone. In which case we see the following:

Screenshot 2024-07-23 at 9 33 17 AM

How could the throttle_threshold be higher than the rejection_threshold? This is due to the query being throttled by an allocation policy with a throttle threshold of 292000000000, but it's also rejected by ConcurrentRateLimitAllocationPolicy The mismatch is caused by not separating out the rejection policy and the throttle policy when creating Sentry tags and spans.

@xurui-c xurui-c requested review from a team as code owners July 23, 2024 16:34
Copy link

sentry-io bot commented Jul 23, 2024

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: src/sentry/utils/snuba.py

Function Unhandled Issue
_bulk_snuba_query RateLimitExceeded: Query on could not be run due to allocation policies, info: {'details': {'ConcurrentRateLimitAllo... ...
Event Count: 798
_bulk_snuba_query RateLimitExceeded: Query on could not be run due to allocation policies, info: {'details': {'ConcurrentRateLimitAllo... ...
Event Count: 225
_bulk_snuba_query RateLimitExceeded: Query on could not be run due to allocation policies, info: {'details': {'ReferrerGuardRailPolicy... ...
Event Count: 118
_bulk_snuba_query RateLimitExceeded: Query on could not be run due to allocation policies, info: {'details': {'ConcurrentRateLimitAllo... ...
Event Count: 36

Did you find this useful? React with a 👍 or 👎

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Jul 23, 2024
@xurui-c xurui-c enabled auto-merge (squash) July 23, 2024 16:44
@xurui-c xurui-c merged commit 7f16895 into master Jul 23, 2024
50 checks passed
@xurui-c xurui-c deleted the rachel/fixThresholds branch July 23, 2024 17:08
@github-actions github-actions bot locked and limited conversation to collaborators Aug 8, 2024
@xurui-c xurui-c changed the title Separates rejecting policy and throttling policy in Sentry tags and spans [CapMan visibility] Separates rejecting policy and throttling policy in Sentry tags and spans Nov 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants