Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ratelimit service is not working properly, misbehaving while flushing #821

Open
mks27 opened this issue Jan 23, 2025 · 0 comments
Open

Ratelimit service is not working properly, misbehaving while flushing #821

mks27 opened this issue Jan 23, 2025 · 0 comments

Comments

@mks27
Copy link

mks27 commented Jan 23, 2025

Hi Team, I have implemented Istio global rate-limiting using the rate-limit service and redis service provided by envoy.
I have observed following issues in the rate-limit service.

  1. when the rate-limit service are flushing counters and gauge, my requests are crossing rate-limiting threshold but they are not getting 429. These requests are also not reported in rate-limit service logs. after some requests ,ratelimiting starts working properly.
  2. when the rate-limit threshold has been reached for the requested URL, getting 429 but some hits are reaching to the respected service and not giving 429.

I am using below images -
ratelimit - envoyproxy/ratelimit:4c088856
redis - redis:6.2.11-alpine

This is my ratelimit-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ratelimit
  namespace: istio-iut
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ratelimit
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: ratelimit
    spec:
      containers:
      - image: envoyproxy/ratelimit:4c088856
        imagePullPolicy: Always
        name: ratelimit
        command: ["/bin/ratelimit"]
        env:
        - name: LOG_LEVEL
          value: debug
        - name: REDIS_SOCKET_TYPE
          value: tcp
        - name: REDIS_URL
          value: redis:6379
        - name: USE_STATSD
          value: "false"
        - name: RUNTIME_ROOT
          value: /data
        - name: RUNTIME_SUBDIRECTORY
          value: ratelimit
        - name: RUNTIME_WATCH_ROOT
          value: "false"
        - name: RUNTIME_IGNOREDOTFILES
          value: "true"
        - name: MERGE_DOMAIN_CONFIG
          value: "true"
        - name: HOST
          value: "::"
        - name: GRPC_HOST
          value: "::"
        ports:
        - containerPort: 8080
        - containerPort: 8081
        - containerPort: 6070
        volumeMounts:
        - name: config-volume
          mountPath: /data/ratelimit/config
      volumes:
      - name: config-volume
        projected:
          sources:
          - configMap:
              name: ratelimit-config-api1
          - configMap:
              name: ratelimit-config-api2

This is my redis-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: istio-iut
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - image: redis:6.2.11-alpine
        imagePullPolicy: Always
        name: redis
        ports:
        - name: redis
          containerPort: 6379
      restartPolicy: Always
      serviceAccountName: ""

Please let me know , what is the reason behind this behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant