Analyzing the design suggested by CS team with pros,cons and effect of edge cases

[rmsamitha]

Analyzing the design suggested by CS team with pros,cons and effect of edge cases

[rmsamitha]

Flow of the design suggested, is presented in a diagram:

[rmsamitha]

Concerns

• Master cannot be a GW node since we don’t recommend hazelcast clustering
• How the Redis server can act as a master - has to figure-out
• If the local counter value was increased in some time window, do we need to maintain that increased counter value to next time units too? Is there a trend to have a higher load to a certain node? This maintenance would add unnecessary complexity.

Pros

• Performance hit will be less. (no synchronous flow is introduced additionally other than the retrieval of the additional batch of tokens)
• Configured BE throttle limit will be universal. (not a per GW node limit)

Cons

• Premature throttling (throttle at lower limit) i.e.
• Above approach has assumed that the load is evenly distributed.
• Load Balancing is not guaranteed to be even. (also in Round-Robin)
• GW1 can consume all its quota and then total of the buffer too.
• Other GWs might not have consumed much of their quota.
• But GW1 will not accept requests further and gets throttled.
• Still the requests are accepted by other GWs in cluster.
• This will add a new problem which is at the cost of solving our original problem. (original: throttle after delay or slip, new: premature throttling)
• This can be a critical issue again since expected request load will not be handled by the GWs.

[rmsamitha]

Completed the task and documented all details : 28/04/2022