Skip to content

Use a shared informer when getting node topology#5722

Merged
kleimkuhler merged 1 commit intolinkerd:mainfrom
fpetkovski:ld-5684
Feb 12, 2021
Merged

Use a shared informer when getting node topology#5722
kleimkuhler merged 1 commit intolinkerd:mainfrom
fpetkovski:ld-5684

Conversation

@fpetkovski
Copy link
Contributor

Getting information about node topology queries the k8s api directly.
In an environment with high traffic and high number of pods, the
k8s api server can become overwhelmed or start throttling requests.

This MR introduces a node informer to resolve the bottleneck and
fetch node information asynchronously.

Fixes #5684

@fpetkovski fpetkovski requested a review from a team as a code owner February 11, 2021 12:54
@fpetkovski fpetkovski force-pushed the ld-5684 branch 4 times, most recently from 262ace6 to b1f86b3 Compare February 11, 2021 13:35
@mateiidavid
Copy link
Member

Hey @fpetkovski thank you for taking care of this! 🥳 I really appreciate it.

This looks good to me. It is pretty much the way I would have done it, save for different (and possibly less inspired) names.

Copy link
Contributor

@kleimkuhler kleimkuhler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! It makes sense that this address the bottleneck we're seeing in that issue.

Getting information about node topology queries the k8s api directly.
In an environment with high traffic and high number of pods, the
k8s api server can become overwhelmed or start throttling requests.

This MR introduces a node informer to resolve the bottleneck and
fetch node information asynchronously.

Fixes linkerd#5684

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>
Copy link
Member

@adleong adleong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic!!! Thank you, @fpetkovski! 💯

@kleimkuhler kleimkuhler merged commit 73f9fb3 into linkerd:main Feb 12, 2021
jijeesh pushed a commit to jijeesh/linkerd2 that referenced this pull request Mar 23, 2021
Getting information about node topology queries the k8s api directly.
In an environment with high traffic and high number of pods, the
k8s api server can become overwhelmed or start throttling requests.

This MR introduces a node informer to resolve the bottleneck and
fetch node information asynchronously.

Fixes linkerd#5684

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>
Signed-off-by: Jijeesh <jijeesh.ka@gmail.com>
jijeesh pushed a commit to jijeesh/linkerd2 that referenced this pull request Apr 21, 2021
Getting information about node topology queries the k8s api directly.
In an environment with high traffic and high number of pods, the
k8s api server can become overwhelmed or start throttling requests.

This MR introduces a node informer to resolve the bottleneck and
fetch node information asynchronously.

Fixes linkerd#5684

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>
Signed-off-by: Jijeesh <jijeesh.ka@gmail.com>
@christianhuening
Copy link
Contributor

christianhuening commented Apr 27, 2021

@adleong much appreciated to see this solved! We just discovered that we ran into the same issue. Sadly, we currently cannot update to L5d 2.10.0 as far as I understand it, since we're still using K8s 1.15 and 2.10.0 requires K8s 1.16.x Is there a possibility to backport this fix to 2.9.4 ?

kleimkuhler pushed a commit that referenced this pull request Apr 29, 2021
Getting information about node topology queries the k8s api directly.
In an environment with high traffic and high number of pods, the
k8s api server can become overwhelmed or start throttling requests.

This MR introduces a node informer to resolve the bottleneck and
fetch node information asynchronously.

Fixes #5684

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>
kleimkuhler added a commit that referenced this pull request Apr 29, 2021
Getting information about node topology queries the k8s api directly.
In an environment with high traffic and high number of pods, the
k8s api server can become overwhelmed or start throttling requests.

This MR introduces a node informer to resolve the bottleneck and
fetch node information asynchronously.

Fixes #5684

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>
Co-authored-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
kleimkuhler pushed a commit that referenced this pull request Apr 29, 2021
Getting information about node topology queries the k8s api directly.
In an environment with high traffic and high number of pods, the
k8s api server can become overwhelmed or start throttling requests.

This MR introduces a node informer to resolve the bottleneck and
fetch node information asynchronously.

Fixes #5684

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Linkerd 2.9 destination component high latency and node topology request failures

5 participants