Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BPF] conntrack more resistant to spurious RSTs #9692

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

tomastigera
Copy link
Contributor

@tomastigera tomastigera commented Jan 10, 2025

When we see an RST on a connection, we mark the connection for cleanup
from the conntrack table. That happens immediatelly when the connection
becomes idle and the TCPResetSeen timeout expires (by default 40s). Not
a huge problem for busy connections or connections with keepalive.
Nevertheless, an attacker may taint connections and once the connections
are removed from conntrack, the endpoints need to get into
retransmittions and timeouts to close their ends.

This change removes the taint if the connection sees traffic after the
RST. First it extends the timeout to 2m and if there is still traffic,
it removes the taint completely.

This mechanism is not bullet-proof since and idle connection might still
be GCed, but it reduces the possibility of connection breakage when
there is a temporary hickup in the network. Also allows to reduce the
RSTSeen timeout.

fixes #9605

Description

Related issues/PRs

Todos

  • Tests
  • Documentation
  • Release note

Release Note

ebpf: conntrack more resistant to spurious RSTs

Reminder for the reviewer

Make sure that this PR has the correct labels and milestone set.

Every PR needs one docs-* label.

  • docs-pr-required: This change requires a change to the documentation that has not been completed yet.
  • docs-completed: This change has all necessary documentation completed.
  • docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

  • release-note-required: This PR has user-facing changes. Most PRs should have this label.
  • release-note-not-required: This PR has no user-facing changes.

Other optional labels:

  • cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
  • needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

@marvin-tigera marvin-tigera added this to the Calico v3.30.0 milestone Jan 10, 2025
@marvin-tigera marvin-tigera added release-note-required Change has user-facing impact (no matter how small) docs-pr-required Change is not yet documented labels Jan 10, 2025
Not used for anything useful and not used by the bpf conntrack cleanup
at all.
@tomastigera tomastigera force-pushed the tomas-bpf-spurious-rst-fix branch 2 times, most recently from b7e283c to 47d4582 Compare January 13, 2025 18:35
@tomastigera tomastigera changed the title [BPF] remove created value from conntrack [BPF] conntrack more resistant to spurious RSTs Jan 13, 2025
@tomastigera tomastigera force-pushed the tomas-bpf-spurious-rst-fix branch 2 times, most recently from db7e82d to 9abc2e4 Compare January 14, 2025 18:16
When we see an RST on a connection, we mark the connection for cleanup
from the conntrack table. That happens immediatelly when the connection
becomes idle and the TCPResetSeen timeout expires (by default 40s). Not
a huge problem for busy connections or connections with keepalive.
Nevertheless, an attacker may taint connections and once the connections
are removed from conntrack, the endpoints need to get into
retransmittions and timeouts to close their ends.

This change removes the taint if the connection sees traffic after the
RST. First it extends the timeout to 2m and if there is still traffic,
it removes the taint completely.

This mechanism is not bullet-proof since and idle connection might still
be GCed, but it reduces the possibility of connection breakage when
there is a temporary hickup in the network. Also allows to reduce the
RSTSeen timeout.
@tomastigera tomastigera force-pushed the tomas-bpf-spurious-rst-fix branch from 9abc2e4 to 2da39b9 Compare January 14, 2025 21:10
@tomastigera tomastigera added docs-not-required Docs not required for this change and removed docs-pr-required Change is not yet documented labels Jan 14, 2025
@tomastigera tomastigera marked this pull request as ready for review January 14, 2025 23:14
@tomastigera tomastigera requested a review from a team as a code owner January 14, 2025 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-not-required Docs not required for this change release-note-required Change has user-facing impact (no matter how small)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BPF] Conntrack entry cleaned up by TCP RSTs with unexpected sequence numbers
2 participants