segfault when multi thread repeated recycling in client

[answer9030]

NOTE: The iperf3 issue tracker is for registering bugs, enhancement
requests, or submissions of code. It is not a means for asking
questions about building or using iperf3. Those are best directed
towards the Discussions section for this project at
https://github.com/esnet/iperf/discussions
or to the iperf3 mailing list at [email protected].
A list of frequently-asked questions
regarding iperf3 can be found at http://software.es.net/iperf/faq.html.

Context

• Version of iperf3: 3.16

• Hardware:

• Operating system (and distribution, if any): centos

Please note: iperf3 is supported on Linux, FreeBSD, and macOS.
Support may be provided on a best-effort basis to other UNIX-like
platforms. We cannot provide support for building and/or running
iperf3 on Windows, iOS, or Android.

• Other relevant information (for example, non-default compilers,
  libraries, cross-compiling, etc.): glibc-2.38-29

Please fill out one of the "Bug Report" or "Enhancement Request"
sections, as appropriate. Note that submissions of bug fixes, new
features, etc. should be done as a pull request at
https://github.com/esnet/iperf/pulls

Bug Report

Long duration streaming with multiple threads resulted in segment fault on the client side at the end of the test.

Use the following command for testing:
server side: iperf -s
client side: iperf -c serverIP -t 2400 -P 8

At the end of the test, client side encountered segment fault.

• Expected Behavior
  Test completed normally.

• Actual Behavior

1. The testing environment spans multiple network nodes.
2. iperf's management connection does not send packets for a long time, triggering a timeout for connectioin tracking on intermediate node.
3. At the end of the test, the client reclaimed the thread and sent an end message using the management connection, but the message was discarded by the intermediate link.
4. Resulting in management connection timeout, triggering thread recycling again.
5. When the number of threads reaches 5 or more, glibc has already unmapped the address pointed to by “sp->thr”, resulting in a segment fault during the second thread recycling.

• Steps to Reproduce
  Use the following command for testing:
  server side: iperf -s
  client side: iperf -c serverIP -t 2400 -P 8

• Possible Solution

Enhancement Request

• Current behavior

• Desired behavior

• Implementation notes

[bmah888]

First and foremost, iperf3 shouldn't crash. So there's clearly something wrong.

We note that the control connection getting stomped on could be fixed by enabling TCP keepalives on the control connection, per PR #1423.

[answer9030]

The reason for the crash is that:

1. when there are more than 5 threads, glibc's pthread_join function will unmap thread resources and release the address pointed to by sp ->thr;
2. If there is a duplicate recycling thread at this time, the memory address pointed to by "sp ->thr" no longer exists, which will cause segment fault.

[bmah888]

Fixed by the merge of #1761.