Add netperf UDP stream test #2075

dPhanekham · 2020-01-07T21:59:12Z

Add support for netperf UDP stream test.
Add flags to set size of stream tests
Also add same flag to container_netperf benchmark

s-deitz · 2020-01-08T21:26:15Z

perfkitbenchmarker/linux_benchmarks/container_netperf_benchmark.py

@@ -27,6 +27,9 @@

 FLAGS = flags.FLAGS

+flags.DEFINE_integer('container_netperf_tcp_stream_send_size_in_bytes', 131072,


Please add a comment as to why this default value was selected.

This value, 131072 bytes (128KB) was selected at the recommendation of @jonesrick.

"The 128K send size is deliberate - otherwise netperf will default to 16K based on Linux send socket buffer size initial values, which has enough overhead to preclude achieving "link rate" as it were. And the default receive size will be 87380 bytes for similar reasons on the receive side. That would be a good thing to change in the netperf command-lines in PKB. And it would match the default send size in iperf3. 64K may be sufficient but 128K would give additional cushion. The need for receive side (-M) isn't as clear-cut but it won't hurt."

Or, if preferred, I could give it a default of None and let netperf select the size unless this flag is set.

I think setting the default is important. I'd just like a comment in the code so that we know why we selected this default. Noting that the Linux default of 16K cannot achieve "link rate" and that the higher default value will let us would be sufficient.

Do we also need to set -s?

Per https://hewlettpackard.github.io/netperf/doc/netperf.html#UDP_005fSTREAM
If the value of the -m option is larger than the local send socket buffer size (-s option) netperf will likely abort with an error message about how the send call failed:

netperf -t UDP_STREAM -H 192.168.2.125 UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.125 (192.168.2.125) port 0 AF_INET udp_send: data send error: Message too long

Added a comment to clarify default size. Addressed -s flag in a comment below

Thanks for adding the comment. Since comments are easier to parse if they contain punctuation and consist of complete sentences, perhaps edit it so that it reads more like the following:

We set the default to 128KB (131072 bytes) to override the Linux default

of 16K so that we can achieve the "link rate".

Hmm. Looks like the prefix of "#" changed my font size. I just meant to write a comment.

changed the comment

s-deitz · 2020-01-08T21:27:01Z

perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py

+flags.DEFINE_integer('netperf_udp_stream_send_size_in_bytes', 1024,
+                     'Send size to use for UDP_STREAM tests (netperf -m flag)')
+flags.DEFINE_integer('netperf_tcp_stream_send_size_in_bytes', 131072,
+                     'Send size to use for TCP_STREAM tests (netperf -m flag)')


Comment here as well.

comment added

perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py

s-deitz · 2020-01-08T21:33:34Z

perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py

+                     verbosity=verbosity)
+
+  if benchmark_name.upper() == 'UDP_STREAM':
+    netperf_cmd += (' -R 1 -m {send_size} '.format(


Is this a change in response size as well?

I have also added the -M flag to set the remote receive size as well. I do not believe it changes that response size, as UDP_STREAM and TCP_STREAM test throughput unidirectionally.

perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py

dlott · 2020-01-09T00:04:11Z

perfkitbenchmarker/linux_benchmarks/container_netperf_benchmark.py

@@ -27,6 +27,9 @@

 FLAGS = flags.FLAGS

+flags.DEFINE_integer('container_netperf_tcp_stream_send_size_in_bytes', 131072,


Do we also need to set -s?

Per https://hewlettpackard.github.io/netperf/doc/netperf.html#UDP_005fSTREAM
If the value of the -m option is larger than the local send socket buffer size (-s option) netperf will likely abort with an error message about how the send call failed:

netperf -t UDP_STREAM -H 192.168.2.125 UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.125 (192.168.2.125) port 0 AF_INET udp_send: data send error: Message too long

dlott · 2020-01-09T00:07:00Z

perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py

@@ -356,7 +360,7 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams):
  enable_latency_histograms = FLAGS.netperf_enable_histograms or num_streams > 1
  # Throughput benchmarks don't have latency histograms
  enable_latency_histograms = enable_latency_histograms and \
-      benchmark_name != 'TCP_STREAM'
+      (benchmark_name != 'TCP_STREAM' and benchmark_name != 'UDP_STREAM')


More readable as
(benchmark_name not in ['TCP_STREAM' , 'UDP_STREAM'])

Also if you could fix the \ continue as well, that'd be great.

when testing UDP_STREAM, i found the default send socket buffer size to be 212,992 bytes, which is larger than the max possible UDP message size of 65,507 bytes, so it should not be a problem and we don't need the -s flag. I can add in an upper limit of 65507 bytes, because it will cause an error if you try to use a larger size than that.

What platforms have you tested this on? My reading is that there may be some where -s is required (no idea which) but I don't see a downside to specifying both -s and -S

I've tested on ubuntu18, debian9, and centos8. So far I haven't needed to specify the -s/-S flags for things to work. But I can certainly add them in as pkb flags and give them default values

jonesrick · 2020-01-09T00:51:44Z

The "-R 1" as a test-specific option for a UDP_STREAM test is needed to disable setting SO_DONTROUTE on the data socket, which is set by default to address issues where various entities/groups in the have used the UDP_STREAM test as part of system link up/down testing and taken-out a corporate security video feed as a result. The way netmasks and such are set in GCP mean that all destinations are not "local" (need to be routed to) thus the "-R 1" is needed to successfully run a netperf UDP_STREAM test in GCP.

…

On Wed, Jan 8, 2020 at 4:08 PM dlott ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py <#2075 (comment)> : > @@ -377,7 +389,19 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams): server_ip=server_ip, length=FLAGS.netperf_test_length, output_selector=OUTPUT_SELECTOR, - confidence=confidence, verbosity=verbosity) + confidence=confidence, + verbosity=verbosity) + + if benchmark_name.upper() == 'UDP_STREAM': + netperf_cmd += (' -R 1 -m {send_size} -M {send_size} '.format( What is -R 1 for? I don't see it in https://hewlettpackard.github.io/netperf/doc/netperf.html#UDP_005fSTREAM ------------------------------ In perfkitbenchmarker/linux_benchmarks/container_netperf_benchmark.py <#2075 (comment)> : > @@ -27,6 +27,9 @@ FLAGS = flags.FLAGS +flags.DEFINE_integer('container_netperf_tcp_stream_send_size_in_bytes', 131072, Do we also need to set -s? Per https://hewlettpackard.github.io/netperf/doc/netperf.html#UDP_005fSTREAM If the value of the -m option is larger than the local send socket buffer size (-s option) netperf will likely abort with an error message about how the send call failed: netperf -t UDP_STREAM -H 192.168.2.125 UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.125 (192.168.2.125) port 0 AF_INET udp_send: data send error: Message too long ------------------------------ In perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py <#2075 (comment)> : > @@ -356,7 +360,7 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams): enable_latency_histograms = FLAGS.netperf_enable_histograms or num_streams > 1 # Throughput benchmarks don't have latency histograms enable_latency_histograms = enable_latency_histograms and \ - benchmark_name != 'TCP_STREAM' + (benchmark_name != 'TCP_STREAM' and benchmark_name != 'UDP_STREAM') More readable as (benchmark_name not in ['TCP_STREAM' , 'UDP_STREAM']) Also if you could fix the \ continue as well, that'd be great. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2075?email_source=notifications&email_token=AILLKXGUBRBFU2V7EAFSTC3Q4ZTHRA5CNFSM4KD7UUSKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCRDV2AY#pullrequestreview-340221187>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AILLKXB3H6YKSYEBJLJGYWDQ4ZTHRANCNFSM4KD7UUSA> .

jonesrick · 2020-01-09T00:53:41Z

A test-specific -s option is not needed with TCP_STREAM and a 131072 byte send size because TCP is a byte-stream rather than message-oriented protocol.

…

On Wed, Jan 8, 2020 at 4:08 PM dlott ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py <#2075 (comment)> : > @@ -377,7 +389,19 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams): server_ip=server_ip, length=FLAGS.netperf_test_length, output_selector=OUTPUT_SELECTOR, - confidence=confidence, verbosity=verbosity) + confidence=confidence, + verbosity=verbosity) + + if benchmark_name.upper() == 'UDP_STREAM': + netperf_cmd += (' -R 1 -m {send_size} -M {send_size} '.format( What is -R 1 for? I don't see it in https://hewlettpackard.github.io/netperf/doc/netperf.html#UDP_005fSTREAM ------------------------------ In perfkitbenchmarker/linux_benchmarks/container_netperf_benchmark.py <#2075 (comment)> : > @@ -27,6 +27,9 @@ FLAGS = flags.FLAGS +flags.DEFINE_integer('container_netperf_tcp_stream_send_size_in_bytes', 131072, Do we also need to set -s? Per https://hewlettpackard.github.io/netperf/doc/netperf.html#UDP_005fSTREAM If the value of the -m option is larger than the local send socket buffer size (-s option) netperf will likely abort with an error message about how the send call failed: netperf -t UDP_STREAM -H 192.168.2.125 UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.125 (192.168.2.125) port 0 AF_INET udp_send: data send error: Message too long ------------------------------ In perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py <#2075 (comment)> : > @@ -356,7 +360,7 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams): enable_latency_histograms = FLAGS.netperf_enable_histograms or num_streams > 1 # Throughput benchmarks don't have latency histograms enable_latency_histograms = enable_latency_histograms and \ - benchmark_name != 'TCP_STREAM' + (benchmark_name != 'TCP_STREAM' and benchmark_name != 'UDP_STREAM') More readable as (benchmark_name not in ['TCP_STREAM' , 'UDP_STREAM']) Also if you could fix the \ continue as well, that'd be great. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2075?email_source=notifications&email_token=AILLKXGUBRBFU2V7EAFSTC3Q4ZTHRA5CNFSM4KD7UUSKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCRDV2AY#pullrequestreview-340221187>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AILLKXB3H6YKSYEBJLJGYWDQ4ZTHRANCNFSM4KD7UUSA> .

jonesrick · 2020-01-09T18:22:26Z

I don't mean to "bikeshed" but since netperf will "process" the values given to the -m (and -M and -s and -S) option as occasion warrants, you could make this a string and then when someone says 128k they will get 128000 and when they say 128K they will get 131072. Netperf will interpret a suffix of k|m|g as powers of ten, and K|M|G as powers of two. Of course, then if they give a rather bogus string, Metcalf only knows what netperf will do with it :)

…

On Thu, Jan 9, 2020 at 9:39 AM Derek ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py <#2075 (comment)> : > @@ -65,8 +65,12 @@ flags.DEFINE_integer('netperf_thinktime_run_length', 0, 'The number of contiguous numbers to sum at a time in the ' 'thinktime array.') +flags.DEFINE_integer('netperf_udp_stream_send_size_in_bytes', 1024, + 'Send size to use for UDP_STREAM tests (netperf -m flag)') +flags.DEFINE_integer('netperf_tcp_stream_send_size_in_bytes', 131072, + 'Send size to use for TCP_STREAM tests (netperf -m flag)') comment added — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2075?email_source=notifications&email_token=AILLKXAH2QLBB3XPED43SVTQ45ONLA5CNFSM4KD7UUSKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCRHKCMY#discussion_r364872841>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AILLKXCRQMCS3VBOF3MXSYDQ45ONLANCNFSM4KD7UUSA> .

jonesrick · 2020-01-09T18:25:32Z

In the version of the netperf manual at kent.edu that specific case the -R appears to be in a section on DLPI-test-specific command-line options. That part of the manual does not pertain to the UDP_STREAM test.

…

On Thu, Jan 9, 2020 at 10:00 AM s-deitz ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py <#2075 (comment)> : > @@ -377,7 +389,19 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams): server_ip=server_ip, length=FLAGS.netperf_test_length, output_selector=OUTPUT_SELECTOR, - confidence=confidence, verbosity=verbosity) + confidence=confidence, + verbosity=verbosity) + + if benchmark_name.upper() == 'UDP_STREAM': + netperf_cmd += (' -R 1 -m {send_size} -M {send_size} '.format( I see it here as response size: http://www.cs.kent.edu/~farrell/dist/ref/Netperf.html — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2075?email_source=notifications&email_token=AILLKXAOPEB6CROU4MBX26TQ45Q5DA5CNFSM4KD7UUSKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCRHNBMI#discussion_r364882052>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AILLKXBUW2W4UG4DB4NYV7LQ45Q5DANCNFSM4KD7UUSA> .

perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py

dlott · 2020-01-09T21:10:45Z

perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py

@@ -356,7 +360,7 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams):
  enable_latency_histograms = FLAGS.netperf_enable_histograms or num_streams > 1
  # Throughput benchmarks don't have latency histograms
  enable_latency_histograms = enable_latency_histograms and \
-      benchmark_name != 'TCP_STREAM'
+      (benchmark_name != 'TCP_STREAM' and benchmark_name != 'UDP_STREAM')


What platforms have you tested this on? My reading is that there may be some where -s is required (no idea which) but I don't see a downside to specifying both -s and -S

dlott · 2020-01-09T21:12:06Z

If you move the additions in CHANGES.next.md up a line or two, they'll won't conflict as people add things.

dlott · 2020-01-09T21:18:36Z

tests/data/netperf_results.json

+  [
+    "MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 104.198.67.251 () port 20001 AF_INET : +/-2.500% @ 99% conf.",
+    "Throughput,Throughput Units,50th Percentile Latency Microseconds,90th Percentile Latency Microseconds,99th Percentile Latency Microseconds,Stddev Latency Microseconds,Minimum Latency Microseconds,Maximum Latency Microseconds,Confidence Iterations Run,Throughput Confidence Width (%),Local Transport Retransmissions,Remote Transport Retransmissions",
+    "1102.42,10^6bits/s,3,3,11,46.14,1,15144,1,-1.000,-1,-1"


Is this only a send throughput? Should there also be a receive throughput?

https://hewlettpackard.github.io/netperf/doc/netperf.html#UDP_005fSTREAM

The biggest of these implications is the data which is sent might not be received by the remote. For this reason, the output of a UDP_STREAM test shows both the sending and receiving throughput. On some platforms, it may be possible for the sending throughput to be reported as a value greater than the maximum rate of the link. This is common when the CPU(s) are faster than the network and there is no intra-stack flow-control.

Is do you have a pkb.log that shows the raw output of a run?

jonesrick · 2020-01-09T21:50:23Z

Yes, for netperf TCP_STREAM, a test-specific -R is not necessary. I suspect though we still want the -m option because we want netperf TCP_STREAM to use a send size of 128KiB and not the on-Linux default of 16KiB. If no send size (-m option) is specified on the command line, netperf uses the value of SO_SNDBUF at the time of socket creation. if there is no send socket buffer size option (test-specific -s) specified, that will be the default the system has configured for socket buffer size. Under Linux that is 16 KiB (net.ipv4.tcp_wmem, middle value). Thus, without a -m option, netperf will have a send size of 16 KiB. And that is insufficient to drive more than about 20 Gbit/s. So, we do want to pass a -m option, with a send size of 128KiB, to netperf via the command line for a TCP_STREAM test. This also matches the default send size for TCP in iperf3. There is similar logic for the receive size used by netperf, looking at the -M and possibly the -S option. net.ipv4.tcp_rmem, again middle value. There though, depending on the version of the kernel, netperf will default to either 86KiB or 128KiB because those are the default initial sizes for the SO_RCVBUF of a TCP socket. Whether one wants to have a -s or -S option will depend on circumstances (eg bandwidthXdelay product) and whether or not, under Linux, net.core.[rw]mem_max has been tuned. And/or net.ipv4.tcp{rw}mem.

…

On Thu, Jan 9, 2020 at 1:10 PM dlott ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py <#2075 (comment)> : > @@ -377,7 +392,19 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams): server_ip=server_ip, length=FLAGS.netperf_test_length, output_selector=OUTPUT_SELECTOR, - confidence=confidence, verbosity=verbosity) + confidence=confidence, + verbosity=verbosity) + + if benchmark_name.upper() == 'UDP_STREAM': + netperf_cmd += (' -R 1 -m {send_size} -M {send_size} '.format( + send_size=FLAGS.netperf_udp_stream_send_size_in_bytes)) + metadata['netperf_send_size_in_bytes'] = FLAGS.netperf_udp_stream_send_size_in_bytes + + elif benchmark_name.upper() == 'TCP_STREAM': + netperf_cmd += (' -R 1 -m {send_size} -M {send_size} '.format( Per rick's comment it seems that we can drop the -m and -M here. As well as the -R ------------------------------ In perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py <#2075 (comment)> : > @@ -356,7 +360,7 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams): enable_latency_histograms = FLAGS.netperf_enable_histograms or num_streams > 1 # Throughput benchmarks don't have latency histograms enable_latency_histograms = enable_latency_histograms and \ - benchmark_name != 'TCP_STREAM' + (benchmark_name != 'TCP_STREAM' and benchmark_name != 'UDP_STREAM') What platforms have you tested this on? My reading is that there may be some where -s is required (no idea which) but I don't see a downside to specifying both -s and -S — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2075?email_source=notifications&email_token=AILLKXB7CPPQPCFPGEZKEJ3Q46HFPA5CNFSM4KD7UUSKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCRIGXEA#pullrequestreview-340814736>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AILLKXEOBLQU6QJCJSE5NN3Q46HFPANCNFSM4KD7UUSA> .

jonesrick · 2020-01-09T21:54:32Z

For a UDP_STREAM test the "throughput" output selector will report what the remote netserver received. It will be equivalent to specifying remote_recv_throughput (or is supposed to be). $ netperf -- -O ? | grep -i through THROUGHPUT THROUGHPUT_UNITS THROUGHPUT_CONFID LOCAL_SEND_THROUGHPUT LOCAL_RECV_THROUGHPUT REMOTE_SEND_THROUGHPUT REMOTE_RECV_THROUGHPUT Local would be the netperf side and remote the netserver side.

…

On Thu, Jan 9, 2020 at 1:19 PM dlott ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In tests/data/netperf_results.json <#2075 (comment)> : > @@ -38,5 +38,15 @@ "MIGRATED UDP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 20001 AF_INET to 10.240.31.117 () port 20001 AF_INET : +/-2.500% @ 99% conf. : first burst 0", "Throughput,Throughput Units,Throughput Confidence Width (%),Confidence Iterations Run,Stddev Latency Microseconds,50th Percentile Latency Microseconds,90th Percentile Latency Microseconds,99th Percentile Latency Microseconds,Minimum Latency Microseconds,Maximum Latency Microseconds,Local Transport Retransmissions,Remote Transport Retransmissions", "3313.49,Trans/s,7.546,20,214.64,295,330,406,200,500,0,0" + ], + [ + "MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 104.198.67.251 () port 20001 AF_INET : +/-2.500% @ 99% conf.", + "Throughput,Throughput Units,50th Percentile Latency Microseconds,90th Percentile Latency Microseconds,99th Percentile Latency Microseconds,Stddev Latency Microseconds,Minimum Latency Microseconds,Maximum Latency Microseconds,Confidence Iterations Run,Throughput Confidence Width (%),Local Transport Retransmissions,Remote Transport Retransmissions", + "1102.42,10^6bits/s,3,3,11,46.14,1,15144,1,-1.000,-1,-1" Is this only a send throughput? Should there also be a receive throughput? https://hewlettpackard.github.io/netperf/doc/netperf.html#UDP_005fSTREAM The biggest of these implications is the data which is sent might not be received by the remote. For this reason, the output of a UDP_STREAM test shows both the sending and receiving throughput. On some platforms, it may be possible for the sending throughput to be reported as a value greater than the maximum rate of the link. This is common when the CPU(s) are faster than the network and there is no intra-stack flow-control. Is do you have a pkb.log that shows the raw output of a run? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2075?email_source=notifications&email_token=AILLKXAKWNVIQN3KVZO2EATQ46IEHA5CNFSM4KD7UUSKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCRIID5Y#pullrequestreview-340820471>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AILLKXCWTXF6X7Z6UPTOF33Q46IEHANCNFSM4KD7UUSA> .

jonesrick · 2020-01-10T16:00:28Z

There can be an entire discussion around whether or not a TCP_STREAM test requires a test-specific -s or -S option. It would center on when Linux uses autotuning (no -s or -S option) and the settings for net.ipv4.tcp_[rw]mem on both sides, and when the application makes an explicit setsockopt() call for SO_SNDBUF or SO_RCVBUF and where net.core.[rw]mem_max are set. At this point I would recommend against using -s and/or -S with default values. Unless that default value is "-1" in which case netperf will parse the option(s) but not actually make any setsockopt() calls to set socket buffer size. UDP_STREAM on the other hand possibly wants a -S option but without tuning sysctls (net.core.rmem_max) it likely won't be effective anyway. A UDP_STREAM test (under Linux at least) does *not* want a -s option.

…

On Thu, Jan 9, 2020 at 11:36 PM Derek ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py <#2075 (comment)> : > @@ -356,7 +360,7 @@ def RunNetperf(vm, benchmark_name, server_ip, num_streams): enable_latency_histograms = FLAGS.netperf_enable_histograms or num_streams > 1 # Throughput benchmarks don't have latency histograms enable_latency_histograms = enable_latency_histograms and \ - benchmark_name != 'TCP_STREAM' + (benchmark_name != 'TCP_STREAM' and benchmark_name != 'UDP_STREAM') I've test on ubuntu18, debian9, and centos8. So far I haven't needed to specify the -s/-S flags. But I can certainly certainly add them in as flags and just give them the default values netperf has given them so far in my testing. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2075?email_source=notifications&email_token=AILLKXHFEH4U7CO624LQFGDQ5AQQNA5CNFSM4KD7UUSKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCRJSGGQ#discussion_r365103761>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AILLKXFT7N32FDPE2CYABO3Q5AQQNANCNFSM4KD7UUSA> .

dlott

Looks good.

googlebot added the cla: yes label Jan 7, 2020

dPhanekham force-pushed the netperf_UDP_STREAM branch from 26ea6d8 to a4314b4 Compare January 7, 2020 22:57

s-deitz reviewed Jan 8, 2020

View reviewed changes

cwilkes reviewed Jan 8, 2020

View reviewed changes

perfkitbenchmarker/linux_benchmarks/netperf_benchmark.py Outdated Show resolved Hide resolved

dlott reviewed Jan 9, 2020

View reviewed changes

dPhanekham force-pushed the netperf_UDP_STREAM branch from 61eaa12 to a77116f Compare January 10, 2020 06:39

dPhanekham added 13 commits January 10, 2020 12:17

add UDP_STREAM and send size flags

80e8e7d

add flags and UDP_STREAM and pep fixes

634014f

remove debug statements

6e4eb0d

add flag to container_netperf

c108990

added UDP STREAM results to tests

0c327f1

add -M flag

055d047

remove unnecessary line

01f9d22

add comment justifying default size

edbc749

change to not in

75deef9

add upper and lower bounds to UDP send size

26c5758

revised comment

28eecd7

remove -R 1 flag for TCP_STREAM, it is implicit

b6ff0ce

remove line

e5991df

dPhanekham force-pushed the netperf_UDP_STREAM branch from 640fc3c to e5991df Compare January 10, 2020 18:18

dlott approved these changes Jan 10, 2020

View reviewed changes

s-deitz approved these changes Jan 10, 2020

View reviewed changes

cwilkes approved these changes Jan 10, 2020

View reviewed changes

Merge branch 'master' into netperf_UDP_STREAM

c5a2f69

bvliu merged commit 07441b9 into GoogleCloudPlatform:master Jan 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add netperf UDP stream test #2075

Add netperf UDP stream test #2075

dPhanekham commented Jan 7, 2020

s-deitz Jan 8, 2020

dPhanekham Jan 8, 2020

s-deitz Jan 8, 2020

dlott Jan 9, 2020

dPhanekham Jan 9, 2020

s-deitz Jan 9, 2020

s-deitz Jan 9, 2020

dPhanekham Jan 9, 2020

s-deitz Jan 8, 2020

dPhanekham Jan 9, 2020

s-deitz Jan 8, 2020

dPhanekham Jan 8, 2020

dlott Jan 9, 2020

dlott Jan 9, 2020

dPhanekham Jan 9, 2020

dlott Jan 9, 2020

dPhanekham Jan 10, 2020 •

edited

Loading

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 9, 2020 via email

dlott Jan 9, 2020

dlott commented Jan 9, 2020

dlott Jan 9, 2020

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 10, 2020 via email

dlott left a comment

		@@ -27,6 +27,9 @@

		FLAGS = flags.FLAGS

		flags.DEFINE_integer('container_netperf_tcp_stream_send_size_in_bytes', 131072,

Add netperf UDP stream test #2075

Add netperf UDP stream test #2075

Conversation

dPhanekham commented Jan 7, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

We set the default to 128KB (131072 bytes) to override the Linux default

of 16K so that we can achieve the "link rate".

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dPhanekham Jan 10, 2020 • edited Loading

Choose a reason for hiding this comment

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 9, 2020 via email

Choose a reason for hiding this comment

dlott commented Jan 9, 2020

Choose a reason for hiding this comment

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 9, 2020 via email

jonesrick commented Jan 10, 2020 via email

dlott left a comment

Choose a reason for hiding this comment

dPhanekham Jan 10, 2020 •

edited

Loading