benchmark: maintain no. of parallel requests in throughput benchmark

[unsuitable001]

Spawn a new request when any request is returned within stipulated time duration in case of throughput benchmark to maintain the number of parallel requests performed.

Closes #16

[unsuitable001]

Check the changes. Especially the onError one, if it's the correct way to deal with it. (Request timeouts or page not found or any other problem will be caught by this onError. So, we're assuming they never returned (i.e. returned outside out time limit).)

If this is alright, we may have to update the measurements. And, bump the version?

@mannprerak2 @dcharkes

[dcharkes]

results here: DartIO has higher throughput. It would be interesting to figure out why. (CPU overhead of Cronet vs DartIO?)

https://gist.github.com/dcharkes/2194c401945a60930f07225c792369a6

Yes, we have to update the measurements.

No, we don't have to bump the version. We don't actually change the package itself, just the benchmarks and markdown files which is not something that is subject to versioning in a Dart package. It is not the Dart API or the semantics of the API.

[unsuitable001]

I can't conduct the throughput test on local server. Maxing out my resources and then crashing.

terminate called after throwing an instance of 'std::system_error'
  what():  Resource temporarily unavailable
Aborted (core dumped)

Against https://example.com though, my result seems weird -

CronetThroughputBenchmark: Parallel Requests: 256, Returned in time: 98.
[0722/183631.048860:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
CronetThroughputBenchmark: Parallel Requests: 512, Returned in time: 148.
[0722/183634.464644:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
CronetThroughputBenchmark: Parallel Requests: 1024, Returned in time: 136.

DartIOThroughputBenchmark: Parallel Requests: 32, Returned in time: 5.
DartIOThroughputBenchmark: Parallel Requests: 64, Returned in time: 3.
DartIOThroughputBenchmark: Parallel Requests: 128, Returned in time: 1.
DartIOThroughputBenchmark: Parallel Requests: 256, Returned in time: 0.
DartIOThroughputBenchmark: Parallel Requests: 512, Returned in time: 0.

Should I put these in markdown file? And, can you conduct the local server test? System is crashing every time. Though we can avoid this test if it's causing unrelated issues like this.

[dcharkes]

Hm, you could try to figure out what is going on with example.com. Maybe you're over-saturating network bandwidth or CPU? Understanding it will teach us more about the two different systems.

If I find some time I can run the local tests. (But they might be less representative for real-world performance, so I would not prioritize these.)

[unsuitable001]

A closer inspection of dart:io revealed this when doing throughput test against example.com -

CPU spikes all over the place. But, in case of package:cronet, the graph was more or less consistent.

Maybe, #3 (comment) this was the indication. In our previous throughput testing script, we've noticed -

dart:io: Total Spawned: 512, In Time Returns: 512. This is the best result. After that, numbers dipped to 1 successful request. So, 512 concurrent requests are the limit for dart:io in this case.

😕

[unsuitable001]

Tested on another local server (HTTP/1.1 Python Simple Server) and got this -

dart run benchmark/throughput.dart -u http://0.0.0.0:60000/
CronetThroughputBenchmark: Parallel Requests: 1, Returned in time: 509.
CronetThroughputBenchmark: Parallel Requests: 2, Returned in time: 760.
CronetThroughputBenchmark: Parallel Requests: 4, Returned in time: 1062.
CronetThroughputBenchmark: Parallel Requests: 8, Returned in time: 1066.
CronetThroughputBenchmark: Parallel Requests: 16, Returned in time: 1057.
CronetThroughputBenchmark: Parallel Requests: 32, Returned in time: 1061.
CronetThroughputBenchmark: Parallel Requests: 64, Returned in time: 1069.
CronetThroughputBenchmark: Parallel Requests: 128, Returned in time: 1064.
CronetThroughputBenchmark: Parallel Requests: 256, Returned in time: 1053.
CronetThroughputBenchmark: Parallel Requests: 512, Returned in time: 981.
CronetThroughputBenchmark: Parallel Requests: 1024, Returned in time: 996.
*****
DartIOThroughputBenchmark: Parallel Requests: 1, Returned in time: 538.
DartIOThroughputBenchmark: Parallel Requests: 2, Returned in time: 815.
DartIOThroughputBenchmark: Parallel Requests: 4, Returned in time: 983.
DartIOThroughputBenchmark: Parallel Requests: 8, Returned in time: 1124.
DartIOThroughputBenchmark: Parallel Requests: 16, Returned in time: 1111.
DartIOThroughputBenchmark: Parallel Requests: 32, Returned in time: 1099.
DartIOThroughputBenchmark: Parallel Requests: 64, Returned in time: 1114.
DartIOThroughputBenchmark: Parallel Requests: 128, Returned in time: 1106.

Got no response from dart:io after 128 mark. But, it performs faster than cronet.

[unsuitable001]

@dcharkes Updated the files with the results from your machine as results seems kinda weird on mine. I ran it multiple times and got same response every time. I think we should investigate why this is happening while we continue working on other features/issues as my machine is more representative of a general person's machine (config. wise). I need help regarding that. I've put some of the details in the above comments :)