Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib: revert primordials in a hot path #38246

Closed
wants to merge 8 commits into from

Conversation

mcollina
Copy link
Member

As part of #37937, I have been tracking all regressions that happened in the HTTP response hot path. I have strong evidence that a part of that is caused by the introduction of primordials through our codebase.

I recommend we stop "primordial" activities and we benchmark things extensively before adding them. Specifically there have been a few cases where PRs landed without benchamrks or without benchmarking the hot path where those are going to be used. Quite a few of those areas touched by this PR are extremely hot paths.

                                                                                                 confidence improvement accuracy (*)   (**)   (***)
http/bench-parser.jsn=100000 len=16                                                                              0.10 %       ±1.14% ±1.52%  ±1.98%
http/bench-parser.jsn=100000 len=32                                                                             -0.17 %       ±0.73% ±0.97%  ±1.27%
http/bench-parser.jsn=100000 len=4                                                                               0.12 %       ±0.82% ±1.09%  ±1.42%
http/bench-parser.jsn=100000 len=8                                                                               0.49 %       ±0.93% ±1.24%  ±1.62%
http/check_invalid_header_char.jsn=1000000 input=''                                                             -0.16 %       ±0.50% ±0.67%  ±0.87%
http/check_invalid_header_char.jsn=1000000 input='\\t\\t\\t\\t\\t\\t\\t\\t\\t\\tFoo bar baz'                     0.17 %       ±0.57% ±0.76%  ±1.00%
http/check_invalid_header_char.jsn=1000000 input='\\x7F'                                                         0.34 %       ±0.51% ±0.68%  ±0.89%
http/check_invalid_header_char.jsn=1000000 input='1'                                                             0.04 %       ±0.54% ±0.71%  ±0.93%
http/check_invalid_header_char.jsn=1000000 input='20091'                                                  *      0.56 %       ±0.47% ±0.62%  ±0.81%
http/check_invalid_header_char.jsn=1000000 input='close'                                                 **      0.65 %       ±0.44% ±0.59%  ±0.76%
http/check_invalid_header_char.jsn=1000000 input='en-US'                                                         0.22 %       ±0.46% ±0.62%  ±0.81%
http/check_invalid_header_char.jsn=1000000 input='foo\\nbar'                                                     0.02 %       ±0.46% ±0.61%  ±0.80%
http/check_invalid_header_char.jsn=1000000 input='group_acmeair'                                                -0.39 %       ±0.59% ±0.78%  ±1.03%
http/check_invalid_header_char.jsn=1000000 input='gzip'                                                          0.37 %       ±0.42% ±0.56%  ±0.73%
http/check_invalid_header_char.jsn=1000000 input='keep-alive'                                                    0.23 %       ±0.42% ±0.55%  ±0.72%
http/check_invalid_header_char.jsn=1000000 input='LONG_AND_INVALID'                                             -0.08 %       ±0.17% ±0.22%  ±0.29%
http/check_invalid_header_char.jsn=1000000 input='private'                                                       1.58 %       ±2.31% ±3.08%  ±4.02%
http/check_invalid_header_char.jsn=1000000 input='SAMEORIGIN'                                                    1.61 %       ±3.11% ±4.18%  ±5.54%
http/check_invalid_header_char.jsn=1000000 input='Sat, 07 May 2016 16:54:48 GMT'                                -1.24 %       ±1.65% ±2.20%  ±2.87%
http/check_invalid_header_char.jsn=1000000 input='text/html; charset=utf-8'                                      2.21 %       ±2.36% ±3.16%  ±4.15%
http/check_invalid_header_char.jsn=1000000 input='text/plain'                                                   -1.17 %       ±3.07% ±4.14%  ±5.49%
http/check_invalid_header_char.jsn=1000000 input='中文呢'                                                       -0.08 %       ±0.46% ±0.61%  ±0.80%
http/check_is_http_token.jsn=1000000 key=':'                                                                    -0.20 %       ±0.53% ±0.70%  ±0.92%
http/check_is_http_token.jsn=1000000 key=':alternate-protocol'                                           **      0.75 %       ±0.46% ±0.61%  ±0.79%
http/check_is_http_token.jsn=1000000 key='((((())))'                                                             0.26 %       ±0.55% ±0.73%  ±0.95%
http/check_is_http_token.jsn=1000000 key='@@'                                                                    0.22 %       ±0.48% ±0.64%  ±0.83%
http/check_is_http_token.jsn=1000000 key='Accept-Ranges'                                                         0.01 %       ±0.37% ±0.50%  ±0.65%
http/check_is_http_token.jsn=1000000 key='alt-svc'                                                               0.08 %       ±0.45% ±0.59%  ±0.77%
http/check_is_http_token.jsn=1000000 key='alternate-protocol:'                                                   0.25 %       ±0.26% ±0.34%  ±0.45%
http/check_is_http_token.jsn=1000000 key='alternate-protocol'                                                    0.21 %       ±0.31% ±0.41%  ±0.53%
http/check_is_http_token.jsn=1000000 key='Cache-Control'                                                         0.01 %       ±0.42% ±0.56%  ±0.73%
http/check_is_http_token.jsn=1000000 key='Connection'                                                            0.21 %       ±0.29% ±0.39%  ±0.50%
http/check_is_http_token.jsn=1000000 key='Content-Encoding'                                                      0.16 %       ±0.42% ±0.55%  ±0.72%
http/check_is_http_token.jsn=1000000 key='content-length'                                                        0.22 %       ±0.31% ±0.41%  ±0.53%
http/check_is_http_token.jsn=1000000 key='Content-Location'                                                      0.14 %       ±0.35% ±0.47%  ±0.61%
http/check_is_http_token.jsn=1000000 key='content-type'                                                          0.13 %       ±0.37% ±0.49%  ±0.64%
http/check_is_http_token.jsn=1000000 key='Content-Type'                                                          0.21 %       ±0.34% ±0.46%  ±0.59%
http/check_is_http_token.jsn=1000000 key='date'                                                                  0.07 %       ±0.48% ±0.64%  ±0.84%
http/check_is_http_token.jsn=1000000 key='ETag'                                                                  0.22 %       ±0.40% ±0.53%  ±0.70%
http/check_is_http_token.jsn=1000000 key='Expires'                                                              -0.20 %       ±0.52% ±0.69%  ±0.90%
http/check_is_http_token.jsn=1000000 key='Keep-Alive'                                                           -0.17 %       ±0.32% ±0.42%  ±0.55%
http/check_is_http_token.jsn=1000000 key='Last-Modified'                                                         0.18 %       ±0.37% ±0.49%  ±0.64%
http/check_is_http_token.jsn=1000000 key='location'                                                              0.01 %       ±0.39% ±0.52%  ±0.68%
http/check_is_http_token.jsn=1000000 key='server'                                                                0.05 %       ±0.57% ±0.75%  ±0.98%
http/check_is_http_token.jsn=1000000 key='Server'                                                               -0.12 %       ±0.45% ±0.60%  ±0.78%
http/check_is_http_token.jsn=1000000 key='status'                                                                0.05 %       ±0.51% ±0.68%  ±0.89%
http/check_is_http_token.jsn=1000000 key='TCN'                                                                  -0.20 %       ±0.59% ±0.79%  ±1.03%
http/check_is_http_token.jsn=1000000 key='Transfer-Encoding'                                              *      0.35 %       ±0.32% ±0.43%  ±0.56%
http/check_is_http_token.jsn=1000000 key='Vary'                                                                 -0.18 %       ±0.44% ±0.59%  ±0.77%
http/check_is_http_token.jsn=1000000 key='version'                                                              -0.10 %       ±0.45% ±0.60%  ±0.78%
http/check_is_http_token.jsn=1000000 key='x-frame-options'                                                       0.03 %       ±0.33% ±0.44%  ±0.57%
http/check_is_http_token.jsn=1000000 key='x-xss-protection'                                                     -0.08 %       ±0.31% ±0.41%  ±0.53%
http/check_is_http_token.jsn=1000000 key='中文呢'                                                        **     -0.78 %       ±0.52% ±0.70%  ±0.91%
http/chunked.jsduration=5 c=100 len=1 n=1 benchmarker='wrk'                                                      0.73 %       ±1.89% ±2.52%  ±3.28%
http/chunked.jsduration=5 c=100 len=1 n=16 benchmarker='wrk'                                                     1.10 %       ±1.75% ±2.33%  ±3.04%
http/chunked.jsduration=5 c=100 len=1 n=4 benchmarker='wrk'                                                     -0.15 %       ±1.04% ±1.38%  ±1.79%
http/chunked.jsduration=5 c=100 len=1 n=8 benchmarker='wrk'                                                      0.68 %       ±1.01% ±1.35%  ±1.76%
http/chunked.jsduration=5 c=100 len=256 n=1 benchmarker='wrk'                                                   -0.01 %       ±2.16% ±2.87%  ±3.74%
http/chunked.jsduration=5 c=100 len=256 n=16 benchmarker='wrk'                                                  -0.27 %       ±1.76% ±2.35%  ±3.05%
http/chunked.jsduration=5 c=100 len=256 n=4 benchmarker='wrk'                                                    0.13 %       ±1.15% ±1.53%  ±2.00%
http/chunked.jsduration=5 c=100 len=256 n=8 benchmarker='wrk'                                                    0.34 %       ±1.25% ±1.66%  ±2.17%
http/chunked.jsduration=5 c=100 len=64 n=1 benchmarker='wrk'                                                     0.40 %       ±1.69% ±2.25%  ±2.93%
http/chunked.jsduration=5 c=100 len=64 n=16 benchmarker='wrk'                                                   -0.54 %       ±1.85% ±2.47%  ±3.23%
http/chunked.jsduration=5 c=100 len=64 n=4 benchmarker='wrk'                                                    -1.04 %       ±1.08% ±1.43%  ±1.87%
http/chunked.jsduration=5 c=100 len=64 n=8 benchmarker='wrk'                                                    -0.68 %       ±1.05% ±1.40%  ±1.82%
http/client-request-body.jsmethod='end' len=1024 type='asc' dur=5                                       ***      4.34 %       ±0.66% ±0.88%  ±1.14%
http/client-request-body.jsmethod='end' len=1024 type='buf' dur=5                                       ***      4.06 %       ±0.71% ±0.95%  ±1.24%
http/client-request-body.jsmethod='end' len=1024 type='utf' dur=5                                       ***      4.66 %       ±0.71% ±0.95%  ±1.24%
http/client-request-body.jsmethod='end' len=256 type='asc' dur=5                                        ***      4.13 %       ±0.69% ±0.91%  ±1.19%
http/client-request-body.jsmethod='end' len=256 type='buf' dur=5                                        ***      3.98 %       ±0.70% ±0.93%  ±1.21%
http/client-request-body.jsmethod='end' len=256 type='utf' dur=5                                        ***      3.55 %       ±0.74% ±0.98%  ±1.28%
http/client-request-body.jsmethod='end' len=32 type='asc' dur=5                                         ***      2.22 %       ±0.67% ±0.89%  ±1.16%
http/client-request-body.jsmethod='end' len=32 type='buf' dur=5                                         ***      2.38 %       ±0.86% ±1.14%  ±1.48%
http/client-request-body.jsmethod='end' len=32 type='utf' dur=5                                         ***      2.62 %       ±0.76% ±1.01%  ±1.31%
http/client-request-body.jsmethod='write' len=1024 type='asc' dur=5                                     ***      4.32 %       ±0.72% ±0.96%  ±1.25%
http/client-request-body.jsmethod='write' len=1024 type='buf' dur=5                                     ***      4.32 %       ±0.73% ±0.97%  ±1.26%
http/client-request-body.jsmethod='write' len=1024 type='utf' dur=5                                     ***      4.11 %       ±0.68% ±0.90%  ±1.17%
http/client-request-body.jsmethod='write' len=256 type='asc' dur=5                                      ***      4.03 %       ±0.71% ±0.94%  ±1.23%
http/client-request-body.jsmethod='write' len=256 type='buf' dur=5                                      ***      4.45 %       ±0.72% ±0.95%  ±1.24%
http/client-request-body.jsmethod='write' len=256 type='utf' dur=5                                      ***      3.89 %       ±0.63% ±0.83%  ±1.08%
http/client-request-body.jsmethod='write' len=32 type='asc' dur=5                                       ***      2.27 %       ±0.64% ±0.85%  ±1.11%
http/client-request-body.jsmethod='write' len=32 type='buf' dur=5                                       ***      1.83 %       ±0.65% ±0.87%  ±1.13%
http/client-request-body.jsmethod='write' len=32 type='utf' dur=5                                       ***      2.77 %       ±0.66% ±0.88%  ±1.15%
http/cluster.jsduration=5 c=50 len=1024 type='buffer' benchmarker='wrk'                                 ***      2.76 %       ±0.84% ±1.11%  ±1.45%
http/cluster.jsduration=5 c=50 len=1024 type='bytes' benchmarker='wrk'                                  ***      2.32 %       ±0.68% ±0.90%  ±1.18%
http/cluster.jsduration=5 c=50 len=102400 type='buffer' benchmarker='wrk'                               ***      1.59 %       ±0.63% ±0.84%  ±1.09%
http/cluster.jsduration=5 c=50 len=102400 type='bytes' benchmarker='wrk'                                  *      0.53 %       ±0.50% ±0.67%  ±0.87%
http/cluster.jsduration=5 c=50 len=4 type='buffer' benchmarker='wrk'                                    ***      2.05 %       ±0.88% ±1.17%  ±1.52%
http/cluster.jsduration=5 c=50 len=4 type='bytes' benchmarker='wrk'                                     ***      2.43 %       ±0.77% ±1.03%  ±1.35%
http/cluster.jsduration=5 c=500 len=1024 type='buffer' benchmarker='wrk'                                ***      2.26 %       ±0.77% ±1.03%  ±1.34%
http/cluster.jsduration=5 c=500 len=1024 type='bytes' benchmarker='wrk'                                 ***      2.02 %       ±0.81% ±1.07%  ±1.40%
http/cluster.jsduration=5 c=500 len=102400 type='buffer' benchmarker='wrk'                              ***      2.01 %       ±0.68% ±0.90%  ±1.17%
http/cluster.jsduration=5 c=500 len=102400 type='bytes' benchmarker='wrk'                               ***      1.35 %       ±0.71% ±0.95%  ±1.24%
http/cluster.jsduration=5 c=500 len=4 type='buffer' benchmarker='wrk'                                   ***      3.17 %       ±0.90% ±1.20%  ±1.56%
http/cluster.jsduration=5 c=500 len=4 type='bytes' benchmarker='wrk'                                    ***      2.18 %       ±0.81% ±1.08%  ±1.40%
http/create-clientrequest.jse=1 arg='options' url='idn'                                                         -0.43 %       ±1.00% ±1.34%  ±1.74%
http/create-clientrequest.jse=1 arg='options' url='long'                                                        -0.40 %       ±1.04% ±1.39%  ±1.81%
http/create-clientrequest.jse=1 arg='options' url='wpt'                                                         -1.05 %       ±1.32% ±1.76%  ±2.29%
http/create-clientrequest.jse=1 arg='string' url='idn'                                                    *     -0.82 %       ±0.69% ±0.91%  ±1.19%
http/create-clientrequest.jse=1 arg='string' url='long'                                                  **     -1.06 %       ±0.65% ±0.87%  ±1.13%
http/create-clientrequest.jse=1 arg='string' url='wpt'                                                          -0.62 %       ±1.00% ±1.34%  ±1.74%
http/create-clientrequest.jse=1 arg='URL' url='idn'                                                              0.75 %       ±2.14% ±2.85%  ±3.70%
http/create-clientrequest.jse=1 arg='URL' url='long'                                                             0.16 %       ±0.91% ±1.21%  ±1.58%
http/create-clientrequest.jse=1 arg='URL' url='wpt'                                                              1.11 %       ±1.30% ±1.73%  ±2.26%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=1048576 type='asc' benchmarker='wrk'                  -0.09 %       ±0.65% ±0.86%  ±1.12%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=1048576 type='buf' benchmarker='wrk'           **      1.78 %       ±1.33% ±1.78%  ±2.31%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=1048576 type='utf' benchmarker='wrk'                   0.18 %       ±0.33% ±0.44%  ±0.57%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=131072 type='asc' benchmarker='wrk'                    0.48 %       ±0.57% ±0.75%  ±0.98%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=131072 type='buf' benchmarker='wrk'           ***      2.66 %       ±1.53% ±2.03%  ±2.65%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=131072 type='utf' benchmarker='wrk'                    0.38 %       ±1.10% ±1.46%  ±1.90%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=262144 type='asc' benchmarker='wrk'                   -0.01 %       ±0.43% ±0.57%  ±0.75%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=262144 type='buf' benchmarker='wrk'             *      1.44 %       ±1.30% ±1.73%  ±2.25%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=262144 type='utf' benchmarker='wrk'                   -0.02 %       ±0.33% ±0.44%  ±0.57%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=65536 type='asc' benchmarker='wrk'              *      0.83 %       ±0.80% ±1.06%  ±1.38%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=65536 type='buf' benchmarker='wrk'             **      2.66 %       ±1.64% ±2.19%  ±2.85%
http/end-vs-write-end.jsduration=5 method='end' c=100 len=65536 type='utf' benchmarker='wrk'                     0.80 %       ±1.02% ±1.36%  ±1.77%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=1048576 type='asc' benchmarker='wrk'                -0.14 %       ±0.43% ±0.57%  ±0.74%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=1048576 type='buf' benchmarker='wrk'                 0.25 %       ±1.33% ±1.78%  ±2.32%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=1048576 type='utf' benchmarker='wrk'                 0.10 %       ±0.30% ±0.40%  ±0.52%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=131072 type='asc' benchmarker='wrk'           *      1.08 %       ±1.04% ±1.38%  ±1.80%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=131072 type='buf' benchmarker='wrk'                 -0.22 %       ±1.46% ±1.94%  ±2.53%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=131072 type='utf' benchmarker='wrk'                  0.67 %       ±1.03% ±1.38%  ±1.79%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=262144 type='asc' benchmarker='wrk'                  0.61 %       ±1.67% ±2.23%  ±2.90%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=262144 type='buf' benchmarker='wrk'                  0.75 %       ±1.20% ±1.59%  ±2.07%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=262144 type='utf' benchmarker='wrk'                  0.09 %       ±0.25% ±0.33%  ±0.44%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=65536 type='asc' benchmarker='wrk'           **      1.33 %       ±0.82% ±1.10%  ±1.43%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=65536 type='buf' benchmarker='wrk'                   1.50 %       ±1.56% ±2.07%  ±2.70%
http/end-vs-write-end.jsduration=5 method='write' c=100 len=65536 type='utf' benchmarker='wrk'            *      1.31 %       ±1.03% ±1.37%  ±1.78%
http/headers.jsduration=5 len=1 n=10 benchmarker='wrk'                                                  ***      3.88 %       ±2.01% ±2.67%  ±3.48%
http/headers.jsduration=5 len=1 n=600 benchmarker='wrk'                                                         -0.83 %       ±1.18% ±1.57%  ±2.05%
http/headers.jsduration=5 len=100 n=10 benchmarker='wrk'                                                ***      2.67 %       ±1.33% ±1.77%  ±2.30%
http/headers.jsduration=5 len=100 n=600 benchmarker='wrk'                                                        0.12 %       ±1.95% ±2.59%  ±3.38%
http/http_server_for_chunky_client.jstype='send' n=2000 len=1                                                    0.16 %       ±0.91% ±1.21%  ±1.58%
http/http_server_for_chunky_client.jstype='send' n=2000 len=128                                                  0.76 %       ±0.81% ±1.08%  ±1.40%
http/http_server_for_chunky_client.jstype='send' n=2000 len=16                                                   0.34 %       ±0.78% ±1.04%  ±1.35%
http/http_server_for_chunky_client.jstype='send' n=2000 len=32                                                   0.22 %       ±0.93% ±1.24%  ±1.61%
http/http_server_for_chunky_client.jstype='send' n=2000 len=4                                                    0.99 %       ±1.08% ±1.44%  ±1.88%
http/http_server_for_chunky_client.jstype='send' n=2000 len=64                                                   0.80 %       ±0.80% ±1.06%  ±1.38%
http/http_server_for_chunky_client.jstype='send' n=2000 len=8                                                    0.44 %       ±0.72% ±0.96%  ±1.25%
http/http_server_for_chunky_client.jstype='send' n=5 len=1                                                      -2.01 %       ±6.61% ±8.79% ±11.45%
http/http_server_for_chunky_client.jstype='send' n=5 len=128                                                     1.88 %       ±2.45% ±3.29%  ±4.35%
http/http_server_for_chunky_client.jstype='send' n=5 len=16                                                     -0.47 %       ±2.38% ±3.20%  ±4.23%
http/http_server_for_chunky_client.jstype='send' n=5 len=32                                                     -2.52 %       ±3.94% ±5.31%  ±7.03%
http/http_server_for_chunky_client.jstype='send' n=5 len=4                                                      -0.28 %       ±2.58% ±3.48%  ±4.60%
http/http_server_for_chunky_client.jstype='send' n=5 len=64                                                      2.32 %       ±2.83% ±3.80%  ±5.04%
http/http_server_for_chunky_client.jstype='send' n=5 len=8                                                       0.78 %       ±0.81% ±1.08%  ±1.41%
http/http_server_for_chunky_client.jstype='send' n=50 len=1                                                     -0.00 %       ±1.73% ±2.32%  ±3.05%
http/http_server_for_chunky_client.jstype='send' n=50 len=128                                                    0.37 %       ±0.90% ±1.20%  ±1.56%
http/http_server_for_chunky_client.jstype='send' n=50 len=16                                              *      1.12 %       ±0.92% ±1.23%  ±1.60%
http/http_server_for_chunky_client.jstype='send' n=50 len=32                                                     0.35 %       ±0.75% ±1.00%  ±1.30%
http/http_server_for_chunky_client.jstype='send' n=50 len=4                                                     -0.48 %       ±0.97% ±1.29%  ±1.67%
http/http_server_for_chunky_client.jstype='send' n=50 len=64                                              *      2.07 %       ±2.03% ±2.73%  ±3.60%
http/http_server_for_chunky_client.jstype='send' n=50 len=8                                                     -0.01 %       ±0.63% ±0.84%  ±1.09%
http/http_server_for_chunky_client.jstype='send' n=500 len=1                                                     0.23 %       ±0.79% ±1.05%  ±1.37%
http/http_server_for_chunky_client.jstype='send' n=500 len=128                                                   0.27 %       ±1.01% ±1.35%  ±1.75%
http/http_server_for_chunky_client.jstype='send' n=500 len=16                                                    0.62 %       ±0.82% ±1.09%  ±1.42%
http/http_server_for_chunky_client.jstype='send' n=500 len=32                                             *      1.17 %       ±1.03% ±1.37%  ±1.80%
http/http_server_for_chunky_client.jstype='send' n=500 len=4                                                    -0.04 %       ±0.62% ±0.82%  ±1.07%
http/http_server_for_chunky_client.jstype='send' n=500 len=64                                                    0.26 %       ±1.03% ±1.38%  ±1.80%
http/http_server_for_chunky_client.jstype='send' n=500 len=8                                                    -0.13 %       ±0.88% ±1.16%  ±1.52%
http/incoming_headers.jsduration=5 w=0 headers=20 connections=50 benchmarker='wrk'                      ***      4.25 %       ±2.23% ±2.97%  ±3.86%
http/incoming_headers.jsduration=5 w=6 headers=20 connections=50 benchmarker='wrk'                      ***      4.33 %       ±2.23% ±2.96%  ±3.86%
http/set_header.jsn=1000000 value='Connection'                                                                   0.98 %       ±2.02% ±2.71%  ±3.57%
http/set_header.jsn=1000000 value='Content-Length'                                                              -0.18 %       ±0.65% ±0.86%  ±1.12%
http/set_header.jsn=1000000 value='Content-Type'                                                                 0.25 %       ±2.16% ±2.89%  ±3.77%
http/set_header.jsn=1000000 value='Set-Cookie'                                                                  -0.51 %       ±1.06% ±1.42%  ±1.86%
http/set_header.jsn=1000000 value='Transfer-Encoding'                                                           -0.14 %       ±0.61% ±0.82%  ±1.06%
http/set_header.jsn=1000000 value='Vary'                                                                  *      1.19 %       ±0.96% ±1.28%  ±1.67%
http/set_header.jsn=1000000 value='X-Powered-By'                                                                -0.18 %       ±0.85% ±1.14%  ±1.48%
http/set-header.jsduration=5 res='normal' benchmarker='wrk'                                             ***      3.48 %       ±1.70% ±2.26%  ±2.95%
http/set-header.jsduration=5 res='setHeader' benchmarker='wrk'                                          ***      3.27 %       ±1.83% ±2.43%  ±3.16%
http/set-header.jsduration=5 res='setHeaderWH' benchmarker='wrk'                                          *      1.84 %       ±1.74% ±2.32%  ±3.02%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=1 len=1024 type='buffer' benchmarker='wrk'             **      2.33 %       ±1.52% ±2.03%  ±2.64%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=1 len=1024 type='bytes' benchmarker='wrk'              **      2.75 %       ±1.92% ±2.55%  ±3.32%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=1 len=102400 type='buffer' benchmarker='wrk'          ***      3.28 %       ±1.57% ±2.09%  ±2.72%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=1 len=102400 type='bytes' benchmarker='wrk'                    0.37 %       ±0.90% ±1.20%  ±1.56%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=1 len=4 type='buffer' benchmarker='wrk'                        1.85 %       ±1.86% ±2.47%  ±3.21%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=1 len=4 type='bytes' benchmarker='wrk'                         1.73 %       ±1.84% ±2.45%  ±3.19%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=4 len=1024 type='buffer' benchmarker='wrk'             **      3.32 %       ±1.94% ±2.58%  ±3.37%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=4 len=1024 type='bytes' benchmarker='wrk'              **      2.87 %       ±1.87% ±2.49%  ±3.25%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=4 len=102400 type='buffer' benchmarker='wrk'                   0.88 %       ±1.67% ±2.23%  ±2.92%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=4 len=102400 type='bytes' benchmarker='wrk'            **      1.38 %       ±0.90% ±1.20%  ±1.56%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=4 len=4 type='buffer' benchmarker='wrk'               ***      3.70 %       ±1.97% ±2.62%  ±3.42%
http/simple.jsduration=5 chunkedEnc=0 c=50 chunks=4 len=4 type='bytes' benchmarker='wrk'                ***      4.32 %       ±1.76% ±2.34%  ±3.05%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=1 len=1024 type='buffer' benchmarker='wrk'           ***      3.23 %       ±1.84% ±2.45%  ±3.19%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=1 len=1024 type='bytes' benchmarker='wrk'            ***      3.93 %       ±1.67% ±2.23%  ±2.90%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=1 len=102400 type='buffer' benchmarker='wrk'         ***      2.46 %       ±1.37% ±1.83%  ±2.38%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=1 len=102400 type='bytes' benchmarker='wrk'            *     -2.39 %       ±1.88% ±2.51%  ±3.26%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=1 len=4 type='buffer' benchmarker='wrk'                *      2.29 %       ±1.75% ±2.34%  ±3.05%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=1 len=4 type='bytes' benchmarker='wrk'                 *      1.85 %       ±1.57% ±2.09%  ±2.72%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=4 len=1024 type='buffer' benchmarker='wrk'                    0.26 %       ±1.57% ±2.08%  ±2.72%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=4 len=1024 type='bytes' benchmarker='wrk'            ***      2.72 %       ±1.34% ±1.79%  ±2.33%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=4 len=102400 type='buffer' benchmarker='wrk'                  0.90 %       ±1.54% ±2.05%  ±2.68%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=4 len=102400 type='bytes' benchmarker='wrk'                   0.26 %       ±0.87% ±1.16%  ±1.51%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=4 len=4 type='buffer' benchmarker='wrk'               **      2.02 %       ±1.49% ±1.98%  ±2.57%
http/simple.jsduration=5 chunkedEnc=0 c=500 chunks=4 len=4 type='bytes' benchmarker='wrk'               ***      3.41 %       ±1.53% ±2.04%  ±2.66%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=1 len=1024 type='buffer' benchmarker='wrk'            ***      2.84 %       ±1.55% ±2.07%  ±2.70%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=1 len=1024 type='bytes' benchmarker='wrk'                      1.72 %       ±1.91% ±2.54%  ±3.31%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=1 len=102400 type='buffer' benchmarker='wrk'          ***      2.98 %       ±1.46% ±1.95%  ±2.54%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=1 len=102400 type='bytes' benchmarker='wrk'             *      0.92 %       ±0.72% ±0.96%  ±1.26%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=1 len=4 type='buffer' benchmarker='wrk'                **      2.46 %       ±1.69% ±2.25%  ±2.93%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=1 len=4 type='bytes' benchmarker='wrk'                 **      2.96 %       ±1.81% ±2.41%  ±3.14%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=4 len=1024 type='buffer' benchmarker='wrk'              *      1.62 %       ±1.50% ±2.00%  ±2.60%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=4 len=1024 type='bytes' benchmarker='wrk'             ***      2.83 %       ±1.50% ±1.99%  ±2.59%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=4 len=102400 type='buffer' benchmarker='wrk'            *      1.81 %       ±1.40% ±1.86%  ±2.42%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=4 len=102400 type='bytes' benchmarker='wrk'            **      1.30 %       ±0.95% ±1.26%  ±1.64%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=4 len=4 type='buffer' benchmarker='wrk'                **      2.31 %       ±1.46% ±1.94%  ±2.53%
http/simple.jsduration=5 chunkedEnc=1 c=50 chunks=4 len=4 type='bytes' benchmarker='wrk'                 **      2.31 %       ±1.50% ±2.00%  ±2.60%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=1 len=1024 type='buffer' benchmarker='wrk'                    0.96 %       ±1.22% ±1.62%  ±2.11%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=1 len=1024 type='bytes' benchmarker='wrk'            ***      3.31 %       ±1.48% ±1.97%  ±2.56%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=1 len=102400 type='buffer' benchmarker='wrk'          **      2.41 %       ±1.55% ±2.06%  ±2.68%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=1 len=102400 type='bytes' benchmarker='wrk'                   0.18 %       ±0.65% ±0.87%  ±1.13%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=1 len=4 type='buffer' benchmarker='wrk'                       1.40 %       ±1.67% ±2.22%  ±2.89%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=1 len=4 type='bytes' benchmarker='wrk'                **      2.71 %       ±1.72% ±2.29%  ±2.99%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=4 len=1024 type='buffer' benchmarker='wrk'           ***      2.06 %       ±1.18% ±1.58%  ±2.05%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=4 len=1024 type='bytes' benchmarker='wrk'            ***      2.52 %       ±1.29% ±1.72%  ±2.23%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=4 len=102400 type='buffer' benchmarker='wrk'           *      1.43 %       ±1.37% ±1.83%  ±2.38%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=4 len=102400 type='bytes' benchmarker='wrk'           **      1.64 %       ±1.03% ±1.37%  ±1.79%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=4 len=4 type='buffer' benchmarker='wrk'               **      2.18 %       ±1.54% ±2.05%  ±2.67%
http/simple.jsduration=5 chunkedEnc=1 c=500 chunks=4 len=4 type='bytes' benchmarker='wrk'                        0.50 %       ±1.37% ±1.82%  ±2.37%
http/upgrade.jsn=1000                                                                                    **      0.64 %       ±0.45% ±0.60%  ±0.78%
http/upgrade.jsn=5                                                                                      ***     -0.81 %       ±0.34% ±0.46%  ±0.59%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 222 comparisons, you can thus
expect the following amount of false-positive results:
  11.10 false positives, when considering a   5% risk acceptance (*, **, ***),
  2.22 false positives, when considering a   1% risk acceptance (**, ***),
  0.22 false positives, when considering a 0.1% risk acceptance (***)

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 15, 2021
@mcollina
Copy link
Member Author

There are definitely more to be reverted, I did not touch the http client, fs, util, and many other subsystems.

@nodejs-github-bot
Copy link
Collaborator

@ronag
Copy link
Member

ronag commented Apr 15, 2021

@nodejs/tsc Should we put a pause on further porting to primordials until we have clear guidelines where they are safely applicable and what the workflow is otherwise?

@mcollina mcollina added the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Apr 15, 2021
@@ -88,7 +72,7 @@ const { CRLF } = common;

const kCorked = Symbol('corked');

const nop = FunctionPrototype;
const nop = function () {};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const nop = function () {};
const nop = function() {};

@targos
Copy link
Member

targos commented Apr 15, 2021

@nodejs/tsc Should we put a pause on further porting to primordials until we have clear guidelines where they are safely applicable and what the workflow is otherwise?

+1. I think we need a volunteer to investigate what needs to be done in V8 to make uncurryThis as fast as direct builtin calls.

@@ -234,7 +249,7 @@ function Writable(options) {
// the WritableState constructor, at least with V8 6.5.
const isDuplex = (this instanceof Stream.Duplex);

if (!isDuplex && !FunctionPrototypeSymbolHasInstance(Writable, this))
if (!isDuplex && !realHasInstance.call(Writable, this))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not strictly the same. I guess this is from the revert?

@@ -132,7 +120,7 @@ const DEFAULT_IPV6_ADDR = '::';

const isWindows = process.platform === 'win32';

const noop = FunctionPrototype;
const noop = function () {};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const noop = function () {};
const noop = function() {};

lib/net.js Show resolved Hide resolved
Copy link
Contributor

@aduh95 aduh95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've learned from #37168 that primordials added some performance benefits to streams, what about those?
I've spent days trying to investigate which changes were "safe" performance-wise, I'm sorry I failed. But I feel that reverting the whole commit is a bit "lazy", I'd prefer if we investigate what are the actual changes that are introducing performance penalty – I reckon that means running loads of HTTP benchmark runs (one for each change), which would take a long time.

I'm putting a "Request change", I'm open to dismiss it if the consensus is that my ask is not reasonable.

@ronag
Copy link
Member

ronag commented Apr 15, 2021

But I feel that reverting the whole commit is a bit "lazy", I'd prefer if we investigate what are the actual changes that are introducing performance penalty – I reckon that means running loads of HTTP benchmark runs (one for each change), which would take a long time.

A valid point. However, even if I did agree (which I'm not sure I do) I think the "lazy" approach is appropriate at this point as we are so close to a v16 release and (I assume?) we want to resolve the primary performance regression before that. @mcollina has already put a huge amount of time into sorting out the performance regressions. Also before we put more energy in primordials, I think it would be a good idea to see what the TSC's preliminary thoughts are about the whole situation.

@mcollina
Copy link
Member Author

Every single run of the http benchmarks is around 9/10 hours on my workstation and I have been working around the clock to reach this stage. Running those for every individual change is unrealistic given the effort required to validate them all.

My analysis is that some of the primordials prevent functions to be inlined and/or optimized as well as their non-primordial counterparts.

After identifying this fact, I took a lazy approach as much as I could and removed them from the "serving http calls hot path".

@aduh95
Copy link
Contributor

aduh95 commented Apr 15, 2021

I've opened an alternative PR (#38248) with a smaller diff. PTAL.

@jasnell
Copy link
Member

jasnell commented Apr 15, 2021

I appreciate the reluctance to revert the entire commit but I think it's the right thing to do here. We can then open new prs that make these changes more incrementally, allowing us to benchmark individual changes as we go.

Copy link
Member Author

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, I only reverted one commit as-is. For the rest, I was driven by a combination of flamegraphs and benchmarks.

ObjectDefineProperty,
Promise,
ReflectApply,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aduh95 the entirety of this file is a critical hot path.

@@ -586,7 +574,7 @@ Socket.prototype._read = function(n) {


Socket.prototype.end = function(data, encoding, callback) {
ReflectApply(stream.Duplex.prototype.end, this, [data, encoding, callback]);
stream.Duplex.prototype.end.call(this, data, encoding, callback);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aduh95 note the difference here. ReflectApply was allocating an additional array.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree using ReflectApply here was a mistake. Although, in most cases I've encountered, replacing it with FunctionPrototypeCall was good enough. I'll update the other PR.

@@ -141,13 +139,13 @@ function slowCases(enc) {
case 4:
if (enc === 'UTF8') return 'utf8';
if (enc === 'ucs2' || enc === 'UCS2') return 'utf16le';
enc = StringPrototypeToLowerCase(`${enc}`);
enc = `${enc}`.toLowerCase();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All those changes where needed because slowCases did not inline anymore.

@@ -22,11 +22,7 @@
'use strict';

const {
ArrayPrototypePushApply,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the ArrayPrototype operations where clearly problematic in my analysis.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised that all of them would be, but I guess if that's the case it's OK to remove them all and try to add them back later if V8 optimize this away.

@@ -625,7 +616,7 @@ function socketOnData(server, socket, parser, state, d) {
function onRequestTimeout(socket) {
socket[kRequestTimeout] = undefined;
// socketOnError has additional logic and will call socket.destroy(err).
ReflectApply(socketOnError, socket, [new ERR_HTTP_REQUEST_TIMEOUT()]);
Reflect.apply(socketOnError, socket, [new ERR_HTTP_REQUEST_TIMEOUT()]);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note fore me, `ReflectApply is not actualy needed here, we can avoid an array allocation.

@@ -645,7 +636,7 @@ function onParserTimeout(server, socket) {
socket.destroy();
}

const noop = FunctionPrototype;
const noop = function() {};
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some good suspicion that using a noop function defined elsewhere is not optimized away. No hard data to prove it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've seen that before. Not sure what causes it but reused noops definitely cause issues.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All noop functions have been replaced by the FunctionPrototype in most of the files I checked :/.

@benjamingr
Copy link
Member

@aduh95 sorry for the (probably silly) question.

Given we can't and probably won't be able to tamper-proof the code - is the primordials work and the (negative) impact on code readability is even worth it at this point?

Like, what actual value are we actually getting from these changed at this point?

@ljharb
Copy link
Member

ljharb commented Apr 15, 2021

@benjamingr why wouldn't we be able to tamper-proof the code, modulo performance concerns? It is entirely achievable for node to be robust against the code it's trying to run, and arguably it's a massive bug that it's never been.

@benjamingr
Copy link
Member

@benjamingr why wouldn't we be able to tamper-proof the code, modulo performance concerns?

It appears that for the foreseeable future big 2-3x performance regressions are going to prevent this from happening to the entire Node.js API. I think there are enough people in the project that feel that a significant performance regression is not worth adding that additional guarantee to the API.

The guarantee is only meaningful to some people (like me) if it actually provides a security guarantee - if it's impossible to provide one providing the pretence of one is actually worse.

It is entirely achievable for node to be robust against the code it's trying to run, and arguably it's a massive bug that it's never been.

Our security model never made such a guarantee. Node does not currently enable (certainly out of the box) running untrusted code. The docs are super explicit about this in places where it might be implied (like the vm module).

@aduh95
Copy link
Contributor

aduh95 commented Apr 15, 2021

Like, what actual value are we actually getting from these changed at this point?

@benjamingr I think identifying which primordials are harmful for performance would help V8 improve their optimization engines, and I'm still hoping Node.js core internals will be tamper-proof one day.
I'd like to point out that use of primordials is sometimes actually beneficial from a performance perspective (E.G.: #36347, and ironically 419686c, which is being reverted here, showed perf improvements), I think we shouldn't throw them all away without a good reason. Since none of those changes are breaking, I don't think we need to rush here, I'd prefer to wait for more data and land only the subset of actually perf-sensitive changes as a semver-patch later.

@mcollina
Copy link
Member Author

I would prefer to not stall this. The whole premise of primordials is to increase safety while not slowing things down. The vast majority of those PR were never benchmarked against http, which is one of our primary use cases.

Seeing a 20% drop in #37937 was definitely a significant surprise. We are still far from getting back on track and the longer we wait the harder it gets, as we risk introducing new bottlenecks without noticing.

@BethGriggs
Copy link
Member

BethGriggs commented Apr 15, 2021

I agree with @mcollina's sentiment of not stalling. I do think the optics of Node.js v16.0.0 going out with a known significant performance regression is relevant. I imagine when a new major release goes out a number of our users will run benchmarks, if we can patch (or revert) any known regressions beforehand then I think it makes sense to do so.

@@ -379,7 +376,7 @@ function howMuchToRead(n, state) {
return 0;
if (state.objectMode)
return 1;
if (NumberIsNaN(n)) {
if (Number.isNaN(n)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any evidence that "static" primordials can lead to performance issues? I'm okay with reverting the prototype ones but this is too much imo.

Copy link
Member

@targos targos Apr 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm asking because NumberIsNan is literally Number.isNaN (same function identity)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't. The approach I used is to remove all of them from hot paths, literally with %s/primordial/replacement/g. If you want I can try adding those back in.

I'm 99.9% sure on the ones from *Prototype* are the major culprit (functions and arrays specifically).

Copy link
Member

@mhdawson mhdawson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mcollina
Copy link
Member Author

Closing in favor of #38248. It is a good compromise and it focus on all the primordials we identified causing the slowdown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. tsc-agenda Issues and PRs to discuss during the meetings of the TSC.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants