Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

force capture backtraces on VortexErrors #873

Closed
wants to merge 1 commit into from
Closed

Conversation

lwwmanning
Copy link
Member

No description provided.

@lwwmanning lwwmanning added the benchmark Run benchmarks on this branch label Sep 18, 2024
@github-actions github-actions bot removed the benchmark Run benchmarks on this branch label Sep 18, 2024
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vortex random_access

Benchmark suite Current: 8edba1d Previous: f14f45a Ratio
vortex/tokio local disk 1469087 ns/iter (± 35878) 1204308 ns/iter (± 40526) 1.22
vortex/localfs 1643926 ns/iter (± 19694) 1363453 ns/iter (± 28062) 1.21
parquet/tokio local disk 181900558 ns/iter (± 1244452) 178573668 ns/iter (± 4407348) 1.02

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vortex bytes_at

Benchmark suite Current: 8edba1d Previous: f14f45a Ratio
bytes_at/array_data 603 ns/iter (± 0) 603 ns/iter (± 1) 1
bytes_at/array_data #2 1042 ns/iter (± 2) 1021 ns/iter (± 18) 1.02

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vortex DataFusion

Benchmark suite Current: 8edba1d Previous: f14f45a Ratio
arrow/planning 803004 ns/iter (± 23465) 801394 ns/iter (± 12474) 1.00
arrow/exec 1751464 ns/iter (± 35430) 1754329 ns/iter (± 4485) 1.00
vortex-pushdown-compressed/planning 510597 ns/iter (± 1556) 507344 ns/iter (± 1752) 1.01
vortex-pushdown-compressed/exec 3261507 ns/iter (± 18693) 3046599 ns/iter (± 26784) 1.07
vortex-pushdown-uncompressed/planning 513656 ns/iter (± 2727) 506905 ns/iter (± 2326) 1.01
vortex-pushdown-uncompressed/exec 2947065 ns/iter (± 39065) 2919791 ns/iter (± 14408) 1.01
vortex-nopushdown-compressed/planning 706981 ns/iter (± 2375) 707699 ns/iter (± 1898) 1.00
vortex-nopushdown-compressed/exec 8725345 ns/iter (± 208203) 14350777 ns/iter (± 266929) 0.61
vortex-nopushdown-uncompressed/planning 705864 ns/iter (± 2257) 709099 ns/iter (± 2558) 1.00
vortex-nopushdown-uncompressed/exec 1984771 ns/iter (± 4055) 1990897 ns/iter (± 5181) 1.00

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vortex Compression

Benchmark suite Current: 8edba1d Previous: f14f45a Ratio
Yellow Taxi Trip Data/compress 160164788 ns/iter (± 629695) 140163042 ns/iter (± 801888) 1.14
Public BI Benchmark/AirlineSentiment 821750 ns/iter (± 3163) 511415 ns/iter (± 222) 1.61
Public BI Benchmark/Arade 4328607119 ns/iter (± 21603278) 3761025905 ns/iter (± 5143244) 1.15
Public BI Benchmark/Euro2016 3110258399 ns/iter (± 48767599) 2860438858 ns/iter (± 39993075) 1.09
Public BI Benchmark/Food 1540535144 ns/iter (± 5675577) 1369332280 ns/iter (± 5333366) 1.13
Public BI Benchmark/HashTags 4515505195 ns/iter (± 39998271) 4063545682 ns/iter (± 24055373) 1.11
TPCH l_comment Column/compress-default 252339869 ns/iter (± 4010453) 240363739 ns/iter (± 692055) 1.05
TPCH l_comment Column/compress-fsst-chunked 1144070379 ns/iter (± 13773733) 1098840233 ns/iter (± 12999438) 1.04
TPCH l_comment Column/compress-fsst-canonicalized 708543339 ns/iter (± 7789746) 700600384 ns/iter (± 6058583) 1.01

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vortex benchmarks

Benchmark suite Current: 8edba1d Previous: f14f45a Ratio
tpch_q1/vortex-in-memory-no-pushdown 468722611 ns/iter (± 4458226) 461879688 ns/iter (± 3077475) 1.01
tpch_q1/vortex-in-memory-pushdown 535868222 ns/iter (± 1592434) 523149740 ns/iter (± 3032183) 1.02
tpch_q1/arrow 451231514 ns/iter (± 2704100) 447032260 ns/iter (± 3098751) 1.01
tpch_q1/parquet 644778333 ns/iter (± 1707679) 648948151 ns/iter (± 7956680) 0.99
tpch_q1/vortex-file-compressed 618149896 ns/iter (± 2921956) 610890208 ns/iter (± 9958849) 1.01
tpch_q1/vortex-file-uncompressed 659428244 ns/iter (± 3932569) 632439600 ns/iter (± 5723978) 1.04
tpch_q2/vortex-in-memory-no-pushdown 144567348 ns/iter (± 2293852) 148541579 ns/iter (± 4302659) 0.97
tpch_q2/vortex-in-memory-pushdown 144653781 ns/iter (± 2252999) 150887621 ns/iter (± 4453065) 0.96
tpch_q2/arrow 122038218 ns/iter (± 1470443) 124031546 ns/iter (± 2202920) 0.98
tpch_q2/parquet 157482672 ns/iter (± 1385404) 158896173 ns/iter (± 2073068) 0.99
tpch_q2/vortex-file-compressed 155695785 ns/iter (± 1246366) 156750455 ns/iter (± 1674197) 0.99
tpch_q2/vortex-file-uncompressed 164231704 ns/iter (± 2366432) 163293598 ns/iter (± 1190892) 1.01
tpch_q3/vortex-in-memory-no-pushdown 150265290 ns/iter (± 1135946) 149693356 ns/iter (± 2796232) 1.00
tpch_q3/vortex-in-memory-pushdown 207784382 ns/iter (± 3420327) 192024654 ns/iter (± 7333517) 1.08
tpch_q3/arrow 145445470 ns/iter (± 2020510) 151173886 ns/iter (± 2769794) 0.96
tpch_q3/parquet 334114234 ns/iter (± 3091192) 342704692 ns/iter (± 5812958) 0.97
tpch_q3/vortex-file-compressed 323099504 ns/iter (± 2826267) 286885933 ns/iter (± 2929862) 1.13
tpch_q3/vortex-file-uncompressed 380957971 ns/iter (± 1190366) 356609340 ns/iter (± 2092025) 1.07
tpch_q4/vortex-in-memory-no-pushdown 107526861 ns/iter (± 1011814) 107185765 ns/iter (± 1077663) 1.00
tpch_q4/vortex-in-memory-pushdown 181696641 ns/iter (± 1895324) 143524344 ns/iter (± 2500008) 1.27
tpch_q4/arrow 102658538 ns/iter (± 1408107) 107655854 ns/iter (± 1319134) 0.95
tpch_q4/parquet 217336842 ns/iter (± 2389557) 222921259 ns/iter (± 2550700) 0.97
tpch_q4/vortex-file-compressed 330680705 ns/iter (± 3029888) 268313587 ns/iter (± 4793722) 1.23
tpch_q4/vortex-file-uncompressed 399638999 ns/iter (± 4448234) 314603548 ns/iter (± 2273681) 1.27
tpch_q5/vortex-in-memory-no-pushdown 304292049 ns/iter (± 3987215) 302192759 ns/iter (± 4110096) 1.01
tpch_q5/vortex-in-memory-pushdown 318067177 ns/iter (± 4258473) 311208286 ns/iter (± 3549608) 1.02
tpch_q5/arrow 292257787 ns/iter (± 1624087) 296813796 ns/iter (± 2263770) 0.98
tpch_q5/parquet 454575106 ns/iter (± 6055950) 448221090 ns/iter (± 4376687) 1.01
tpch_q5/vortex-file-compressed 360703454 ns/iter (± 4386153) 348447719 ns/iter (± 2972916) 1.04
tpch_q5/vortex-file-uncompressed 359782815 ns/iter (± 4311171) 349344160 ns/iter (± 2129832) 1.03
tpch_q6/vortex-in-memory-no-pushdown 38643539 ns/iter (± 305629) 38066285 ns/iter (± 864563) 1.02
tpch_q6/vortex-in-memory-pushdown 143189162 ns/iter (± 603843) 91880919 ns/iter (± 383334) 1.56
tpch_q6/arrow 36665151 ns/iter (± 137750) 34761982 ns/iter (± 481222) 1.05
tpch_q6/parquet 151824241 ns/iter (± 938852) 148576284 ns/iter (± 1296662) 1.02
tpch_q6/vortex-file-compressed 116031682 ns/iter (± 628186) 77163514 ns/iter (± 769309) 1.50
tpch_q6/vortex-file-uncompressed 243537906 ns/iter (± 3627875) 169717128 ns/iter (± 5789579) 1.43
tpch_q7/vortex-in-memory-no-pushdown 568195544 ns/iter (± 7727507) 556393001 ns/iter (± 23698489) 1.02
tpch_q7/vortex-in-memory-pushdown 644780604 ns/iter (± 10103017) 599744491 ns/iter (± 3301311) 1.08
tpch_q7/arrow 562817606 ns/iter (± 3962822) 539553105 ns/iter (± 6411940) 1.04
tpch_q7/parquet 708509324 ns/iter (± 6933911) 705749956 ns/iter (± 11950297) 1.00
tpch_q7/vortex-file-compressed 707022780 ns/iter (± 5588372) 661269547 ns/iter (± 7820481) 1.07
tpch_q7/vortex-file-uncompressed 784502676 ns/iter (± 10618550) 717390415 ns/iter (± 3812166) 1.09
tpch_q8/vortex-in-memory-no-pushdown 213596616 ns/iter (± 982441) 214106345 ns/iter (± 740542) 1.00
tpch_q8/vortex-in-memory-pushdown 231806171 ns/iter (± 791307) 226319703 ns/iter (± 3484983) 1.02
tpch_q8/arrow 208378502 ns/iter (± 611642) 214715994 ns/iter (± 3771551) 0.97
tpch_q8/parquet 478408148 ns/iter (± 1759347) 473554749 ns/iter (± 2538560) 1.01
tpch_q8/vortex-file-compressed 282214943 ns/iter (± 1429622) 266486592 ns/iter (± 1857795) 1.06
tpch_q8/vortex-file-uncompressed 307754869 ns/iter (± 5681802) 291502758 ns/iter (± 5980310) 1.06
tpch_q9/vortex-in-memory-no-pushdown 401055993 ns/iter (± 2274128) 395192675 ns/iter (± 1943894) 1.01
tpch_q9/vortex-in-memory-pushdown 410943315 ns/iter (± 3992879) 397415770 ns/iter (± 834027) 1.03
tpch_q9/arrow 400176521 ns/iter (± 1682121) 389569639 ns/iter (± 7320816) 1.03
tpch_q9/parquet 691538144 ns/iter (± 3262634) 684888505 ns/iter (± 7376860) 1.01
tpch_q9/vortex-file-compressed 457110253 ns/iter (± 1353643) 438258368 ns/iter (± 7598713) 1.04
tpch_q9/vortex-file-uncompressed 477640071 ns/iter (± 2200227) 480955306 ns/iter (± 5038800) 0.99
tpch_q10/vortex-in-memory-no-pushdown 224554428 ns/iter (± 994723) 226046551 ns/iter (± 2595417) 0.99
tpch_q10/vortex-in-memory-pushdown 277229096 ns/iter (± 846112) 261828979 ns/iter (± 1723691) 1.06
tpch_q10/arrow 217551821 ns/iter (± 2396551) 214356978 ns/iter (± 549396) 1.01
tpch_q10/parquet 471337335 ns/iter (± 924028) 471864858 ns/iter (± 3960785) 1.00
tpch_q10/vortex-file-compressed 478818214 ns/iter (± 4711223) 452272980 ns/iter (± 5087412) 1.06
tpch_q10/vortex-file-uncompressed 413096876 ns/iter (± 2334191) 389063182 ns/iter (± 7024937) 1.06
tpch_q11/vortex-in-memory-no-pushdown 221002106 ns/iter (± 2983756) 215392172 ns/iter (± 5359998) 1.03
tpch_q11/vortex-in-memory-pushdown 221353417 ns/iter (± 1684828) 216949528 ns/iter (± 2550297) 1.02
tpch_q11/arrow 176362483 ns/iter (± 2902205) 176409366 ns/iter (± 4702977) 1.00
tpch_q11/parquet 183409567 ns/iter (± 1212413) 190107370 ns/iter (± 6890211) 0.96
tpch_q11/vortex-file-compressed 226881798 ns/iter (± 2248967) 224448436 ns/iter (± 3024412) 1.01
tpch_q11/vortex-file-uncompressed 233776851 ns/iter (± 886686) 231659525 ns/iter (± 4099349) 1.01
tpch_q12/vortex-in-memory-no-pushdown 176587089 ns/iter (± 644986) 177740634 ns/iter (± 1683642) 0.99
tpch_q12/vortex-in-memory-pushdown 364527834 ns/iter (± 837970) 254860784 ns/iter (± 1161144) 1.43
tpch_q12/arrow 165494977 ns/iter (± 367103) 165273140 ns/iter (± 961182) 1.00
tpch_q12/parquet 350743619 ns/iter (± 2436743) 352068445 ns/iter (± 2636228) 1.00
tpch_q12/vortex-file-compressed 744300974 ns/iter (± 3806271) 575933257 ns/iter (± 5820825) 1.29
tpch_q12/vortex-file-uncompressed 564677123 ns/iter (± 2268264) 351856000 ns/iter (± 2918588) 1.60
tpch_q13/vortex-in-memory-no-pushdown 178681432 ns/iter (± 5287049) 164279523 ns/iter (± 6866203) 1.09
tpch_q13/vortex-in-memory-pushdown 178497069 ns/iter (± 2729115) 163140660 ns/iter (± 14031464) 1.09
tpch_q13/arrow 172602983 ns/iter (± 4597348) 160955641 ns/iter (± 17729929) 1.07
tpch_q13/parquet 318650645 ns/iter (± 3654112) 305841673 ns/iter (± 3061812) 1.04
tpch_q13/vortex-file-compressed 210493951 ns/iter (± 1769319) 206909714 ns/iter (± 1734778) 1.02
tpch_q13/vortex-file-uncompressed 214883191 ns/iter (± 1446720) 214283652 ns/iter (± 4051306) 1.00
tpch_q14/vortex-in-memory-no-pushdown 38382491 ns/iter (± 276243) 40431818 ns/iter (± 592849) 0.95
tpch_q14/vortex-in-memory-pushdown 116488606 ns/iter (± 850447) 88516398 ns/iter (± 892898) 1.32
tpch_q14/arrow 39600800 ns/iter (± 247229) 39784495 ns/iter (± 434423) 1.00
tpch_q14/parquet 220597817 ns/iter (± 856744) 222673183 ns/iter (± 1380481) 0.99
tpch_q14/vortex-file-compressed 112986572 ns/iter (± 409773) 88214842 ns/iter (± 258223) 1.28
tpch_q14/vortex-file-uncompressed 185444651 ns/iter (± 2176645) 144920148 ns/iter (± 2027347) 1.28
tpch_q15/vortex-in-memory-no-pushdown 66555313 ns/iter (± 925093) 69925514 ns/iter (± 379219) 0.95
tpch_q15/vortex-in-memory-pushdown 180991925 ns/iter (± 1408993) 124757304 ns/iter (± 521822) 1.45
tpch_q15/arrow 65366018 ns/iter (± 432113) 66342686 ns/iter (± 690736) 0.99
tpch_q15/parquet 294355062 ns/iter (± 4440182) 289717873 ns/iter (± 2569075) 1.02
tpch_q15/vortex-file-compressed 207508435 ns/iter (± 4183337) 152168506 ns/iter (± 1408894) 1.36
tpch_q15/vortex-file-uncompressed 351597450 ns/iter (± 3259938) 276373214 ns/iter (± 11059601) 1.27
tpch_q16/vortex-in-memory-no-pushdown 121628925 ns/iter (± 255924) 119345226 ns/iter (± 593243) 1.02
tpch_q16/vortex-in-memory-pushdown 125920601 ns/iter (± 220444) 124708820 ns/iter (± 556574) 1.01
tpch_q16/arrow 106241630 ns/iter (± 517466) 106797434 ns/iter (± 1502989) 0.99
tpch_q16/parquet 122949469 ns/iter (± 445958) 121776100 ns/iter (± 714772) 1.01
tpch_q16/vortex-file-compressed 140117009 ns/iter (± 810869) 137711153 ns/iter (± 668423) 1.02
tpch_q16/vortex-file-uncompressed 138835537 ns/iter (± 700072) 136383860 ns/iter (± 508263) 1.02
tpch_q17/vortex-in-memory-no-pushdown 689808707 ns/iter (± 11481383) 664879117 ns/iter (± 13393600) 1.04
tpch_q17/vortex-in-memory-pushdown 701145392 ns/iter (± 20544756) 661808705 ns/iter (± 12848360) 1.06
tpch_q17/arrow 604710264 ns/iter (± 12847499) 550112729 ns/iter (± 8934105) 1.10
tpch_q17/parquet 588389436 ns/iter (± 3863208) 587249827 ns/iter (± 4112007) 1.00
tpch_q17/vortex-file-compressed 611723028 ns/iter (± 3329361) 639554678 ns/iter (± 16936248) 0.96
tpch_q17/vortex-file-uncompressed 670045726 ns/iter (± 6416306) 683183788 ns/iter (± 17491683) 0.98
tpch_q18/vortex-in-memory-no-pushdown 1045122379 ns/iter (± 6917747) 1011255791 ns/iter (± 9664724) 1.03
tpch_q18/vortex-in-memory-pushdown 1045077548 ns/iter (± 6558120) 1016027714 ns/iter (± 8122498) 1.03
tpch_q18/arrow 1038409702 ns/iter (± 6566618) 1051209569 ns/iter (± 36763090) 0.99
tpch_q18/parquet 1214259674 ns/iter (± 6493039) 1170968312 ns/iter (± 14508995) 1.04
tpch_q18/vortex-file-compressed 1088240543 ns/iter (± 7684691) 1161414580 ns/iter (± 11105024) 0.94
tpch_q18/vortex-file-uncompressed 1123290150 ns/iter (± 16150123) 1205800514 ns/iter (± 10444648) 0.93
tpch_q19/vortex-in-memory-no-pushdown 164046816 ns/iter (± 453870) 163337569 ns/iter (± 231117) 1.00
tpch_q19/vortex-in-memory-pushdown 326589302 ns/iter (± 2479475) 257806045 ns/iter (± 3217667) 1.27
tpch_q19/arrow 150626265 ns/iter (± 257982) 149829086 ns/iter (± 349165) 1.01
tpch_q19/parquet 468247718 ns/iter (± 1288789) 465770389 ns/iter (± 2920116) 1.01
tpch_q19/vortex-file-compressed 784108503 ns/iter (± 1100231) 714911338 ns/iter (± 3433738) 1.10
tpch_q19/vortex-file-uncompressed 462045264 ns/iter (± 3703691) 364227394 ns/iter (± 2508336) 1.27
tpch_q20/vortex-in-memory-no-pushdown 264513537 ns/iter (± 2777568) 249184954 ns/iter (± 1711428) 1.06
tpch_q20/vortex-in-memory-pushdown 324342477 ns/iter (± 3575267) 279440275 ns/iter (± 1770692) 1.16
tpch_q20/arrow 249282269 ns/iter (± 1865288) 237107149 ns/iter (± 1116060) 1.05
tpch_q20/parquet 364169576 ns/iter (± 2736872) 355145951 ns/iter (± 1872562) 1.03
tpch_q20/vortex-file-compressed 356891449 ns/iter (± 4929291) 309752390 ns/iter (± 1959280) 1.15
tpch_q20/vortex-file-uncompressed 459848056 ns/iter (± 2495365) 400674605 ns/iter (± 4444505) 1.15
tpch_q21/vortex-in-memory-no-pushdown 853803207 ns/iter (± 4625869) 833772887 ns/iter (± 2694226) 1.02
tpch_q21/vortex-in-memory-pushdown 981790578 ns/iter (± 2878622) 890630562 ns/iter (± 3037851) 1.10
tpch_q21/arrow 833668674 ns/iter (± 3332056) 824462975 ns/iter (± 925823) 1.01
tpch_q21/parquet 977028346 ns/iter (± 5781147) 980620646 ns/iter (± 12885923) 1.00
tpch_q21/vortex-file-compressed 1324358428 ns/iter (± 11772777) 1186910300 ns/iter (± 11798799) 1.12
tpch_q21/vortex-file-uncompressed 1459069671 ns/iter (± 11627115) 1323855427 ns/iter (± 12143244) 1.10
tpch_q22/vortex-in-memory-no-pushdown 98324695 ns/iter (± 792277) 99183298 ns/iter (± 1007675) 0.99
tpch_q22/vortex-in-memory-pushdown 98294055 ns/iter (± 672989) 98745983 ns/iter (± 833551) 1.00
tpch_q22/arrow 66700091 ns/iter (± 428636) 66089268 ns/iter (± 588688) 1.01
tpch_q22/parquet 95918819 ns/iter (± 821693) 101936791 ns/iter (± 1278450) 0.94
tpch_q22/vortex-file-compressed 102124223 ns/iter (± 533987) 110140769 ns/iter (± 1661515) 0.93
tpch_q22/vortex-file-uncompressed 109169611 ns/iter (± 643131) 120238517 ns/iter (± 841608) 0.91

This comment was automatically generated by workflow using github-action-benchmark.

@a10y
Copy link
Contributor

a10y commented Sep 18, 2024

What's the difference vs not forced?

@a10y
Copy link
Contributor

a10y commented Sep 18, 2024

Ah ok so it just overrides the var and always captures. This feels like a decision we might not want to make for library users?

@lwwmanning
Copy link
Member Author

Yeah, I was curious to see if there would be a perf diff if we did this for our errors. Appears to slow down TPC-H by a chunk, so not worth it

@lwwmanning lwwmanning closed this Sep 19, 2024
@lwwmanning lwwmanning deleted the wm/backtrace branch September 19, 2024 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants