Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the results from the faster-greedy-dag extractor #34

Closed
wants to merge 3 commits into from

Conversation

TrevorHansen
Copy link
Contributor

Compared to the current/old faster-greedy-dag extractor, this is a bit slower but produces better results. I think it's mostly luck though, it just visits nodes in a way that for the benchmarks we have produces better results. I don't think there's any theoretical reason it's better.

``
###################################################
faster-greedy-dag-new vs faster-greedy-dag-old

extractors: ['faster-greedy-dag-new', 'faster-greedy-dag-old']
data/babble/physics_scientific_unsolved_4h_ellisk_2019-07-20T18.09.34--bench000_it0.json differs in dag cost: 89 91
data/babble/list_list_hard_test_ellisk_2019-02-15T11.26.41--bench007_it10.json differs in dag cost: 419 418
data/babble/list_list_hard_test_ellisk_2019-02-15T11.43.28--bench007_it7.json differs in dag cost: 298 299
data/babble/list_list_hard_test_ellisk_2019-02-15T11.43.28--bench009_it13.json differs in dag cost: 395 392
data/babble/list_list_hard_test_ellisk_2019-02-15T11.43.28--bench008_it8.json differs in dag cost: 330 329
data/babble/list_list_hard_test_ellisk_2019-02-15T11.43.28--bench011_it19.json differs in dag cost: 364 363
data/babble/list_list_hard_test_ellisk_2019-02-15T11.31.39--bench010_it16.json differs in dag cost: 347 346
data/babble/physics_scientific_unsolved_4h_ellisk_2019-07-20T18.09.34--bench001_it1.json differs in dag cost: 144 143
data/babble/list_list_hard_test_ellisk_2019-02-15T11.31.39--bench009_it11.json differs in dag cost: 367 366
data/babble/list_list_hard_test_ellisk_2019-02-15T11.31.39--bench006_it6.json differs in dag cost: 246 247
data/babble/list_list_hard_test_ellisk_2019-02-15T11.43.28--bench010_it17.json differs in dag cost: 381 380
data/babble/physics_scientific_unsolved_4h_ellisk_2019-07-20T18.05.46--bench004_it4.json differs in dag cost: 183 182
data/babble/physics_scientific_unsolved_4h_ellisk_2019-07-20T18.09.34--bench006_it7.json differs in dag cost: 153 152
data/babble/list_list_hard_test_ellisk_2019-02-15T11.35.48--bench010_it12.json differs in dag cost: 371 369
data/babble/physics_scientific_unsolved_4h_ellisk_2019-07-20T18.16.45--bench001_it1.json differs in dag cost: 126 125
data/babble/list_list_hard_test_ellisk_2019-02-15T11.26.41--bench010_it15.json differs in dag cost: 427 426
data/babble/list_list_hard_test_ellisk_2019-02-15T11.35.48--bench005_it5.json differs in dag cost: 245 244
data/babble/list_list_hard_test_ellisk_2019-02-15T11.43.28--bench005_it5.json differs in dag cost: 231 232
data/babble/list_list_hard_test_ellisk_2019-02-15T11.39.19--bench004_it5.json differs in dag cost: 235 234
data/babble/physics_scientific_unsolved_4h_ellisk_2019-07-20T18.09.34--bench004_it4.json differs in dag cost: 122 120
data/babble/physics_scientific_unsolved_4h_ellisk_2019-07-20T18.09.34--bench005_it5.json differs in dag cost: 141 140
data/flexc/67addae9-f7db-4ebb-a02e-e1e045f87fc4.json differs in dag cost: 37 36
data/flexc/4a29e9e7-a232-4608-b7d2-37796d0ecd97.json differs in dag cost: 37 36
data/flexc/03d369d0-bb23-477a-8ea1-a452efc850fe.json differs in dag cost: 37 36
data/flexc/db344dbd-fb20-4e9e-badb-076d7a3809bc.json differs in dag cost: 37 36
data/flexc/86d12271-435d-463d-a7b4-e671423f0ced.json differs in dag cost: 37 36
data/tensat/nasneta.json differs in dag cost: 16.374893040569077 16.322293040515433
data/tensat/nasneta_acyclic.json differs in dag cost: 16.35845509585488 16.301627096203447
data/egg/math_simplify_factor.json differs in dag cost: 6 5
data/fuzz/14.json differs in dag cost: 114.32003810419093 124.07667387412016
data/fuzz/15.json differs in dag cost: 134.39646038194644 160.99453847121393
data/fuzz/1.json differs in dag cost: 108.96027900550405 143.4529619974897
data/fuzz/6.json differs in dag cost: 97.14818527661234 137.76199410064035
data/fuzz/5.json differs in dag cost: 46.30896318297725 54.138384368771
data/fuzz/12.json differs in dag cost: 115.532456438086 178.68563937989165
data/rover/box_filter_3iteration_egraph.json differs in dag cost: 1701 1819
cumulative time for faster-greedy-dag-new: 3585ms
cumulative time for faster-greedy-dag-old: 3180ms
cumulative tree cost for faster-greedy-dag-new: 32017750419805
cumulative tree cost for faster-greedy-dag-old: 32017750420420
cumulative dag cost for faster-greedy-dag-new: 79973
cumulative dag cost for faster-greedy-dag-old: 80251
Cumulative time for faster-greedy-dag-new: 3585ms
Cumulative time for faster-greedy-dag-old: 3180ms
faster-greedy-dag-new / faster-greedy-dag-old
geo mean
tree: 0.9907
dag: 0.9960
micros: 1.2354
quantiles
tree: 0.3535, 1.0000, 1.0000, 1.0000, 1.1419
dag: 0.6466, 1.0000, 1.0000, 1.0000, 1.2000
micros: 0.4228, 1.1455, 1.1981, 1.3196, 3.3375

@oflatt
Copy link
Member

oflatt commented Jan 6, 2024

This is an interesting experiment- not sure if we should merge it since we don't have a reason why it's better
Maybe you could merge it as faster-greedy-dag-alt? Or just keep this a draft.

@mwillsey
Copy link
Member

Yeah I think we close this for now, unless something changes and makes it better. Fewer more distinct extractors seems like the way to go for now. Happy to re-open if other feel differently.

@mwillsey mwillsey closed this Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants