Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Qualifying round" optimization #523

Open
Pr0methean opened this issue Jun 21, 2023 · 4 comments
Open

"Qualifying round" optimization #523

Pr0methean opened this issue Jun 21, 2023 · 4 comments
Labels
T-Feature Requests for a new feature to be added T-Performance Relates to slowness in areas of the software or ideas to improve performance

Comments

@Pr0methean
Copy link
Contributor

Pr0methean commented Jun 21, 2023

It would probably improve performance with very little risk of suboptimal compression, if we had the option to discard filtering strategies whose result was e.g. more than 10% and 8KiB larger than the smallest when processed with fast compression, but then use more thorough compression to make the most of a clear winner (sort of like what the Brute strategy does, but applied to the whole image) or choose among those that were within the margin of error. This is especially true when processing a lot of images in parallel, since narrowing down the options would free up threads for subsequent images. The fast compression could also be used to prioritize the most promising filter strategies when under deadline pressure in sequential mode.

@Pr0methean
Copy link
Contributor Author

Pr0methean commented Jun 22, 2023

It's possible this could be implemented as a variant on the fast-evaluation option, where evaluation would be used to select not only reductions but also, subsequently, filter strategies. In my app, Brute is best more often than Bigrams (it's second to None 🤣), so I'd want to have it, Bigrams and None tested with all possible reduction variants, and then have the other 7 each evaluated with whichever reduction(s) had scored best or near-best among the big 3.

@andrews05
Copy link
Collaborator

I'll just note that #509 was recently merged which greatly reduces the losses from fast-evaluation. If you're wanting more speed, I'd definitely recommend using it.

@andrews05 andrews05 added T-Feature Requests for a new feature to be added T-Performance Relates to slowness in areas of the software or ideas to improve performance labels Aug 22, 2023
@andrews05
Copy link
Collaborator

Just to comment further here, this is something I'd like to pursue for a future release (though no promises as to when I'll get to it).

The idea would basically be to combine reduction and filter evaluations into one, with numReductions * numFilters fast trials. Then pick some number of top candidates to run full trials of, with thresholds dependent on the -o level.

@TPS
Copy link

TPS commented Sep 2, 2024

#640 (comment) §2 seems to be a similar idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-Feature Requests for a new feature to be added T-Performance Relates to slowness in areas of the software or ideas to improve performance
Projects
None yet
Development

No branches or pull requests

3 participants