Create benchmark suite

[shssoichiro]

I'd like to have some benchmarks to compare this with optipng. I'd like to compare compression, speed, and memory usage--cargo bench is probably out of the question, so maybe this can be done in Python or similar.

Ideally this would be reusable to help optimize oxipng as well. Maybe have the benchmarks run against oxipng only by default, and a flag to have it run the comparison benchmarks.

[SethDusek]

Something like this? Note: It isn't complete yet

[shssoichiro]

Yes, something like that is a good start.

[SethDusek]

Another alternative could be an openbenchmarking test, and it could get nice graphs and stuff both compared to optipng, and the improvement in oxipng over a period of months (see http://linuxbenchmarking.com/)

[shssoichiro]

Interesting. This does seem useful. Does it have the option to benchmark aspects other than the running time?

[shssoichiro]

I've created a benchmark suite for cargo. It only tests the running time, and I'd still eventually like to benchmark memory usage, but I think this will suffice for now. It at least gives a baseline for optimizing performance.