Skip to content

Improve benchmark test image - #9831

Merged
hugovk merged 2 commits into
python-pillow:mainfrom
akx:better-benchmark-image
Aug 9, 2026
Merged

Improve benchmark test image#9831
hugovk merged 2 commits into
python-pillow:mainfrom
akx:better-benchmark-image

Conversation

@akx

@akx akx commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

The old image was highly channel-correlated, low-color, low-structure and non-representative of anything real. My bad in #9654 :)

The new image contains both low-energy, high-energy (full noise) and zero-energy segments.
If someone has an idea for an even better synthetic image, I'm all ears!

Let's see how the Codspeed numbers change - at least locally, test_quantize and test_histogram got a lot slower, because they actually need to do work 😄

Old New
old new2

@codspeed-hq

codspeed-hq Bot commented Jul 29, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 12 untouched benchmarks
🆕 333 new benchmarks
⏩ 335 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
🆕 test_allocate[1237x811-L] N/A 544.7 µs N/A
🆕 test_allocate[1237x811-LA] N/A 4.1 ms N/A
🆕 test_allocate[1237x811-RGB] N/A 4.1 ms N/A
🆕 test_allocate[1237x811-RGBA] N/A 4.1 ms N/A
🆕 test_alpha_composite[1237x811-LA-mixed] N/A 25.6 ms N/A
🆕 test_alpha_composite[1237x811-LA-opaque] N/A 25.6 ms N/A
🆕 test_alpha_composite[1237x811-LA-transparent] N/A 9.2 ms N/A
🆕 test_alpha_composite[1237x811-RGBA-mixed] N/A 25.6 ms N/A
🆕 test_alpha_composite[1237x811-RGBA-opaque] N/A 25.6 ms N/A
🆕 test_alpha_composite[1237x811-RGBA-transparent] N/A 9.2 ms N/A
🆕 test_blend[1237x811-L] N/A 2.7 ms N/A
🆕 test_blend[1237x811-LA] N/A 10.5 ms N/A
🆕 test_blend[1237x811-RGB] N/A 10.5 ms N/A
🆕 test_blend[1237x811-RGBA] N/A 10.5 ms N/A
🆕 test_box_blur[1237x811-L-1] N/A 24 ms N/A
🆕 test_box_blur[1237x811-L-10] N/A 24 ms N/A
🆕 test_box_blur[1237x811-L-30] N/A 23.9 ms N/A
🆕 test_box_blur[1237x811-LA-1] N/A 54.9 ms N/A
🆕 test_box_blur[1237x811-LA-10] N/A 55.1 ms N/A
🆕 test_box_blur[1237x811-LA-30] N/A 55.2 ms N/A
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing akx:better-benchmark-image (854990f) with main (bdfa0e2)2

Open in CodSpeed

Footnotes

  1. 335 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on main (204f2e9) during the generation of this report, so bdfa0e2 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@akx

akx commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Merging this PR will degrade performance by 29.32%

As predicted! But no, code performance didn't change, just measurements for the more accurate...

@hugovk

hugovk commented Aug 3, 2026

Copy link
Copy Markdown
Member

Good idea. Do you think making the image non-square would have any benefit?

@akx

akx commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

@hugovk Good point, it might. I'm a little afk right now (I'll let you know it's quite warm in Dubrovnik). And maybe also NPOT (non-power-of-2)... Honestly some tests (resizing, etc.) could benefit from running on NPOT and non-NPOT?

@hugovk

hugovk commented Aug 8, 2026

Copy link
Copy Markdown
Member

Yes, sounds good.

akx added 2 commits August 9, 2026 14:40
The old image was (e.g. in the case of palette optimization) pessimal in some cases.
@akx
akx force-pushed the better-benchmark-image branch from 69f73df to 854990f Compare August 9, 2026 12:40
@akx

akx commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

@hugovk Made it NPOT and non-square :)

@hugovk
hugovk merged commit 29d5680 into python-pillow:main Aug 9, 2026
64 of 66 checks passed
@hugovk

hugovk commented Aug 9, 2026

Copy link
Copy Markdown
Member

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants