New media-container hides information from user

[zolveuran]

Hi there,

Currently porting my project to using the new pytest-html v4.x version.

I noticed that images (haven't tried any other of the extra types yet) are now put in a media-container that can be scrolled through using small arrow buttons. Previously in v3.x the plots were put in a grid in the right hand side of the test log for each test case.

This is purely my opinion based on my usage and needs for pytest-html. At first I was confused since I initially though only a single plot had been added to the report. Some debugging and staring later I noticed that the rest of the plots were hidden in the carousel.

I discussed it with my colleagues and we agree that the new behavior effectively hides a lot of information from the user/engineer. Whether a test passes or not, a lot of information can be gained at a glance by quickly looking at all the images/plots at the same time. Eg. if we have plots for current, voltage and power consumption, you could easily see whether some condition in test caused a spike in current consumption and/or a voltage drop. This behavior might not be a show-stopper or cause the test to fail (this time around anyway), but it would make the engineers aware of the behavior and either start an investigation now or in the future. It might also save a lot of time during debugging down the road because "someone noticed something" in one of the plots at an earlier time.

We usually scroll through the test cases in the report after a test run and look briefly at the graphs whether a test passed or not. This was also more in line of the "show all details" button, since we aren't shown all the details anymore.

Also, some of the senior engineers and project managers haven't been too happy being sent a .zip with a HTML file, assets folder and a folder full of plots. A nice side effects of the old behavior was that I could quickly save the report as a PDF and send it to them for inspection. While not a full-blown PDF export feature, it did the trick.

Otherwise, super happy with this plugin!

TL;DR: We would want a way to remove the media-container and display all the plots to avoid hiding information in the report.

[BeyondEvil]

Sounds like a reasonable use-case.

Maybe we can make it so that when you click "Show all details", it reveals all the images at the same time - just like in v3.x?

Another option is to have a query-param that loads the report with all images viewable. 🤔