fullscreen-playback-of-switching-sets__ss1-1.html failed in all TV sets in London plugfest

[jpiesing]

In both fullscreen-playback-of-switching-sets__ss1-1.html and ss1-2, the observation "[OF] Video: The presented sample shall match the one reported by the currentTime value within the tolerance of +/-(2/framerate + 20ms)" failed on all 8 TV sets in the London plugfest.

[yanj-github]

Hi @jpiesing,
After change the tolerance to 1frame + 150ms now just failing on one occasion. It is the point when switching happens. We did double check the OF to make sure it uses right frame rate.
Allowed tolerance is 1 frames, 150ms. Time difference between Test Runner reported media currentTime and actual media time exceeded tolerance for following events: currentTime=46.11 time_diff=210.0; Total failure count is 1.

@FritzHeiden can you kindly help to double check the currentTime report at the switching point to make sure it reported correctly please?

[FritzHeiden]

@yanj-github Updating the qr code is coupled to several events. Some of them include: video enters waiting state, currently playing representation changes, video buffers, video starts playing, timeupdate event

Anytime the qr code is updated it uses the currentTime value reported by the API

[jpiesing]

@yanj-github Updating the qr code is coupled to several events. Some of them include: video enters waiting state, currently playing representation changes, video buffers, video starts playing, timeupdate event

Anytime the qr code is updated it uses the currentTime value reported by the API

@FritzHeiden @yanj-github Would it help if the two of you discussed this in a zoom/teams call? Otherwise I'm not sure this is going anywhere.

[yanj-github]

@jpiesing and @FritzHeiden I have created a new issue for this #172

[jpiesing]

@yanj-github Now #172 is closed, is there anything left in this issue?

[yanj-github]

@jpiesing It can be closed if you are happy. The latest result I sent you these are all PASS now.

[jpiesing]

@jpiesing It can be closed if you are happy. The latest result I sent you these are all PASS now.

That's not correct - at least how I interpret the words. Of course you may have meant something different.

I can see failures due to last frame not being shown - which may be another example of #156.

ss1-2 fails with a duration error.

[yanj-github]

@jpiesing I thought this issue only meant current time match test. If you meant test with other observations then yes some observations failed.

Regarding to duration check it might be related to cta-wave/device-observation-framework#59 (comment). A1 (low resolution) presented at the beginning this is harder to detect when fading in it might impact how accurate we can calculate duration.
Here are the options:

• Better recording such as recording taken in controlled environment (as we suggested in OF documentation) will make is pass. Recommended.
• Use more recent version of OpenCV. Not recommended at this stage we are close to release.
• Update playout parameter to have higher resolution content to begin with. This is the best option as we dont want to make is so hard to PASS duration check. Feedback and help required on this, I am not sure the impact and how easy to test this. NOTE: A1 is on 12.5 fps frame duration is 80ms but duration tolerance is 50ms. With ideal recording the detected duration will be accurate but on the worst case detected duration may be short for up to 80ms.

[jpiesing]

2024-05-28: This remains open until it's been run with the new debugging support so we can see what happens with the last frame. It needs to be re-run on the exact same model of TV as exhibited the original problem.