drm/vc4: Add support for scaled modes in VEC

[kFYatek]

The first new commit in this PR adds support for recreating the "fake 480p/576p" mode that the Pi boots up with the firmware framebuffer when set to sdtv_mode=16/sdtv_mode=18. This new mode is reported as 720x480 (scaled)/720x576 (scaled) and is now the default when sdtv_mode=16/sdtv_mode=18 is set in config.txt (although it's chosen based on the video= kernel command line option), to replicate the firmware behaviour. In this mode, the graphics are rendered in 720x480/720x576, but scaled to 240p/288p for display in the composite progressive mode.

This closely replicates the firmware behaviour, with some differences: trapezoidal filtering is used for this scaling (as per vc4_get_scaling_mode() in vc4_plane.c), so the effect is that each two lines are averaged instead of every other one dropped - this actually improves image quality compared to the firmware mode; things that are separate DRM planes (like the X mouse cursor) may also be located a bit imprecisely on the screen - but that's generally not a problem in practice.

This has already been accepted into Lakka (libretro's fork of LibreELEC), see libretro/Lakka-LibreELEC#1290, as it provides the preferred video mode for emulators in the KMS driver.

---

The second commit adds support for horizontal scaling, which makes aspect ratio correction possible. For example:

xrandr --newmode 856x480i 16.364 856 881 958 1040 480 487 493 525 Interlace

This modeline provides square (virtual) pixels for a 16:9 NTSC display. Note that it results in normal 13.5 MHz physical pixel clock (16.364 * 858 / 1040), but specifying this virtual pixel clock of 16.364 MHz makes the refresh rate display correctly in xrandr.

---

I suppose that this PR might be controversial, and if you say that "scaling does not belong in the driver", then it's totally understandable and I won't push for this change. However, this replicates functionality that was there in the firmware and some software in the Pi ecosystem may rely on this kind of behaviour being available, so I thought it might be worth it to try offering this idea.

[kFYatek]

I've rebased the code now that #4406 is merged.

[pelwell]

Any comments on this PR, @6by9 and @popcornmix ? I spotted a typo in the second commit message (caled), but I suspect the feedback is more likely to be about whether this is a good thing to support rather than how it has been coded.

[ToKe79]

@kFYatek may I kindly ask for your input?

When using 5.10.y / 5.10.76 (581049d) and this patch (combination of #4406 and #4636 - probably older versions of those PRs), the TV out works as expected.

When using 5.10.y / 5.10.95 (b0272c6) which has already #4406 merged and using this patch (your rebased #4636), we do not get TV out by adding e.g. video=Composite-1:720x480@60e to cmdline.txt.

Are we missing something, that was present in the earlier versions of the patches/PRs?

If you do not want to pollute this PR, you can also comment / request more details in libretro/Lakka-LibreELEC@20ee335 (commit that adds this PR as a patch on top of 5.10.95).

Thank you for your help!

EDIT: Turns out, all works as expected. Previously it was not required adding ,composite=1 to dtoverlay=vc4-kms-v3d in config.txt on Pi3, only on Pi4. Turns out now the "requirements" on the config.txt are same for both. Sorry for the panic :-)

[kFYatek]

@ToKe79 No problem. I only now got the time to look at it, but glad to hear the problem is solved :)

[jayare5]

Hey, I have a question. Could this be reimplemented for the newest update? It no longer has the fake 480p scaled to 240p

[6by9]

I'd somehow totally missed this one, but it's been brought up through #6242.

If I'm reading it correctly, you're effectively trying to hack the DRM mode around to influence the resolution that the framebuffer emulation ends up with. Planes and crtc modes then get unmangled to display the image. If so, then it's not the right approach.
The DRM API allows you to apply a plane of any geometry with any scaling and in any location on a display. Ideally you use that API directly and take full control.

Thinking it through on #6242, I do wonder if there is an option of overriding DRM's fbdev framebuffer emulation to allow a configurable resolution and controlling where it is rendered on which displays. See that issue for further comments, but it just depends on how big the overall patch becomes. It's unlikely to be acceptable to mainline (they'll say that the client should just use the DRM APIs properly), but may be small enough to consider keeping downstream.

[kFYatek]

@6by9 Yes, I think that's more or less it, although I'm not exactly an expert on the DRM architecture so I admit that I was working a little bit in the dark about what I'm actually doing 😅

Anyway, here's a little bit of rationale for why this PR was created:

• Old firmware-based mode-switching on the Raspberry Pi implemented the progressive mode as 720x480 (NTSC) or 720x576 (PAL) framebuffer with every other line discarded and the actual resulting image being 240 or 288 progressive lines. This is indeed weird, kludgy and borderline evil, but it is a behavior that software targeting the Pi came to expect. Adding support for downscaling the progressive modes 2x along the vertical axis restores this functionality.
• After progressive modes have been implemented in the DRM driver (drm/vc4: Add support for non-standard modes in VEC #4406), they are available as 720x240 and 720x288 resolutions. Those result in WILDLY non-square pixel aspect ratios (pixels being about twice as tall as wide), which is unacceptable for most modern software that expect square(ish) pixels.
• Yes, DRM supports scaling when ordered so from the userland, but that feature isn't widely supported in practice. You can do that through xrandr, but virtually no GUI-based frontend for it allows setting it. And for apps other than Xorg, the chance of it being supported is even slimmer. The primary use case for the composite output on the Pi, particularly in the progressive mode, seems to be retro gaming emulators, and those apparently don't typically have support for DRM scaling or working with wildly non-square pixels. Especially considering that everything worked with firmware mode-switching, so migrating to KMS "breaks" it.
• This PR also introduces support for horizontal scaling. Firmware-based mode-switching provided support for aspect ratio scaling - you could set sdtv_aspect=3 in config.txt to enable 16:9 pseudo-modes (960x480 / 960x576). None of the modes available actually give true square pixels for either 4:3 or 16:9, but I digress. Anyway, that functionality was lost.
• Because the output is analogue anyway, and virtually no composite displays are able to resolve detail beyond the 6.75 MHz bandwidth, it's essentially indistinguishable whether the DAC is operating at a different sampling rate, or whether horizontal scaling is applied in the digital domain - so supporting fake pixel clock scaling like that doesn't feel like especially evil. And it also allows defining modes like 360x240 for the progressive mode to solve the progressive mode pixel aspect ratio issue the other way.

So the tl;dr is that some software used to rely on settings available through firmware-based mode switching that is now impossible to replicate via DRM, at least not in a straightforward manner, and this PR introduces a way to do so.

It is kludgy, it's a hack, it compensates for deficiencies in userland software in kernel mode, which makes it borderline evil, I can admit that myself. But it also improves compatibility with existing software.

So... I can port this to the current branch, but I'd like your opinion on whether that's even something worth doing, and if so then whether to only support the progressive mode downscaling, or to retain the horizontal rescaling too.