Implement device font rendering

[Herschel]

In Flash, text fields can either use "outline fonts" (embedded glyph shapes in the SWF) or "device fonts" (use the OS to render).
For device fonts, we currently cheat by embedding a version of Noto Sans into Ruffle, and rendering this the same as an outline font.

However, this is reaching the limit of its usefulness: the faux-device-font only has basic characters embedded and will display nothing for international text or other glyphs that it is missing (#1616). We need to properly render device text, using system fonts as appropriate.

Let's add a new TextBackend that handles text rendering for each platform:

• desktop: Use font-kit
  • This doesn't provide shaping, but should be good enough to implement basic text layout as a start.
• web: Use canvasRenderingContext2d.measureText/fillText.
  • On WebGL, copy the canvas-drawn text into a texture.
  • In the future, keep an eye on: https://web.dev/local-fonts/

[SamMorrowDrums]

I think this also covers #1724

[SamMorrowDrums]

I started trying to have a look at this, and I was trying to understand where to start. I found some code for identifying the best match font available in font-config crate, something along these lines looks like a promising start to me:

use font_kit::properties::Properties;
use font_kit::source::SystemSource;


for family in args[1].split(',') {
        let family = family.replace('\'', "");
        let family = family.trim();
        families.push(match family {
            "serif" => FamilyName::Serif,
            "sans-serif" => FamilyName::SansSerif,
            "monospace" => FamilyName::Monospace,
            "cursive" => FamilyName::Cursive,
            "fantasy" => FamilyName::Fantasy,
            _ => FamilyName::Title(family.to_string()),
        });
    }

let handle = SystemSource::new().select_best_match(&families, &properties)?;
let font = handle.load()?;

And I found some methods that might be helpful for the first pass at rendering output:

font.rasterize_glyph(
        &mut canvas,
        glyph_id,
        size,
        Transform2F::from_translation(-raster_rect.origin().to_f32()) * transform,
        hinting_options,
        rasterization_options,
    )

I found various places where fonts are consumed and I wasn't sure exactly how to hook in to the codebase. In core/src/player.rs there is load_device_font. And in core/src/html/layout.rs there is a resolve_font method, and of course there is core/src/font.rs.

So @Herschel - my main questions are really:

• Is the idea that
  • there is an abstract definition of a TextBackend in Core, with implementations that live in their respective homes:
    • the font-config backend should live in Desktop
    • the canvasRenderingContext2d.measureText/fillText. should live in web
  • the TextBackend is responsible for the loading, and rasterization of fonts
    • So the raster data is then passed on to the RenderBackend (this boundary is something I'm a bit unsure of)
• Should the SWF embedded fonts be handled also by TextBackend, or is that only for device fonts?

Anyway, it sounds like perhaps this is too big a project (in an area like Fonts that I'm not experienced with) - so perhaps I'm just highlighting that I'm not the person for the job - but it was on my list of possible contribs, so I at least waned to give it a valiant effort.

Let me know what you think at any rate :-)

[SamMorrowDrums]

I think the core of my confusion is what to output, / what modules responsibility should be. I'm assuming when rendering text fields what we want is to make sure the font is available or an appropriate fallback identified ideally - then let the system handle the raster, but they work in quite different ways:

Canvas in JS only needs a string combining the variant, size, weight and font face, and a colour and layout etc.

const canvas = document.getElementById('canvas');
const ctx = canvas.getContext('2d');
ctx.textAlign = "start";
ctx.textBaseline = "bottom";
ctx.font = 'bold 48px serif';
ctx.fillStyle = '#FFFFFF';
ctx.fillText('Hello world', 50, 100);

And I'm guessing we need to access the measureText in order to manually handle text wrapping?

But then the Desktop via Font-Kit I'm not sure what to do after trying to identify the font itself. The output of font.rasterize_glyph different and perhaps not the correct approach, and maybe we just want to load the raw font data and then consume it in the same way as we are consuming noto currently?

Also, as far as webGL is concerned - there is perhaps an interesting tutorial on creating text using off-screen canvas, and generating a texture to use in webGL (which seems far simpler than trying to do font rendering properly in webGL) https://delphic.me.uk/tutorials/webgl-text

[ousia]

@SamMorrowDrums,

it would be really helpful for people with non-English alphabets (or writing systems) if you could implement this.

Many thanks for your help.

[ousia]

@Herschel, until this is implemented would it be possible to add all characters from the Latin-1 Supplement Unicode block to the Noto Sans font embedded in Ruffle?

Besides one question in discussions, 9 issues will be fixed with that simple measure.

Just in case it might be relevant to be considered.

[SamMorrowDrums]

I need to be honest too, I haven't had the opportunity to look into this further, and I think it was something that is too out of my knowledge area to do well, so it is unlikely I'll be looking at this. At least for a while. Extra chars in embedded font is certainly worthy of consideration though as a stop-gap. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

…

On Wednesday, February 24th, 2021 at 16:13, Pablo Rodríguez ***@***.***> wrote: @Herschel, until this is implemented would it be possible to add all characters from the Latin-1 Supplement Unicode block to the Noto Sans font embedded in Ruffle? Besides one question in discussions, 9 issues will be fixed with that simple measure. Just in case it might be relevant to be considered. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ousia]

BTW, questions #2783 and #4363 are related to this issue.

[ousia]

@Herschel, since the implementation of this issue might take a while, would it be possible that the embedded device font doesn’t remove chars, so chars with diacritical marks in Latin texts may be displayed?

(Sorry, I don’t remember which one is the embedded device font, so I can’t say whether other non-Latin scripts may benefit from this.)

[torokati44]

+1 from me on adding at least the accented Latin glyphs to the embedded pseudo-device-font.
I'd do it myself if I knew how to. Not that I really tried to figure it out yet...

[whj2015]

Is there a temporary solution to this problem

[ousia]

Is there a temporary solution to this problem

I’m afraid that only embedding the fonts in SWF files is could be a fix right now.

(And I agree this makes no sense for SWF files that cannot be recreated.)

[ROBERT-MCDOWELL]

@Herschel
since july 1st 2022, maybe it's time to use local font access API!!
"The Local Font Access API was part of the capabilities project and is now shipped!!!" :D
I'm so frustrated to not know Rust enough to accelerate Ruffle! if you know a good website to learn Rust quikcly please tell me thanks

[ousia]

@ROBERT-MCDOWELL, I’m afraid it seems that it might take a while before Firefox will have this implemented (after having considered their position on the it) .

[ROBERT-MCDOWELL]

@ousia nevermind firefox does it or not, they will follow chrome/safari which is already implemented.

[Alexander-Tolano]

For an example swf, visit https://judithedelman.com/ and click on News

[Lord-McSweeney]

The main part of this issue has been closed by #14535.

Web still requires site configuration to load "device" fonts into.