[Sparse Strip]: Text API (outlines only) by taj-p · Pull Request #883 · linebender/vello

taj-p · 2025-03-31T04:28:14Z

Intent

Adds an API to vello_cpu and vello_hybrid for drawing text:

// A Vec of glyph positions: Vec<Glyph { id: u32,  x: f32, y: f32 }>
let glyphs = // ...

scene.glyph_run(&font)
    .normalized_coords(...)
    .font_size(size)
    .hint(true)
    .fill_glyphs(glyphs.iter());

Background

This API does not perform text shaping and layout. Those functions should be performed by other libraries like Parley or Cosmic Text. The API proposed here merely takes glyph positions and rendering attributes and renders them to screen.

Comparison with Vello

The API should feel familiar to Vello's implementation with one change: We defer to the scene's style and transform.

The glyph run doesn't support a global transform as in Vello.
Instead of specifying the brush, we use the stroke / fill settings applied to the scene and choose between the two via [stroke|fill]_glyphs(...).

Happy to change this, but I don't see why the consumer can't use the existing transform/paint/stroke APIs on Scene. In other words, the scene's transform/paint/stroke applying to every primitive except text seems unexpected.

If text deserves special attention, I'm keen to understand that choice and this PR is certainly flexible to that (and any other) change. Please let me know 🙏 .

Design

To support both CPU and Hybrid variants of the renderer, we expose a GlyphRunBuilder from vello_common that accepts a GlyphRenderer trait. The builder encapsulates the logic to "prepare" some glyph for rendering before passing it to the GlyphRenderer to draw to screen. I've implemented GlyphRenderer for both our CPU and Hybrid variants. The CPU variant implementation is copied below:

vello/sparse_strips/vello_cpu/src/render.rs

Lines 185 to 209 in 87d1801

    
           impl GlyphRenderer for RenderContext { 
        
               fn fill_glyphs(&mut self, glyphs: impl Iterator<Item = PreparedGlyph>) { 
        
                   for glyph in glyphs { 
        
                       match glyph { 
        
                           PreparedGlyph::Outline(glyph) => { 
        
                               let transform = self.transform * glyph.local_transform; 
        
                               flatten::fill(&glyph.path, transform, &mut self.line_buf); 
        
                               self.render_path(self.fill_rule, self.paint.clone()); 
        
                           } 
        
                       } 
        
                   } 
        
               } 
        
               fn stroke_glyphs(&mut self, glyphs: impl Iterator<Item = PreparedGlyph>) { 
        
                   for glyph in glyphs { 
        
                       match glyph { 
        
                           PreparedGlyph::Outline(glyph) => { 
        
                               let transform = self.transform * glyph.local_transform; 
        
                               flatten::stroke(&glyph.path, &self.stroke, transform, &mut self.line_buf); 
        
                               self.render_path(Fill::NonZero, self.paint.clone()); 
        
                           } 
        
                       } 
        
                   } 
        
               } 
        
           }

Next Steps

This PR only supports outlined glyphs. We need to support Emoji through bitmap and colr variants. I'd prefer to do that in separate PRs and keep this PR focused on the API and overall strategy.

Unfortunately, I suspect these other two variants cannot be started until we support:

Clipping for colr glyphs (via ColorPainter)
Images for bitmap glyphs

Perhaps since clipping is underway, I could look at image support? But, again, I worry that, like fast text rendering, image rendering would likely need some form of caching 🤔 .

Notes

I haven't implemented caching of glyphs nor hinting instances. As per discussions, we wanted to keep the text API free from caching for now.

sparse_strips/vello_cpu/src/render.rs

LaurenzV

There are definitely some things we will need to change in the long run, but I think it's a good start! :)

sparse_strips/vello_common/Cargo.toml

sparse_strips/vello_api/src/glyph.rs

sparse_strips/vello_common/src/glyph.rs

sparse_strips/vello_cpu/src/render.rs

sparse_strips/vello_cpu/tests/basic.rs

sparse_strips/vello_common/src/glyph.rs

DJMcNab

Thanks! It's really great to get this working. It's impressive how little code this takes.

Nothing fundamental here, but no shortage of nits!

It will be good to get benchmarks in for this (not now)

sparse_strips/vello_api/src/glyph.rs

sparse_strips/vello_common/Cargo.toml

sparse_strips/vello_common/src/glyph.rs

sparse_strips/vello_cpu/tests/basic.rs

DJMcNab · 2025-03-31T10:24:57Z

sparse_strips/vello_hybrid/examples/text.rs

+    let font = Font::new(Blob::new(Arc::new(ROBOTO_FONT)), 0);
+    let font_ref = {
+        let file_ref = FileRef::new(font.data.as_ref()).unwrap();
+        match file_ref {
+            FileRef::Font(f) => f,
+            FileRef::Collection(collection) => collection.get(font.index).unwrap(),
+        }
+    };
+    let axes = font_ref.axes();
+    let size = 52_f32;
+    let font_size = skrifa::instance::Size::new(size);
+    let variations: Vec<(&str, f32)> = vec![];
+    let var_loc = axes.location(variations.as_slice());
+    let charmap = font_ref.charmap();
+    let metrics = font_ref.metrics(font_size, &var_loc);
+    let line_height = metrics.ascent - metrics.descent + metrics.leading;
+    let glyph_metrics = font_ref.glyph_metrics(font_size, &var_loc);
+
+    let mut pen_x = 0_f32;
+    let mut pen_y = 0_f32;
+
+    let text = "Hello, world!";
+
+    let glyphs = text
+        .chars()
+        .filter_map(|ch| {
+            if ch == '\n' {
+                pen_y += line_height;
+                pen_x = 0.0;
+                return None;
+            }
+            let gid = charmap.map(ch).unwrap_or_default();
+            let advance = glyph_metrics.advance_width(gid).unwrap_or_default();
+            let x = pen_x;
+            pen_x += advance;
+            Some(Glyph {
+                id: gid.to_u32(),
+                x,
+                y: pen_y,
+            })
+        })
+        .collect::<Vec<_>>();


I'd lean towards just using Parley here. Like yes, it's a lot more expensive, but we expect the majority of users of this to use it.
We have evidence of people copying very similar code uncritically, because text layout is not a familiar question to most people.

It would also be nice to use system fonts, but using Roboto would also be reasonable.

Yes, it makes the code longer, but it better emphasises the separation of concerns.

I'll migrate this example to use Parley. I might not get time to it today.

Updated example to use Parley in 7fdd1af (will update to use a variable font in separate commit)

DJMcNab · 2025-03-31T10:27:47Z

sparse_strips/vello_hybrid/examples/text.rs

+
+    // Fill the text
+    ctx.glyph_run(&font)
+        .normalized_coords(vec![])


It could be fun to really demonstrate variable font support here. The one tricky thing is that we don't have one in-repo. https://github.com/linebender/xilem/tree/main/xilem/resources/fonts/roboto_flex would be possible here (100KB)

I ran out of time this morning to finish this. I've put it on my list of things to do.

DJMcNab · 2025-03-31T10:28:19Z

sparse_strips/vello_hybrid/examples/text.rs

I'm also slightly concerned by the proliferation of examples here. What happens when we need to e.g. update winit; that seems like it would become very painful.

Agree! Updating winit can be a pain! After taj-p#5, I'll refactor Vello Hybrid examples to instead use 1 winit example that cycles through a set number of scenes (while being able to pass flags to load a particular scene on start) (similar to how the root examples/scenes crate works) - WDYT @grebmeg ?

💯 yes — thank you!

sparse_strips/vello_common/Cargo.toml

DJMcNab

A few nits, but otherwise I think this is ready.

DJMcNab · 2025-04-02T06:16:31Z

sparse_strips/vello_common/src/glyph.rs

+#[derive(Clone, Debug)]
+struct GlyphRun<'a> {
+    /// Font for all glyphs in the run.
+    pub font: Font,


It's not immediately obvious to me why the fields of this private struct are public. But OTOH, if clippy isn't complaining about it, that's probably my lack of understanding here.

Nice catch! Made private in 6e661a4

DJMcNab · 2025-04-02T06:24:57Z

sparse_strips/vello_common/src/glyph.rs

+            let [a, b, c, d, _, _] = self.run.transform.as_coeffs();
+            if a == d && b == 0.0 && c == 0.0 {
+                // TODO: Cache hinting instance.
+                HintingInstance::new(&outlines, size, self.run.normalized_coords, HINTING_OPTIONS)


So just to really lay this out, currently we scale glyphs post-hinting. Which as an MVP is fine, but it is definitely wrong behaviour (because you're not really hinting unless you have a scale of exactly 1.0!).

We should have a comment here, probably.

Good callout. Added comment in 8284434. I also only applied hinting when scale is 1.0. Immediately after this, I'll prepare taj-p#5 for review.

DJMcNab · 2025-04-02T06:30:40Z

sparse_strips/vello_hybrid/examples/text.rs

+
+fn draw_text(
+    ctx: &mut Scene,
+    text: String,


Should this be a &str?

Made &str in 3d6735d

DJMcNab · 2025-04-02T06:31:07Z

sparse_strips/vello_hybrid/examples/text.rs

+    builder.push_default(StyleProperty::LineHeight(1.3));
+    builder.push_default(StyleProperty::FontSize(32.0));
+
+    let mut layout: Layout<ColorBrush> = builder.build(&text);


In theory, we should be caching the layout. I don't want to block this PR on that, though

DJMcNab · 2025-04-02T06:32:09Z

sparse_strips/vello_hybrid/examples/text.rs

+    let normalized_coords = run
+        .normalized_coords()
+        .iter()
+        .map(|coord| NormalizedCoord::from_bits(*coord))


This is where we'd use bytemuck::cast_slice, I think.

DJMcNab · 2025-04-02T06:34:58Z

sparse_strips/vello_hybrid/examples/text.rs

+        scene: Scene::new(900, 600),
+    };
+
+    app.font_cx.collection.register_fonts(ROBOTO_FONT.to_vec());


One of "always using roboto" and collecting system fonts is unnecessary.

In terms of teaching how we expect people to use this, I'd probably lean towards only using system fonts (acknowledging the difficulty that has with web support - but this example doesn't yet run on the web)

Use system fonts in 9ab7341 and d4d5658

Ah, it fails CI when we try to build the vello_hybrid crate to Wasm. Instead of opting vello_hybrid out of Wasm build, I've reverted these changes. I will implement them as part of this idea next.

sparse_strips/vello_hybrid/examples/text.rs

raphlinus · 2025-04-02T19:48:55Z

Thanks for all the comments in review. One point I'll make (based on the initial comments) is that emoji is not necessarily bitmap based, although that will be the case on Apple platforms. The COLRv1 format is outline based with a fairly sophisticated imaging model. It should be possible to port Vello's implementation (which lives mostly in vello/src/scene.rs).

…to tajp/sparseStrips/textApi

…extApi

… size when hinting (#889) In #883, we discussed (#883 (comment)) how to handle hinting when the glyph is later transformed. The approach Vello uses (and shown in this PR): if hinting is requested and the transform contains a uniform scale (without skew / rotation), extract the scale from the transform and apply it to the font size and glyph positions. This behaviour is the same used by Vello: 1. Extracting and applying scale to font size when hinting is enabled: https://github.com/linebender/vello/blob/61f82e4e66a9111d6b2ee17f173167c3601c2e55/vello_encoding/src/resolve.rs#L414-L428 2. Applying that scale to glyph positions: https://github.com/linebender/vello/blob/61f82e4e66a9111d6b2ee17f173167c3601c2e55/vello_encoding/src/resolve.rs#L334 If a consumer doesn't want this behaviour, then they can disable hinting on scaling. You can see what effect this has on our snapshot: https://github.com/user-attachments/assets/9e8577d2-94e8-4722-8878-3a55dc64b6a6

[Sparse Strip]: Text API (outlines only)

87d1801

taj-p commented Mar 31, 2025

View reviewed changes

sparse_strips/vello_cpu/src/render.rs Outdated Show resolved Hide resolved

taj-p mentioned this pull request Mar 31, 2025

[Sparse Strip]: Text API (outlines only) taj-p/vello#3

Closed

taj-p marked this pull request as ready for review March 31, 2025 04:46

LaurenzV reviewed Mar 31, 2025

View reviewed changes

tomcur reviewed Mar 31, 2025

View reviewed changes

sparse_strips/vello_cpu/tests/basic.rs Show resolved Hide resolved

sparse_strips/vello_cpu/tests/basic.rs Show resolved Hide resolved

sparse_strips/vello_common/src/glyph.rs Outdated Show resolved Hide resolved

DJMcNab reviewed Mar 31, 2025

View reviewed changes

taj-p added 3 commits April 1, 2025 05:25

Don't expose Skrifa public API + use norm coords slice

db226ef

Hardcode NonZero fill text rendering

897c7a0

Add disclaimer to layout_glyphs

6ccfb54

DJMcNab reviewed Mar 31, 2025

View reviewed changes

sparse_strips/vello_common/Cargo.toml Outdated Show resolved Hide resolved

taj-p added 6 commits April 1, 2025 11:56

Disable hinting for rotated, skewed text

a57bed9

Prevent allocating a new path for each glyph

c458e8c

Set no features on bytemuck

b25c507

clippy

f4742e6

Add test for scaled glyphs

f0af18e

Improve doco

18bf5cd

taj-p mentioned this pull request Apr 1, 2025

Apply uniform scaling to glyphs by scaling font size and glyph positions taj-p/vello#5

Closed

taj-p added 4 commits April 2, 2025 07:24

Parley example

7fdd1af

.

c64fd57

more clippy

3ea9728

Minor

31e2d85

DJMcNab approved these changes Apr 2, 2025

View reviewed changes

Make GlypRun members private

6e661a4

taj-p added 5 commits April 3, 2025 06:49

Only use hinting when scale is 1.0

8284434

Use &str

3d6735d

Move bytemuck::cast_slice

72ac145

Update Parley deps

9ab7341

Merge branch 'tajp/sparseStrips/textApi' of github.com:taj-p/vello in…

01174dc

…to tajp/sparseStrips/textApi

taj-p added 4 commits April 3, 2025 09:38

Remove roboto

d4d5658

Merge remote-tracking branch 'upstream/main' into tajp/sparseStrips/t…

249abcf

…extApi

Use Wasm for text example

dffe263

.

d410f7f

taj-p added this pull request to the merge queue Apr 2, 2025

Merged via the queue into linebender:main with commit a271aaa Apr 2, 2025
17 checks passed

taj-p mentioned this pull request Apr 3, 2025

[Sparse Strips]: Take scale from glyph transform and apply it to font size when hinting #889

Merged

	impl GlyphRenderer for RenderContext {
	fn fill_glyphs(&mut self, glyphs: impl Iterator<Item = PreparedGlyph>) {
	for glyph in glyphs {
	match glyph {
	PreparedGlyph::Outline(glyph) => {
	let transform = self.transform * glyph.local_transform;
	flatten::fill(&glyph.path, transform, &mut self.line_buf);
	self.render_path(self.fill_rule, self.paint.clone());
	}
	}
	}
	}

	fn stroke_glyphs(&mut self, glyphs: impl Iterator<Item = PreparedGlyph>) {
	for glyph in glyphs {
	match glyph {
	PreparedGlyph::Outline(glyph) => {
	let transform = self.transform * glyph.local_transform;
	flatten::stroke(&glyph.path, &self.stroke, transform, &mut self.line_buf);
	self.render_path(Fill::NonZero, self.paint.clone());
	}
	}
	}
	}
	}

Conversation

taj-p commented Mar 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Intent

Background

Comparison with Vello

Design

Next Steps

Notes

Uh oh!

Uh oh!

LaurenzV left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DJMcNab left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taj-p Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DJMcNab left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

taj-p commented Mar 31, 2025 •

edited

Loading

taj-p Apr 1, 2025 •

edited

Loading