Proposal: DWriteCore - Cross-platform and Down-level DirectWrite

[duncanmacmichael]

Proposal: DWriteCore - Cross-platform and Down-level DirectWrite

Summary

DWriteCore is the Project Reunion version of DirectWrite (DWrite), the text rendering and layout component that ships in the Windows SDK. DWrite supports a rich array of features that make it the font rendering tool of choice on Windows for most apps, either through direct calls or via Direct2D, including a device-independent text layout system, high quality sub-pixel ClearType text rendering, hardware-accelerated text, multi-format text, advanced OpenType typography features, wide language support, and GDI-compatible layout and rendering. It has been available since Windows Vista SP2 and has evolved over the years to include more advanced features such as variable fonts, which enables developers to apply styles, weights, and other attributes to a font with only one font resource.

Due to the long lifespan of this component, however, older versions of Windows are left behind with new development. In addition, DWrite’s status as the premier text rendering technology is limited only to Windows, leaving cross-platform applications to either write their own text rendering stack or rely on 3rd party solutions.

DWriteCore aims to solve the core problems of version feature orphaning and cross-platform compatibility by removing the library from the system and targeting all possible supported endpoints. To this end, we are integrating DWriteCore into Project Reunion with a public API that lights up all Windows endpoints down to Windows 8 and opens the door for developers to use it cross platform.

Demo

DWriteCore will be demonstrated with a FontViewer app available to developers, which will expand to add functionality as features are ported from system DWrite to DWriteCore.

Rationale

The primary value DWriteCore gives developers in Project Reunion is that it provides access to all current DWrite features to developers all the way down-level to Windows 8. All features of DwriteCore will function the same on all down-level versions; in other words, all current features will work on Windows 8, 8.1, and all versions of Windows 10 without any disparity regarding which features might work on which versions.

Staging plan

Porting system DWrite to DWrite core is a sufficiently large project to span multiple Windows release cycles. At the time of this proposal, it is divided into four Phases, each of which correspond to a chunk of functionality delivered in a release.

Disclaimer
DWriteCore is under active development by the Microsoft developer platform team and the roadmap will continue to evolve based on market changes and customer feedback, so please note that the plans outlined here aren't exhaustive or guaranteed.

Phase 1: Foundation

Phase 1 includes the basic tools a developer needs to consume DWriteCore, including:

• Font enumeration
• Font API
• Shaping
• Low-level rendering APIs (partial – does not use D2D yet but can use IDWriteGlyphRunAnalysis and IDWriteBitmapRenderTarget)
• Basic text layout functionality

With these features, developers can immediately begin to harness some of DWrite’s modern core functionality such as variable fonts down level to Windows 8. This iteration of the library can also be consumed on Android, currently being tested internally. Variable fonts are one of the most important features for DWrite customers; they were introduced in Windows 10 RS3, so accessing them in previous versions is a massive boon to developers.

This phase is complete at the time of this proposal.

Phase 2: Text Rendering and Color Fonts

Phase 2 fleshes out the text rendering aspect of DWrite:

• Text rendering APIs
• Finish bitmap render target
• Color fonts
• Miscellaneous gaps (font cache cleanup, in-memory font loader, etc.)

The banner feature for Phase 2 is color fonts. Color fonts enable developers to render their fonts with more sophisticated color functionality beyond simple single colors; for example, it is what powers the ability to render emoji and toolbar icon fonts (the latter of which is used by Office, for example). Previously, color fonts were first introduced in Windows 8.1 but were heavily expanded upon in the Windows 10 Anniversary Update.

In addition, users should see memory improvements via cleanup of the font cache, and an in-memory font loader allows for faster loading of fonts.

Phase 2 is currently under development at the time of this proposal and is estimated to arrive in Q4 2020.

Phase 3: Text Layout

Phase 3 focuses on bringing DWrite’s powerful layout features to developers:

• Advanced text layout features (e.g. vertical text)
• GDI-compatible layout
• Spacing and justification
• Finish text layout API

Vertical text is particularly important to Far East font users and should open up the audience for DWriteCore developers to these target demographics. The line spacing feature, which allows developers finer grain control over line spacing based on the largest or smallest characters in a glyph run, is also introduced here, which was not developed until Windows 10 Threshold (1511).

Phase 3 is estimated to arrive in Q1 2021.

Phase 4: Font streaming and languages

Phase 4 rounds out the DWriteCore feature set:

• Font streaming (downloadable fonts, remote font file loader, etc.)
• Additional languages
• CFF rasterizer support

Font streaming was introduced in the initial release of Windows 10; it allows users to download fonts on demand as needed, to save disk space. Expanded language support will open up DWriteCore to a wider variety of languages, and the CFF rasterizer is an important rasterizer alternative to TrueType.

Phase 4 is estimated to arrive in Q2 2021.

Invitation to DWrite Developers

DWriteCore, along with other Project Reunion components, will be developed with visibility into current work and openness to developer feedback. Once the project has migrated to GitHub, we invite developers to begin consuming DWriteCore packages and provide insights or requests into feature development.

We will update this section after the repo and demo links become available.

Scope

Capability	Priority
This proposal will allow developers to take advantage of all current and future DWrite features down level to Windows 8	Must
This proposal will allow developers to consume DWrite cross platform, currently targeting iOS, Android, and Mac	Must

[mdtauk]

This is amazing! Thank you for doing this. DirectWrite and WinUI 3 both being open, will enable lots of new possibilities! Making it cross platform is also a great idea, especially as Microsoft now has an Android OS to support, as well as a Linux Kernal, and Linux GUI app support coming to WSL in Windows...

[reflectronic]

Does this mean the patented ClearType technology will be open source and cross-platform?

[duncanmacmichael]

Thanks @mdtauk, glad you're excited for DWriteCore! :) Stay tuned for updates coming soon on how to start consuming what we have available so far.

@reflectronic , good question. We haven't made a final decision on whether to make all of DWriteCore open source, which would include ClearType, but we're hoping to make that call soon pending some internal investigations and will let you know. It will be available cross platform as a minimum though, as part of the whole DWriteCore package.

[mdtauk]

Well there be any performance implications from extracting it from the OS?

And will WinUI move over to supporting this new version of Direct Write, for it's text rendering?

[eugenegff]

Seems that making Direct2D open source and cross-platform is not part of this proposal. So DWriteCore alone should be complemented by something like Skia, that already can use DirectWrite as font enumeration and management engine.

Or maybe this work is intended to be the part of cross-platform TextBox implementation, without having 2D graphic in mind.

[duncanmacmichael]

@mdtauk Looking into this for you now, thanks for your patience.

Edit 1: Having spoken with the dev team, I can confirm that the performance goal for DWriteCore is to have essentially the same performance as inbox DWrite. We do lose out on some optimizations since it'll no longer be part of the system, but we're working on mitigations for those currently. Thanks for the question!

@eugenegff , correct, Direct2D is not in scope for this proposal. It is true that DWrite and D2D have a close relationship, but any discussions about D2D itself don't apply here.

[fschneidereit]

@duncanmacmichael @reflectronic Are there still patents covering ClearType that haven't expired yet? I mean, Wikipedia lists a few, but all of them expired, either in 2018 or 2019.

[duncanmacmichael]

Thanks for the question @fschneidereit . You're correct, the patents for ClearType currently listed on Wikipedia did all expire in 2018 or 2019. I don't know of any additional patents for ClearType not listed there.

If you posed your question relating to the overall question of open sourcing DWriteCore, then there is still current IP for other technologies in the stack that we have to consider besides ClearType. At this time, we have decided not to open source DWriteCore but are open to considering it in the future if circumstances change.

[davelab6]

Having this open source will be helpful for interoperability.

[raphlinus]

Echoing @davelab6, I'd also like to vote for this being open sourced. The Druid project is slowly but surely attempting to solve a similar problem, providing cross-platform (including down to Windows 7 with platform update) APIs for UI primitives including rich text. We've recently put an API in place for basic rich text with implementations backed by DirectWrite and Core Text (work on the Linux port is underway). In time, I hope this results in a high quality, open source text stack. I would basically drop all this work in favor of an open sourced DWriteCore, though, as it's quite excellent, and I'm sure the various problems (such as NFD Hangul not working can be fixed.

If an open source DWriteCore does not happen, we plan to move ahead with our work. Currently the functionality is limited, but I'm happy with the progress we've made and feel I understand the way forward fairly well.

[andriysavin]

I thought Project Reunion was solely about unified Windows API. Should we expect more parts of it to be cross-platform?

[duncanmacmichael]

Thanks for your feedback @davelab6 and @raphlinus, we appreciate it. We're spending our time and resources on preparing for the initial preview release of DWriteCore and Project Reunion right now, but we're cataloging all the feedback to make DWriteCore open source and will continue investigating that path as soon as we can.

@andriysavin That is correct, Project Reunion on the whole is about a unified Windows API and making apps run great on Windows. However, individual components might support cross-platform functionality depending on their customers and needs, and DWriteCore is one of those components. I can't speak to cross-platform plans for any other parts of Project Reunion unfortunately.

[davelab6]

Thanks for your feedback @davelab6 and @raphlinus, we appreciate it. We're spending our time and resources on preparing for the initial preview release of DWriteCore and Project Reunion right now, but we're cataloging all the feedback to make DWriteCore open source and will continue investigating that path as soon as we can.

Thanks for the thoughtful reply, that sounds really great. @raphlinus blogged a bit more of the context for this, emphasis mine,

Note that there is a proposal for a cross-platform implementation and also potentially to take it open-source. If that were to happen, it would be something of a game changer.

https://raphlinus.github.io/text/2020/10/26/text-layout.html

[Eli-Black-Work]

Yay! I love ClearType ^_^

How will subpixel rendering (ClearType) work on devices that can be rotated, such as tablets or phones? I remember one of the the rationales for not adding ClearType support to UWP was that ClearType doesn't work on phones that are being viewed vertically, since the RGB subpixel alignment ends up being the "wrong way".

Would ClearType be enabled when the user is viewing the device horizontally and then disabled when the device is rotated to vertical position?

(Backstory and more info: microsoft/microsoft-ui-xaml#768 (comment))

[duncanmacmichael]

Thanks for the question, @Bosch-Eli-Black . I asked about this internally and got this answer from our DWrite dev:

"As to whether ClearType is enabled, there isn’t any default at the level of the DWrite API. It’s up to the client to specify the DWRITE_TEXT_ANTIALIAS_MODE, which can be either ClearType or grayscale. However, it is true that XAML disables ClearType and also that most (all?) of the templates for UWP apps that use DirectX disable ClearType. There are a variety of reasons for this.

For XAML, the main issue is composition. For XAML, each glyph run is a visual, which is then composed together with other visuals to form a scene. ClearType blending is special, requiring three alpha channels. You can’t draw ClearType text onto a transparent background and then compose the result with other primitives using normal alpha blending.

Screen rotation is another issue. DWrite can detect if a monitor is inverted (i.e., rotated 180 degrees) and switch between RGB and BGR, but DWrite does not turn off ClearType if the monitor is rotated 90 degrees. It’s debatable and a matter of personal preference whether ClearType still has some value when a monitor is rotated 90 degrees. Note that it’s still up to the client to call IDWriteFactory::CreateMonitorRenderingParams, since otherwise DWrite doesn’t know what monitor you’re rendering on.

A third issue back in Windows 8 was performance. As part of making Windows 8 “fast and fluid” we were trying to make hardware-accelerated glyph rendering really fast on low-end GPUs, and at the time we were able to do better with grayscale antialiasing than with ClearType. Later, I think we were able to close the gap, so performance is no longer a reason to turn of ClearType."

Hope this helps answer your question!

[Eli-Black-Work]

Thanks, @duncanmacmichael (and DWrite dev)! Very illuminating 🙂

[nickrandolph]

@duncanmacmichael is there an update on whether DWriteCore will be open sourced? if so, likely timeline; if not, is there documentation somewhere listing which parts of ProjectReunion will be open sourced and which won't?

[duncanmacmichael]

Hi @nickrandolph, thanks for checking in. :) I don't have an update on that right now, sorry, as we're focusing all our efforts on completing the DWrite to DWriteCore port. Unfortunately I can't comment on the rest of Reunion since a lot of our plans are still in development, and going open source would be up to each individual component.

[nickrandolph]

@andrewleader is there a discussion somewhere that's tracking where the different components are at with regards to open source, or perhaps some documentation stating what the approach is to open sourcing all (ideally), or parts of reunion?

[andrewleader]

@nickrandolph good question, I'm not sure we're tracking that globally somewhere. And to be transparent, I don't think that is one of our main focuses for 2021. For all-up new code, our hope is to have that be open-source, but for 2021 I think the best we can realistically do is have our specs in the open and plan in the open, and a lot of code will still not be open source.

If this is something you think we should take as a main focus for 2021, please do let us know, however, remember that means we would have to choose to not do something else.

Meanwhile, I can check in on where our teams are at in terms of open source to start collecting a roadmap of where everything is in terms of open source.

[wjk]

@andrewleader I think open-sourcing should be a main goal going forward, to avoid the problem we currently have in the WinUI 3 repo: Dozens of issues that could be easily fixed by the community via PR, but cannot be fixed by anyone outside of Microsoft since the code still hasn’t been released. I think the WinUI devs’ refusal to open-source anything until everything is ready is doing their users an enormous disservice.

However, seeing the source for DWriteCore is quite low on my personal priority list. I don’t recommend we push anything back in favor of open-sourcing that codebase. There are many other features and projects discussed in this repo’s issues that would have a much greater impact if addressed in the near term (examples include #8, #491, or the fascinating but completely vague “subset” API discussed in the docs folder README). In the long term I believe everything should be open-sourced, but when you have infinite time scales you can accomplish anything. 😄

Please do gather a summary for the open-sourcing schedule for the rest of Reunion; it would also serve as a list of what Microsoft is working on internally, which is also something that has been sorely lacking about WinUI and Reunion. You discuss projects and APIs under consideration solely via email, leaving the rest of the world in the dark about what you’re considering. Being able to comment on your plans much earlier in their lifecycle (even if the comment is limited to only “we should do this now” or “this can wait”) would be amazing. Reunion has promised so much, and delivered so little. Let’s work on changing that, together.

[davelab6]

Open source DWrite will have a huge benefit to Microsoft as the OpenType/OpenFont format evolves this year. With COLRv1 winding down, more ambitious improvements to the spec will be proposed by non microsoft people, and insight into DWrite internals will help us understand how things can work better for Microsoft.

[nickrandolph]

I agree with both @wjk and @davelab6 - pushing to open source has a major benefit in that the community can contribute solutions to issues as they arise. The community can also push to codebase to a higher standard, resulting in fewer long term issues.
I understand there's a trade off but I think the cost has to be paid once v1 ships.

[riverar]

Given the initial undock work has yet to be completed, I don't see a lot of contribution opportunities for the community in this area for 2021. I think pushing it off to next year makes sense.

@andrewleader Looking at the proposal, it claims to reach far down-level, to Windows 8. But this is in conflict with the min plat Final Decision #413. So something needs to change here, right? Please don't reply with "some components can choose their own support matrix".

[andrewleader]

Excellent feedback. Mid-next week, I should have statuses on where we are in terms of open source and where we plan/hope to be, I'll share that back with this thread mid-next week.

You discuss projects and APIs under consideration solely via email, leaving the rest of the world in the dark about what you’re considering

We are certainly changing that, and that's one of our top priorities. Examples of this change include our Release cadences and channels topic (we have some team members discussing openly on GitHub, and yes we still have some discussing privately in email but we're working on changing that behavior), our push notification API spec was done in the open too (it had 222 comments!), etc.

Looking at the proposal, it claims to reach far down-level, to Windows 8. But this is in conflict with the min plat

@duncanmacmichael does DWriteCore within Reunion actually work on Windows 8? Or was this detail never updated?

[duncanmacmichael]

I agree with @riverar that the undock work completion is our first priority, though we do see the value in open sourcing DWriteCore. We're not closing the door on that, just moving it down the priority list a little. :)

@andrewleader, within Reunion DWriteCore supports whatever the minimum version is that Reunion supports, which is RS5. We're working on aligning with our versioning messaging internally at the moment, so I'll come back to update this with more info soon.

[AlvisDEV]

What is the current process of the cross platform support? Is there any samples or instructions about how to use DWriteCore on Android NDK?
After almost one year the info shows in this discussion still targets down to Windows 8. Is that plan still in active development or Moved to the same minimum version that Reunion supports?

[duncanmacmichael]

Hi @AlvisDEV, thanks for the question! Since the date of this proposal we've worked on porting DWrite features to DWriteCore and have been investigating the question of downlevel and cross-plat support outside of the Windows AppSDK. At the moment, DWriteCore is still Windows-only through the WinAppSDK, but our downlevel and cross-plat plans are still in motion despite no exact ETA. We still want to make this happen, but it's more complicated than we had anticipated. :)

Can I ask what your interest in DWriteCore on Android is? Customer feedback is always helpful in steering these conversations.

[eugenegff]

@duncanmacmichael We at BeLight Software have crossplatform application Live Home 3D https://www.livehome3d.com, It allows unexperienced person to quickly design her house plan in 2D and then refine it and walk through it in 3D. App is released for macOS, WinRT and iOS. It uses Metal + CoreGraphics + CoreText API on macOS and iOS, and Direct3D 11 + Direct2D + DirectWrite on Windows.

Right now we are in process of porting to fourth platform, Android. We plan to use Vulkan + Skia, but miss rich 2D text drawing functionality. DirectWrite over Skia would be the perfect fit for us.

[juan-]

Thank you everyone for the feedback! Thanks to you, we shipped the first version of DWriteCore as part of WindowsAppSDK 1.0. We will open new issues for future updates and will now close this one.

[AlvisDEV]

@duncanmacmichael Font enumeration and layout ability. Using IDWriteTextLayout to set rich text layout like color fonts and inline object is my major demand. That means I could use the same "upper level" code to handle text, but handle rendering by my self.
For rendering, I am interested in using the CustomTextRenderer (https://docs.microsoft.com/en-us/windows/win32/directwrite/how-to-implement-a-custom-text-renderer) to draw text with OpenGL or Vulkan. Specifically, in DrawGlyphRun callback, I would be able to get some "open-format glyphRun/PathGeometry" which should be friendly to interact with other rendering api.
I use highly customized CustomTextRenderer to transform and set color of a segment of text in DrawGlyphRun in my current Windows Project. Arguments like baselineOriginX are used.
I am not interested in an off-screen picture but the ability of low-level interaction.
Sorry I just saw the topic is closed and I am happy to move my reply to your new issue if you have one.