Clarify premultiplication #6500

adroitwhiz · 2021-03-16T18:13:17Z

This is mostly an editorial change so I'm not sure if this follows the normal checklist. This is cherrypicked from my larger ImageData premultiplication proposal so it mostly focuses on the canvas API, and I've not cross-referenced the "premultiplication" section with any of the ImageBitmap stuff. Should I do so?

/canvas.html ( diff )
/imagebitmap-and-animations.html ( diff )
/index.html ( diff )

domenic

I really love these kind of clarifications, and am excited about helping this land! However in its current state I still find things a bit confusing. Maybe that's OK, since it's better than the status quo? But I tried to give some guidance in review comments.

domenic · 2021-03-16T18:22:47Z

source

@@ -60357,10 +60357,98 @@ interface <dfn>Path2D</dfn> {
   </dd>
  </dl>

+  <div class="note">
+   <p>
+    <dfn data-x="concept-premultiplied-alpha">Premultiplied alpha</dfn> refers to one way of representing transparency in an image, the other being non-premultiplied alpha.


You can't have normative definitions inside non-normative notes. Is the intention here to provide a normative definition, or just a discussion?

This is intended to be a normative definition--should I just move it outside of the note?

I'm not very sure where to put this definition, actually. Right now, it's defined just before its first usage, but maybe there's somewhere better to put it. The only "somewhere better" I could think of is the CSS Compositing and Blending spec, and I think putting it there would result in scope creep as that spec would likely then have to be updated/clarified as well.

Looking into this more (tangentially), there seem to be only two passing references to premultiplied alpha in CSS Compositing and Blending, and none in CSS Color. Are there any other specifications to look at, or is all of this just implicit currently?

I suspect it's all implicit currently, although we could try to loop in the relevant CSS folks to be sure.

I like your idea of stepping back and finding a better location for this. (Note that in general there's no need to define before use; we have hyperlinks 😊.) I'm currently thinking maybe it deserves its own section, as a sibling to https://html.spec.whatwg.org/#colour-spaces-and-colour-correction ? Or, I guess, if we want to keep it 2D context specific, as a sibling of https://html.spec.whatwg.org/#drawing-model ?

I've created a new section for premultiplied alpha, and moved the "output bitmap must be premultiplied" part into it.

In the future, it may be good to move the "what does premultiplied alpha mean conceptually" part into one of the CSS specs, but that would involve a lot more rewriting.

domenic · 2021-03-16T18:23:24Z

source

@@ -60357,10 +60357,98 @@ interface <dfn>Path2D</dfn> {
   </dd>
  </dl>

+  <div class="note">


This is an extremely long note, and seems to include an example inside of it. Perhaps it'd be best as a statement of fact (not inside a note wrapper), plus an example (div class=example)?

domenic · 2021-03-16T18:24:32Z

source

+      <td><img src="images/premultiplied-example-5.png" width="96" height="96" alt="Fully-transparent (&quot;invisible&quot;) turquoise">
+   </table>
+
+   <p>


These two paragraphs feel like normative definitions that could help make steps elsewhere in the spec more precise. WDYT?

While the ImageBitmap section might be able to make use of these conversion steps, I don't think it's a good idea to link to the canvas section from the ImageBitmap section. Maybe in the future, once this is moved to a more general place?

I think it's fine to link them. That's part of the benefit of having one big spec :).

domenic · 2021-03-16T18:25:24Z

source

+  <p>
+   A <code>CanvasRenderingContext2D</code> object has an <dfn>output bitmap</dfn> that
+   is initialized when the object is created. This output bitmap must use <span data-x="concept-premultiplied-alpha">
+   premultiplied alpha</span> to represent transparent colors.


It's not clear to me what "use premultiplied alpha to represent transparent colors" means in terms of how it affects algorithms or data structures... Maybe it's clear to domain experts?

Maybe it would be better if "represent" was the primary verb instead of "use" ("represent colors using premultiplied alpha")--it affects the range of colors that can be stored in the canvas (you can actually additively blend a canvas without mix-blend-mode by taking advantage of this-- see this comment), determines how the compositor blends the canvas onto other elements, and means that functions like getImageData, for instance, which fetches non-premultiplied color values from the canvas, will lose precision for transparent colors (as it does in all current browsers) because of conversion.

Maybe some language from the WebGL spec could be adapted here:

premultipliedAlpha

If the value is true the page compositor will assume the drawing buffer contains colors with premultiplied alpha. If the value is false the page compositor will assume that colors in the drawing buffer are not premultiplied.

Although I'm not sure "the page compositor will assume" works as well here. In WebGL, you have far more low-level control over the pixel color values that you display, so you're telling the compositor how to interpret the colors that you have rendered, whereas on a 2D canvas, you specify CSS color values and expect shapes drawn with those colors to be composited the same way as with CSS. With a 2D canvas context, you don't merely want the compositor to "assume" a certain representation--you want that representation to be used both when you draw to the canvas and when the canvas is composited onto the page.

My takeaway from this is that the language you added here (either with "use" or "represent" as primary) is enough to make it clear to implementers and domain experts, so my confusion shouldn't count for much.

If you wanted to help other folks like myself understand the implications, what might work would be an example or note (either here or in the premultiplied alpha section) explaining how failing to follow this sentence would produce bad results, stated in very concrete terms like "if we did the wrong thing, then calling canvas.x() would result in Y instead of the desired Z."

adroitwhiz · 2021-03-16T21:28:33Z

source

@@ -94710,7 +94821,7 @@ dictionary <dfn>ImageBitmapOptions</dfn> {
    if the <code data-x="dom-ImageBitmapOptions-premultiplyAlpha">premultiplyAlpha</code>
    option is set to "<code data-x="dom-PremultiplyAlpha-premultiply">premultiply</code>",
    the <span data-x="concept-ImageBitmap-bitmap-data">bitmap data</span>'s color channels are
-    premultiplied by its alpha channel.
+    <span data-x="concept-premultiplied-alpha">premultiplied by its alpha channel</span>.</p>


"are premultiplied by its alpha channel" in the sense of "this is the form that the output will take", not "this is the operation that we will perform on the input".

domenic

Sorry for the delay! I pushed some minor source formatting changes (most notably, using class="example" instead of class="note", and removing the <div w-nodev> wrapper so that users of the dev edition can see this helpful section.

The only remaining thing is some better alt text, and then I think we should merge!

domenic · 2021-03-31T22:32:54Z

source

+       <td>255, 127, 0, 255
+       <td>255, 127, 0, 255
+       <td>Completely-opaque orange
+       <td><img src="images/premultiplied-example-1.png" width="96" height="96" alt="Completely-opaque orange">


This isn't good alt text, as it just repeats what's nearby. Can you give a more detailed description for non-sighted users, per the guidelines in https://html.spec.whatwg.org/#alt ?

I have no idea how to textually describe these images in a meaningful way. Should I just make the alt-text blank?

No, I don't think that would give a good experience.

Added alt text.

domenic

Absolutely perfect, thank you!

domenic · 2021-04-05T18:59:32Z

I went to go verify you, but I noticed you signed the participant agreement with a pseudonym. As it's a legal contract, our lawyers are not comfortable with using pseudonyms; see whatwg/sg#93. Would you be able to update https://github.com/whatwg/participant-data/blob/main/individuals.json with the name you normally use for signing legal contracts, as was done e.g. in whatwg/participant-data#34 ?

adroitwhiz · 2021-04-05T19:04:45Z

Yup

adroitwhiz added 3 commits March 16, 2021 14:08

Elaborate on what premultiplied alpha is

1214d64

Add alt attributes to images

4b127c0

Clarify some premultiplied alpha stuff

a7ceae5

domenic reviewed Mar 16, 2021

View reviewed changes

adroitwhiz added 6 commits March 16, 2021 15:33

Move premultiplied alpha definition to subsection

2e36881

Move "output bitmap premultiplied" to the section

76fbc56

Also specify offscreen canvas premultiplication

72d43ce

Note why canvas premultiplication is mandatory

8c9cb50

Rewrap text

62311e1

Link to premultiplication info in ImageBitmap

f13e1e3

adroitwhiz commented Mar 16, 2021

View reviewed changes

annevk added the topic: canvas label Mar 17, 2021

Source formatting nits and include in the dev edition

2bbf985

domenic reviewed Mar 31, 2021

View reviewed changes

Clarify alt text

9ee37bd

domenic approved these changes Apr 5, 2021

View reviewed changes

domenic merged commit f4bbe03 into whatwg:main Apr 5, 2021

adroitwhiz mentioned this pull request Apr 5, 2021

Premultiplied alpha? WICG/canvas-color-space#40

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clarify premultiplication #6500

Clarify premultiplication #6500

adroitwhiz commented Mar 16, 2021 •

edited by pr-preview bot

Loading

domenic left a comment

domenic Mar 16, 2021

adroitwhiz Mar 16, 2021

adroitwhiz Mar 16, 2021

domenic Mar 16, 2021

adroitwhiz Mar 16, 2021

domenic Mar 16, 2021

adroitwhiz Mar 16, 2021

domenic Mar 16, 2021

adroitwhiz Mar 16, 2021

adroitwhiz Mar 16, 2021

domenic Mar 16, 2021

adroitwhiz Mar 16, 2021

domenic Mar 16, 2021

adroitwhiz Mar 16, 2021

domenic Mar 16, 2021

adroitwhiz Mar 16, 2021

domenic left a comment

domenic Mar 31, 2021

adroitwhiz Mar 31, 2021

domenic Apr 1, 2021

adroitwhiz Apr 5, 2021

domenic left a comment

domenic commented Apr 5, 2021

adroitwhiz commented Apr 5, 2021

Clarify premultiplication #6500

Clarify premultiplication #6500

Conversation

adroitwhiz commented Mar 16, 2021 • edited by pr-preview bot Loading

domenic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

premultipliedAlpha

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenic left a comment

Choose a reason for hiding this comment

domenic commented Apr 5, 2021

adroitwhiz commented Apr 5, 2021

adroitwhiz commented Mar 16, 2021 •

edited by pr-preview bot

Loading