Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clarify premultiplication #6500

Merged
merged 11 commits into from
Apr 5, 2021
Merged

Clarify premultiplication #6500

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
1214d64
Elaborate on what premultiplied alpha is
adroitwhiz Mar 19, 2020
4b127c0
Add alt attributes to images
adroitwhiz Mar 19, 2020
a7ceae5
Clarify some premultiplied alpha stuff
adroitwhiz Mar 29, 2020
2e36881
Move premultiplied alpha definition to subsection
adroitwhiz Mar 16, 2021
76fbc56
Move "output bitmap premultiplied" to the section
adroitwhiz Mar 16, 2021
72d43ce
Also specify offscreen canvas premultiplication
adroitwhiz Mar 16, 2021
8c9cb50
Note why canvas premultiplication is mandatory
adroitwhiz Mar 16, 2021
62311e1
Rewrap text
adroitwhiz Mar 16, 2021
f13e1e3
Link to premultiplication info in ImageBitmap
adroitwhiz Mar 16, 2021
2bbf985
Source formatting nits and include in the dev edition
domenic Mar 31, 2021
9ee37bd
Clarify alt text
adroitwhiz Apr 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added images/premultiplied-example-1.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/premultiplied-example-2.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/premultiplied-example-3.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/premultiplied-example-4.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/premultiplied-example-5.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
136 changes: 124 additions & 12 deletions source
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60360,7 +60360,7 @@ interface <dfn>Path2D</dfn> {
<div w-nodev>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an extremely long note, and seems to include an example inside of it. Perhaps it'd be best as a statement of fact (not inside a note wrapper), plus an example (div class=example)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed


<p>A <code>CanvasRenderingContext2D</code> object has an <dfn>output bitmap</dfn> that
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't have normative definitions inside non-normative notes. Is the intention here to provide a normative definition, or just a discussion?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intended to be a normative definition--should I just move it outside of the note?

I'm not very sure where to put this definition, actually. Right now, it's defined just before its first usage, but maybe there's somewhere better to put it. The only "somewhere better" I could think of is the CSS Compositing and Blending spec, and I think putting it there would result in scope creep as that spec would likely then have to be updated/clarified as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking into this more (tangentially), there seem to be only two passing references to premultiplied alpha in CSS Compositing and Blending, and none in CSS Color. Are there any other specifications to look at, or is all of this just implicit currently?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect it's all implicit currently, although we could try to loop in the relevant CSS folks to be sure.

I like your idea of stepping back and finding a better location for this. (Note that in general there's no need to define before use; we have hyperlinks 😊.) I'm currently thinking maybe it deserves its own section, as a sibling to https://html.spec.whatwg.org/#colour-spaces-and-colour-correction ? Or, I guess, if we want to keep it 2D context specific, as a sibling of https://html.spec.whatwg.org/#drawing-model ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created a new section for premultiplied alpha, and moved the "output bitmap must be premultiplied" part into it.

In the future, it may be good to move the "what does premultiplied alpha mean conceptually" part into one of the CSS specs, but that would involve a lot more rewriting.

is initialized when the object is created.</p>
is initialized when the object is created.</p>

<!--ADD-TOPIC:Security-->
<p>The <span>output bitmap</span> has an <span
Expand Down Expand Up @@ -64025,7 +64025,7 @@ try {
data-x=""><var>sy</var>+<var>sh</var></span>), (<var>sx</var>,
<var>sy</var>+<var>sh</var>), in the bitmap's coordinate space units.
Pixels outside the <span>output bitmap</span> must be set to <span>transparent black</span>. Pixel
values must not be premultiplied by alpha.</p>
values must not be <span data-x="concept-premultiplied-alpha">premultiplied by alpha</span>.</p>

<p>When the user agent is required to <dfn>create an <code>ImageData</code> object</dfn>, given a
positive integer number of rows <var>rows</var>, a positive integer number of pixels per row
Expand Down Expand Up @@ -64167,10 +64167,11 @@ try {
in the rendering context's <span>output bitmap</span>.</p></li>
</ol>

<p class="note">Due to the lossy nature of converting to and from premultiplied alpha color
values, pixels that have just been set using <code
data-x="dom-context-2d-putImageData">putImageData()</code> might be returned to an equivalent
<code data-x="dom-context-2d-getImageData">getImageData()</code> as different values.</p>
<p class="note">Due to the lossy nature of converting to and from <span
data-x="concept-premultiplied-alpha">premultiplied alpha</span> color values, pixels that have
just been set using <code data-x="dom-context-2d-putImageData">putImageData()</code>, and are not
completely opaque, might be returned to an equivalent <code
data-x="dom-context-2d-getImageData">getImageData()</code> as different values.</p>

<p>The current path, <span data-x="dom-context-2d-transformation">transformation matrix</span>,
<span data-x="shadows">shadow attributes</span>, <span data-x="dom-context-2d-globalAlpha">global
Expand Down Expand Up @@ -65740,6 +65741,116 @@ interface <dfn>OffscreenCanvasRenderingContext2D</dfn> {

</div>

<div w-nodev>
<h5>Premultiplied alpha and the 2D rendering context</h5>

<p>
<dfn data-x="concept-premultiplied-alpha">Premultiplied alpha</dfn> refers to one way of
representing transparency in an image, the other being non-premultiplied alpha.
</p>

<p>
Under non-premultiplied alpha, the red, green, and blue channels of a pixel represent that
pixel's color, and its alpha channel represents that pixel's opacity.
</p>

<p>
Under premultiplied alpha, however, the red, green, and blue channels of a pixel represent the
amounts of color that the pixel adds to the image, and its alpha channel represents the amount
that the pixel obscures whatever is behind it.
</p>

<div class="note">
<p>
For instance, assuming the color channels range from 0 (off) to 255 (full intensity), these
example colors are represented in the following ways:
</p>

<table>
<thead>
<tr>
<th>CSS color representation
<th>Premultiplied representation
<th>Non-premultiplied representation
<th>Description of color
<th>Image of color blended above other content
<tbody>
<tr>
<td>rgba(255, 127, 0, 1)
<td>255, 127, 0, 255
<td>255, 127, 0, 255
<td>Completely-opaque orange
<td><img src="images/premultiplied-example-1.png" width="96" height="96" alt="Completely-opaque orange">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't good alt text, as it just repeats what's nearby. Can you give a more detailed description for non-sighted users, per the guidelines in https://html.spec.whatwg.org/#alt ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea how to textually describe these images in a meaningful way. Should I just make the alt-text blank?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't think that would give a good experience.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added alt text.

<tr>
<td>rgba(255, 255, 0, 0.5)
<td>127, 127, 0, 127
<td>255, 255, 0, 127
<td>Halfway-opaque yellow
<td><img src="images/premultiplied-example-2.png" width="96" height="96" alt="Halfway-opaque yellow">
<tr>
<td>Unrepresentable
<td>255, 127, 0, 127
<td>Unrepresentable
<td>Additive halfway-opaque orange
<td><img src="images/premultiplied-example-3.png" width="96" height="96" alt="Additive halfway-opaque orange">
<tr>
<td>Unrepresentable
<td>255, 127, 0, 0
<td>Unrepresentable
<td>Additive fully-transparent orange
<td><img src="images/premultiplied-example-4.png" width="96" height="96" alt="Additive fully-transparent orange">
<tr>
<td>rgba(255, 127, 0, 0)
<td>0, 0, 0, 0
<td>255, 127, 0, 0
<td>Fully-transparent ("invisible") orange
<td><img src="images/premultiplied-example-5.png" width="96" height="96" alt="Fully-transparent (&quot;invisible&quot;) orange">
<tr>
<td>rgba(0, 127, 255, 0)
<td>0, 0, 0, 0
<td>255, 127, 0, 0
<td>Fully-transparent ("invisible") turquoise
<td><img src="images/premultiplied-example-5.png" width="96" height="96" alt="Fully-transparent (&quot;invisible&quot;) turquoise">
</table>
</div>

<p>
<dfn data-x="convert-to-premultiplied">Converting a color value from a non-premultiplied
representation to a premultiplied one</dfn> involves multiplying the color's red, green, and
blue channels by its alpha channel (remapping the range of the alpha channel such that "fully
transparent" is 0, and "fully opaque" is 1).
</p>

<p>
<dfn data-x="convert-from-premultiplied">Converting a color value from a premultiplied
representation to a non-premultiplied one</dfn> involves the inverse: dividing the color's red,
green, and blue channels by its alpha channel.
</p>

<p>
As certain colors can only be represented under premultiplied alpha (for instance, additive
colors), and others can only be represented under non-premultiplied alpha (for instance,
"invisible" colors which hold certain red, green, and blue values even with no opacity); and
division and multiplication on 8-bit integers (which is how canvas' colors are currently stored)
entails a loss of precision, converting between premultiplied and non-premultiplied alpha is a
lossy operation on colors that are not fully opaque.
</p>

<p>
A <code>CanvasRenderingContext2D</code>'s <span>output bitmap</span> and an
<code>OffscreenCanvasRenderingContext2D</code>'s <span data-x="offscreencontext2d-bitmap"
>bitmap</span> must use premultiplied alpha to represent transparent colors.
</p>

<div class="note">
It is important for canvas bitmaps to represent colors using premultiplied alpha because it
affects the range of representable colors. While additive colors cannot currently be drawn onto
canvases directly because CSS colors are non-premultiplied and cannot represent them, it is
still possible to, for instance, draw additive colors onto a WebGL canvas and then draw that
WebGL canvas onto a 2D canvas via <code data-x="dom-context-2d-drawImage">drawImage</code>.
</div>
</div>

<h3 split-filename="custom-elements" id="custom-elements">Custom elements</h3>

<h4 id="custom-elements-intro">Introduction</h4>
Expand Down Expand Up @@ -94710,7 +94821,7 @@ dictionary <dfn>ImageBitmapOptions</dfn> {
if the <code data-x="dom-ImageBitmapOptions-premultiplyAlpha">premultiplyAlpha</code>
option is set to "<code data-x="dom-PremultiplyAlpha-premultiply">premultiply</code>",
the <span data-x="concept-ImageBitmap-bitmap-data">bitmap data</span>'s color channels are
premultiplied by its alpha channel.
<span data-x="concept-premultiplied-alpha">premultiplied by its alpha channel</span>.</p>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"are premultiplied by its alpha channel" in the sense of "this is the form that the output will take", not "this is the operation that we will perform on the input".


<p>Rejects the promise with an <span>"<code>InvalidStateError</code>"</span>
<code>DOMException</code> if the source image is not in a valid state (e.g., an <code>img</code>
Expand Down Expand Up @@ -95150,14 +95261,15 @@ dictionary <dfn>ImageBitmapOptions</dfn> {
optimal for drawing images onto the canvas.</p></li>

<li><p>If <var>val</var> is "<dfn enum-value for="PremultiplyAlpha"><code
data-x="dom-PremultiplyAlpha-premultiply">premultiply</code></dfn>", the <var>output</var>
that is not premultiplied by alpha must have its color components multiplied by alpha and
that is premultiplied by alpha must be left untouched.</p></li>
data-x="dom-PremultiplyAlpha-premultiply">premultiply</code></dfn>", the <var>output</var> that
is not premultiplied by alpha must have its color components <span
data-x="convert-to-premultiplied">multiplied by alpha</span> and that is premultiplied by alpha
must be left untouched.</p></li>

<li><p>If <var>val</var> is "<dfn enum-value for="PremultiplyAlpha"><code
data-x="dom-PremultiplyAlpha-none">none</code></dfn>", the <var>output</var> that is not
premultiplied by alpha must be left untouched and that is premultiplied by alpha must have
its color components divided by alpha.</p></li>
premultiplied by alpha must be left untouched and that is premultiplied by alpha must have its
color components <span data-x="convert-from-premultiplied">divided by alpha</span>.</p></li>
</ol>
</li>

Expand Down