How does the window controls overlay fit with the page's layout?

[yoavweiss]

https://wicg.github.io/window-controls-overlay/#the-getboundingclientrect-method states "Web content SHOULD not be displayed beneath the overlay." but it's not clear how that should work.
Should the layout of the page pretend that the page is not rectangular? That an inline box is already present there when layout reaches that point? Something else?

[diekus]

Added information about how the DOM Rect and page layout behave with WCO.

[yoavweiss]

I'm not sure that's sufficient for an interoperable implementation. /cc @chrishtr for opinions

[inexorabletash]

I left a comment in email, but repeating it here:

"Web content SHOULD not be displayed beneath the overlay."

That "SHOULD NOT" is a term so the whole thing should be capitalized... but it's also unclear who this is written for. Is this guidance for web authors, indicating that if they opt-in to WCO then it's best if they use CSS to ensure no critical content appears beneath the WCO? Or is this for implementers, indicating....? If it's for web authors, then it shouldn't be normative text, which seems covered by the "The client viewport does not..." note?

I agree that Yoav's comment isn't addressed yet. Maybe a picture like this would help:

Without WCO:

With WCO:

[diekus]

Hola, I have added clarification on when the algorithm should run (pending merge still while I finish adding images of the client area as pictured above).

This should clarify how the content fits the page and what is the area being returned by the method in question.

[yoavweiss]

https://wicg.github.io/window-controls-overlay/#the-getboundingclientrect-method is now much clearer about how this is handled by browsers.
One last nit: can you drop the "must" from the note and replace it with a non-normative word? (e.g. "need to")

[chrishtr]

What if the controls overlay region is not aligned to the left or right of the display?

In the latter case does the API to query available regions in the title bar need to be a list of rects, similar to Element.getClientRects?

What if the controls overlay region is not a rectangle? Should it be required by this spec that it be a rectangle?

This situation is similar to the mobile "notch" situation, where certain mobile phones have a non-rectangular screen. This was solved with just environment variables, which I think only tell the web content which rectangular area is definitely safe to use, and does not attempt to offer up which areas next to the notch are safe also.

I assume it's really important for the web app to be able to use the pixels in the green rectangle area?

[chrishtr]

Will the safe area environment variables I mentioned above be set in the case of this API? It would seem that's needed, because the developers should use that rectangle for their regular content, and only put specially positioned informational or control elements in the LHS of the controls overlay bar.

Also a nit: getBoundingClientRect doesn't seem like a great name in this situation. Its name should indicate its purpose, which is to provide the available area for the web app to draw in.

[diekus]

Hola @chrishtr. I would expect the safe-area-inset-* variables to be set for when a web app is running on mobile notched devices and the title-bar-area-* variables set as well for when the app is displayed in desktop environments, but I don't think there is a case when both need to work at the same time? (please let me know if I am mistaken or if there's maybe some new device you're shipping I have no context of?)

About the non-rectangular overlay, having the bounds of the overlay would still be viable to avoid putting content beneath it. Is there any special consideration you think we are missing?

Regarding overlays that aren't aligned to the left or right or the frame, MacOS is an example where there is overly to both sides of the window and the API still handles that use case. I do not think there is a use case where a title bar has elements that create more than 1 title bar area, but again, please let me know if there is any case where we would need to readjust the API!

Thanks for the feedback, good food for thought!

[chrishtr]

Hola @chrishtr. I would expect the safe-area-inset-* variables to be set for when a web app is running on mobile notched devices and the title-bar-area-* variables set as well for when the app is displayed in desktop environments, but I don't think there is a case when both need to work at the same time?

For the "desktop environment" use case, everything except developer-drawn window controls should be in the safe area (the area of the browser viewport excluding the whole title bar). Do you agree? If so, then how is the developer supposed to figure out the bounds of the safe area and use it in CSS without safe area environment variables?

The title bar this spec has environment variables, but there are none for the safe area.

Regarding overlays that aren't aligned to the left or right or the frame, MacOS is an example where there is overly to both sides of the window and the API still handles that use case. I do not think there is a use case where a title bar has elements that create more than 1 title bar area, but again, please let me know if there is any case where we would need to readjust the API!

What if the OS title bar area is centered? Then the developer-available title bar will be two separate rectangles, to the left and right of the OS title bar. This situation has sort of occurred in real life already, via notches.

[diekus]

Hola @chrishtr. The developer does not draw any window controls, this feature only expands the viewport and provides a JS and CSS way of positioning content on the area that was previously occupied by the title bar. I don't quite understand why a window on the desktop with the feature on would need to specify safe area with inset variables?

About the centered title bar, I do not think it is something that exists atm in desktop environments? If you do have a use case please let us know, but this is not supported because there is no use case that follows this behavior in any supported platform.

I believe even in MacOS on the new notched machines there is never a case when a window's app can maximize to intersect the notch. The notch only clashes with the top menu bar.

[diekus]

Also, to clarify, titlebar-area-height == safe-area-inset-top in this case. Maybe we are looking at the same aspect form different angles!

[chrishtr]

I think I was confused a bit about what the window control overlay is. In this image:

It calls the blue area the "window controls" and "(controls) overlay", but the WindowControlOverlay is in fact the area labeled "title bar area" in the image, correct? If so, then I agree that getBoundingClient(s) rect is a fine name.)

I don't quite understand why a window on the desktop with the feature on would need to specify safe area with inset variables?

How else would content avoid overlapping with the blue OS window control area? As you diagrammed, when the WindowControlOverlay feature takes effect, the client rect for the developer overlaps the blue area.

I believe even in MacOS on the new notched machines there is never a case when a window's app can maximize to intersect the notch. The notch only clashes with the top menu bar.

On a notched mobile device, the notch is usually in the middle. Wouldn't it be nice to expose the sides to the developer, in the same way you're suggesting it on desktop?

[diekus]

It calls the blue area the "window controls" and "(controls) overlay", but the WindowControlOverlay is in fact the area labelled "title bar area" in the image, correct? If so, then I agree that getBoundingClient(s) rect is a fine name.)

Yes, the getClientBoundsRect is referring to the title bar area (the one in light blue). The red and green sections are there to define the concepts that are used on the spec. In several parts of the spec there are mentions to what are the window controls (minimize, restore, close) for example.

How else would content avoid overlapping with the blue OS window control area? As you diagrammed, when the WindowControlOverlay feature takes effect, the client rect for the developer overlaps the blue area.

Yes, the viewport will have areas that are covered by the controls in the corner. The way to avoid content underneath them is by 1- using the CSS environmental variables or 2- the getBoundingClientRect that represent the same area (the one depicted in light blue as Title bar area).

On a notched mobile device, the notch is usually in the middle. Wouldn't it be nice to expose the sides to the developer, in the same way you're suggesting it on desktop?

This is interesting, it is not supported atm. WCO is a feature primarily designed for supporting the use case of being able to create custom title bars on desktop environments. I believe on mobile the inset variables are the ones that define the safe area for content in general, and while this does not include laying content on the sides of a notch, it is not a use case that WCO aims to resolve. I am certain that if a case comes when a supported platform has controls on its windows that are centered or similar then it is something that will need to be considered.

[chrishtr]

Yes, the viewport will have areas that are covered by the controls in the corner. The way to avoid content underneath them is by 1- using the CSS environmental variables or 2- the getBoundingClientRect that represent the same area (the one depicted in light blue as Title bar area).

I guess developers can use titlebar-area-height to set a top margin for their main content. And there examples of sites successfully using the existing API, so that's good.

This is interesting, it is not supported atm. WCO is a feature primarily designed for supporting the use case of being able to create custom title bars on desktop environments. I believe on mobile the inset variables are the ones that define the safe area for content in general, and while this does not include laying content on the sides of a notch, it is not a use case that WCO aims to resolve. I am certain that if a case comes when a supported platform has controls on its windows that are centered or similar then it is something that will need to be considered.

This API should be forward compatible with possible extensions when possible. The example I gave is a very clear possible extension, so the API should accommodate that. I'm not proposing that you have to support that situation right now. It can be as simple as using 'titlebar-area-left-*' as the names of the environment variables, and having the existing safe area environment variables be exposed.

I don't think these changes are hard or difficult to implement. Is there a different concern you have about such changes?

[diekus]

I understand. We believe the the API is forward compatible. If a browser decided to implement this feature on say, Samsung DeX running over android, it would be compatible. Same as if any browser in Linux were to implement this API with a GUI like KDE, Deepin DDE, Xfce, Budgie, LXDE, ...

When the API was being developed we looked at trying to work around notch, but quickly realized that it would become cumbersome to develop for since for example, windows in macOS have the overlay region separated in 2 different regions, (basically 2 notches) and that it made using the feature more cumbersome as a whole trying to position elements in N notches by M areas. To this we add the fact that there is no implementation or windowing system on desktop environments that behaves in the described way. This would imply sacrificing ease of use of the feature to support a use case that does not exist.

The Window Controls Overlay for installed Desktop web apps feature aims at allowing the developer to create custom title bars, and it tries to do it in a way that is straightforward, by allowing the developer to easily identify in CSS and JS the available area they can use to do so.

[chrishtr]

What about my specific suggestion of being forward-compatible with a center notch? The changes required would be:
(a) rename env variables
(b) (optional) also expose the existing safe mode env variables
(c) rename getBoundingClientRect to getBoundingClientRects

[diekus]

I've discussed with the feature crew and we dont think we want to make the API more complex by enabling support for center notches/multiple notches on title bars because it is not a UX pattern that exists on desktop platforms.

We appreciate the suggestions as it gave us different cases to consider we might have not thought of.

[yoavweiss]

I think it'd be interesting to flesh out this potential complexity vs. future compatibility and the make an informed decision based on that.

[slightlyoff]

Hey all,

Chatting with @diekus, @mwjacksonmsft, Hassan, and @amandabaker about this today, and I think we have a crossed wire in this thread.

Per @amandabaker and @diekus investigations, NO OS PUTS WINDOW CONTROLS IN A NOTCH-INTERSECTING AREA.

This makes sense: OSes that provide windows have decades of legacy programs that are not notch-aware and cannot be easily updated, and so they have gone out of their way to ensure that we don't actually create a situation where UI might be hidden by notches. Instead, they have reserved those areas to sections under OS control for drawing content. The web wasn't this lucky when it comes to the content area on, e.g., iOS, but that's a separate concern that's already covered by CSS.

Should some future OS arrive with windowing that intersects notches, it could be handled through a compatible extension to this API to return a set of rects (rather than a single rect) + an event or observer to notify. Full-screen content + notches are potentially one place where an OS could create an adjacent problem, but that's already covered with the existing CSS accommodations for content areas.

For the reasons that have kept them from introducing this issue to windowed content today, we do not anticipate OSes introducing notch/window-control overlaps, and I don't think it's wise to be pre-gaming OS features with web API designs.

Thoughts?

[chrishtr]

It may be unlikely for OSes to introduce notch/window-control overlaps, but there is still an opportunity to solve a more general problem than "make it possible to put web content next to a partial desktop OS window control in the title bar".

The more general problem I think this API should be forward-compatible with is "provide a way for web content to be put in an area not obscured by something outside of the web content". One example of "something" that exists today is an OS window controls. Another that also exists today is notches (which appear on both iOS and Android for some phones, BTW).

Furthermore, there is precedent for platform APIs that allow for this situation, such as on Android.

The forward compatibility can I think be achieved with the following small changes to the API:

1. Rename to getClientRects and return an array. For now, the array would always be of length 1. This naming also has a precedent in the DOM API, for cases where an element is split across multiple areas.

2. (optional) Expose the existing environment variables for the safe area (which I think would be an ergonomic improvement to this API and make it align with the existing safe area variables for mobile web apps/sites).

3. (optional) Consider renaming the environment variables to be a bit more forward-compatible.

So to summarize, all you would have to do for now is rename getBoundingClientRect to getClientRects, and call it a day.

[diekus]

I think it is a very different thing a notch than an overlay of window controls. One has a fixed position in hardware and the other one is always 'attached' to the window. The window can be in different positions in the screen, and, in the hypothetical case that Chris mentions that a notch and a window intersect, it gets messier. How does content flow? Is a notch seen the same as an overlay while dragging? What if a notch touches an overlay of a window? I tend to agree with @slightlyoff this is a case unlikely to happen.

I think if the plan is to solve this for notches, then it is better to look into a different approach that can piggy back of the shape media feature or the shape-inside property for example. Trying to use "Window Controls Overlay" as the solution for all irregular shaped viewports can create confusing scenarios and complicate the use of the env variables as well.

Are all notches created equal? To be or notch to be? That is the question.

We've looked in the past at ways of treating the overlay as a notch and it doubles the number of CSS variables just for one notch in the center (which again, does not exist in any platform today).

Are we overly complicating an API and possibly creating friction to adopt it in its current straightforward way to cater to a UI trend that might go away when under display cameras and sensors become a thing in a couple of years?

The trend of notched phones today is far less than a year ago, I worry we sacrifice API usability for a passing UI trend. Just my 2 cents!

*edited to remove all the typos

[chrishtr]

notch in the center (which again, does not exist in any platform today).

Platform or device? Doesn't the Android thing I linked to provide a platform API?

I don't know why you call a notch hypothetical. They exist in real devices, and clearly intersect the web window.

[diekus]

Sorry for the tons of typos, was eating the best burger ever while I typed the previous message on my phone!

Platform or device? Doesn't the Android thing I linked to provide a platform API?

Yes! what I mean is that there is not platform or device where the notch intersects a window title bar. Oh the notch is not hypothetical. What is hypothetical is notches interfering with windows and overlays of controls in those windows.

Do you think addressing irregular viewport spaces (N notches in several different edges (no pun intended)) might benefit from a different API that could cater to the more universal case I can see you are trying to solve?

[chrishtr]

What is hypothetical is notches interfering with windows and overlays of controls in those windows.

I see. Notches do intersect with a fullscreen website window on mobile devices, but you're right that it doesn't intersect with the mobile window controls.

Given that, it does seem a different enough use case.

That leaves one final question: why are safe-area-inset-* not appropriate in this case? Is it because safe-area-inset-top is redundant with titlebar-area-height ?

[diekus]

I would expect a fully responsive web app to use safe-area-inset-* vars to make sure their app looks appropriately on an iPhone and the titlebar-area-* vars to accommodate certain content in the title bar region of a window. I don't think they are not "not appropriate" per se... but the titlebar-area-* provide a better more understandable naming for what they are achieving in the context of WCO.

Additionally while both the titlebar-area-* and the safe-area-inset-* are of "types" length, the semantics of them are not quite the same as one is a mostly used/understood as "space to leave from the margins of the 'viewport'" and the other one is a coordinate in the viewport and the width and height from that coordinate that define the title bar area. I think for ease of use the titlebar vars can be better suited.

[chrishtr]

Ok fair enough. I'm satisfied. :) Thanks for the useful responses!

[inexorabletash]

I think it'd be worth having samples somewhere that show how to build a PWA that works across both mobile-with-notch (using safe areas) and opts-in to WCO on desktop, showing how the same site can provide a modern, native-caliber experience everywhere.

(I have a hobby site PWA that takes advantage of covering the full viewport / respecting the safe area on mobile, but I haven't integrated WCO.... yet!)

[diekus]

@inexorabletash I agree. I am working on putting together a repo with sample code to enable this type of responsive design.