[css-overflow] Consider adding new CSS primitive for per-element panning and zooming (for web maps)

[zcorpan]

See Maps4HTML/MapML-Specification#71

The Maps for HTML Community Group explores ways to make web maps a first-class feature on the web platform. Styling with CSS is desired, but is not yet fleshed out how it would work.

From what I can tell, web maps commonly use an interaction model that the web doesn't support natively: panning and zooming a single element.

Panning is similar to scrolling, but not quite. Scrolling areas have edges. Panning a web map can wrap around the edges. On desktop, panning a web map is typically done by a drag and drop gesture, and zooming is done by scrolling gesture. On touch-based devices, panning is typically done by swipe gesture with one or two fingers, and zooming by pinch gesture.

While browsers support “page zoom” and “pinch zoom”, there is no per-element zooming primitive.

Currently, JS-based implementations of web maps solve this by using touch/mouse events and preventing the default action (such as page scrolling and pinch zooming). For example, leaflet.js calls stopPropagation() for mousewheel events.

Instead of web maps having to hijack events and implement a custom interaction model, I wonder if this could be supported as a first-class feature that you can opt into. This could have various benefits:

• improved performance
  • the UA could perform panning and zooming in the composition thread, like scrolling. Today's JS-based solutions that respond to events will lag behind with a tick of the event loop, and are subject to jank if another task occupies the main thread.
  • non-passive mousewheel/touch event listeners negatively impact scrolling performance. (Also see https://developers.google.com/web/updates/2016/06/passive-event-listeners )
  • with the implementation being part of the UA, the amount of necessary JS is reduced, which lowers network transfer sizes and JS compilation time.
• support for different/new device form factors: similar to <video controls> or <select>, the UA can adapt the experience to make sense for new device form factors (desktop vs phone vs watch vs in-car navigation system).
• accessibility: a first-class interaction model on the web should be usable for everyone, including when using assistive technologies. I don't know what the current situation is like or what the challenges are. But custom JS-based implementations might neglect this aspect, potentially leaving web maps inaccessible.
• better web developer ergonomics

Also see https://maps4html.org/HTML-Map-Element-UseCases-Requirements/#use-case-set-view-zoom for documentation on this use case and the required capabilities.

[svgeesus]

From what I can tell, web maps commonly use an interaction model that the web doesn't support natively: panning and zooming a single element.

This was also the interaction model for SVG in the plug-ins era (Adobe, Corel etc) but then switched to scrollbars (and no real zoom) in the native browser era. It is still a useful and more convenient way to explore a rich graphic.

So I'm in favor of supporting this interaction model without hacks.

[css-meeting-bot]

The CSS Working Group just discussed Zooming/Panning per-element.

The full IRC log of that discussion

<fantasai> Topic: Zooming/Panning per-element
<fantasai> github: https://github.com//issues/5275
<fantasai> AmeliaBR: This has come up with work I've been doing, and zcorpan was reviewing, on standardizing map viewers
<fantasai> AmeliaBR: A map embed, where you move around the map on the screen and zoom in / out
<fantasai> AmeliaBR: There's no standard way to describe that behavior in specs
<fantasai> AmeliaBR: But it is a very common pattern, with common interaction patterns
<fantasai> AmeliaBR: Mouse behaves a certain way, touch behaves a certain wain
<fantasai> AmeliaBR: In addition to map viewers, also common for product photos on a retail site
<fantasai> AmeliaBR: data visualizations
<fantasai> AmeliaBR: Some things to think about..
<fantasai> AmeliaBR: two different aspects of typical pan+zoom impl
<fantasai> AmeliaBR: One is the UI and interaction patterns
<tantek> listens earnestly, to understand how "pan" is materially different from "scroll" in terms of the control that 'overflow' affords.
<fantasai> AmeliaBR: how the end-user indicates, by gestures or whatever, where they want to go and andzooming in / out
<fantasai> AmeliaBR: The other aspect is loading content, e.g. higher-resolution
<fantasai> AmeliaBR: From a CSS perspective, the are going to be two independent things to think about
<fantasai> AmeliaBR: Loading in separate content, have something similar to a MQ scoped to a particular container that has a zoom state
<fantasai> AmeliaBR: triggers picture sources or whatever
<emilio> q+
<fantasai> AmeliaBR: Basic prerequisite for that is to have a standardized way to describe pan + zoom
<TabAtkins> q+
<fantasai> AmeliaBR: Which as a lot of similarities to scrolling environments
<heycam> q+
<fantasai> AmeliaBR: but the standard actions are a little different, different translation of input
<smfr> q+
<fantasai> AmeliaBR: different established pattern of movement
<fantasai> AmeliaBR: Other thing is that you don't necessarily start at the beginning and scroll down, often starting in the middle
<fantasai> AmeliaBR: and usually not showing scrollbars
<fantasai> AmeliaBR: So established pattern of panning , semi-standardized
<fantasai> AmeliaBR: not always implemented in a way that is keyboard-accessible, or devices otherwise unique wrt inputs
<fantasai> AmeliaBR: So much easier for authors to have a standard way to switch into this mode, don't have to code up a lot of JS
<fantasai> AmeliaBR: and can make it more cross-platform and accessible
<tantek> interesting, Google Maps is a *mix* of infinite pan in the horizontal direction, while having blocking scrolling in the vertical direction!
<fantasai> AmeliaBR: No proposal yet, but wanted to bring up to CSSWG, is it something appropriate to address in CSS? Any opinions on how?
<fremy> q+
<astearns> ack emilio
<fantasai> emilio: I was chatting about this with some of the zooming folks in Gecko
<fantasai> emilio: Zooming and panning is limited to top-level content document
<fantasai> emilio: Depends on specific use cases you want to solve
<fantasai> emilio: Not enabling for iframes etc, is artificial limitation
<fantasai> emilio: Depending on set of use cases, documents already have all these APIs, visual viewport API etc.
<fantasai> emilio: allow authors to react to these things
<fantasai> emilio: depending on set of use cases, if enabling on iframes solves them, could be relatively easy to do
<fantasai> AmeliaBR: That's interesting. So describing it as the same pinch-zoom behavior that we have in most mobile browsers
<fantasai> AmeliaBR: but scoped to a specific elmeent
<fantasai> AmeliaBR: one thing there, I don't know enough about mobile a11y if there's standard mapping to keyboard controls for pinch-zoom view
<astearns> ack tantek
<astearns> ack TabAtkins
<fantasai> TabAtkins: I think this is great and interesting
<chrishtr> q+
<fantasai> TabAtkins: especially on touch devices, interaction is similar wrt scroll vs pan
<fantasai> TabAtkins: but being able to opt into more pan+zoom interaction easily on desktop would be useful
<fantasai> TabAtkins: You said something about being media-query?
<fantasai> TabAtkins: Needs to be a property modifying how we handle overflow
<fantasai> TabAtkins: particularly because of wrapping behavior necessary for some cases
<fantasai> AmeliaBR: MQ thing was for the loading different resolution images
<fantasai> AmeliaBR: e.g. in a picture source
<fantasai> TabAtkins: Talking about level of detail
<fantasai> ?
<fantasai> AmeliaBR: Yes
<fantasai> TabAtkins: Level of detail is a much more complicated topic
<fantasai> TabAtkins: if we can separate from overflow interaction... it's much harder
<fantasai> AmeliaBR: Agree it's the next level, and first level would be to standardize pan-zoom interaction
<tantek> Yandex maps also only does (infinite) pan horizontally and (blocking) scroll vertically.
<fantasai> AmeliaBR: and zoom part of it, so long as it's exposed to authors so they can detect the desired level
<fantasai> AmeliaBR: that can be JS at first
<fantasai> AmeliaBR: and work on declarative way to do that as we go along
<fremy> q?
<fantasai> AmeliaBR: You brought up another point, tantek brought up too,
<fantasai> AmeliaBR: in some environments, such as maps, you want a wrap-around effect
<fantasai> AmeliaBR: if you scroll to the end, scrolls back to the beginning; spherical or cylindrical coordinate system
<fantasai> AmeliaBR: Other areas in CSS where it might be useful, like an image gallery
<fantasai> AmeliaBR: where you scroll to the end, end up at the beginning
<fantasai> AmeliaBR: Again, slightly separate from panning interaction itself
<fantasai> AmeliaBR: but is also a relevant part of the discussions
<astearns> ack heycam
<fantasai> heycam: I think it's worthwhile thinking about this further
<fantasai> heycam: My one concern is that we know what the full set of requirements are around panning and zooming
<fantasai> heycam: so that we don't design a feature that goes 90% of the way, but the last 10% requires reimplementing everything
<fantasai> heycam: E.g. some JS things let you pinch-rotate the map
<fantasai> heycam: if not a good way to do that, they authors might feel like re-implementing everything
<tantek> oh yeah, there's also two-finger-tilt in some maps!
<fantasai> heycam: Might want to list what are the missing features, that doesn't allow doing maps online
<fantasai> heycam: what's missing/janky
<fantasai> heycam: so we know the design space
<astearns> ack smfr
<fantasai> smfr: Echo some of heycam's concerns
<fantasai> smfr: Differnet clients have different needs
<fantasai> smfr: some have very specific requirements
<fantasai> smfr: anything we make might not be enoug, end up using JS again
<fantasai> smfr: Maybe we should focus on primitives that ppl require to do this
<fantasai> smfr: one is always getting gestures right
<fantasai> smfr: sometimes it's vertical scroll, sometimes it's pinch-zooming, inconsistent
<fantasai> smfr: WebKit had some non-standard events like this
<fantasai> smfr: Differerent areas of the page with different zoom levels, could have confusion wrt MQ or something
<fantasai> smfr: Need to figure that out
<tantek> q?
<fantasai> AmeliaBR: Just answering to last bit about CSS transforms, that's how I picture it
<fantasai> AmeliaBR: similar to CSS transforms or SVG viewbox, where it is a localized transform that gets added to that stack
<fantasai> AmeliaBR: That's how it was originally specced for SVG, that there would be a user nteraction that ends up changing the viewbox transform
<fantasai> AmeliaBR: that never got implemented in browser versions of SVG
<Rossen_> Q?
<astearns> ack dbaron
<Rossen_> q+
<fantasai> dbaron: I agree with a lot of the concerns brought up so fora
<fantasai> dbaron: a lot of panning and zooming is complex behavior
<fantasai> dbaron: as mentioned
<fantasai> dbaron: Also many sites change fragment of the URL in response to panning and zooming, so you can bookmark where you are
<fantasai> dbaron: An interesting thing to investigate, if we do or don't want to do something, requires a more concrete proposal about something to add
<fantasai> dbaron: and an analysis of how that thing is useful for all the consumers that are out there
<tantek> q?
<astearns> ack fremy
<fantasai> fremy: I think that's similar to what other said before
<fantasai> fremy: map use case, it's unlikely to desire to do in CSS
<tantek> could see pan as an alternative to scroll, meaning it wraps around in that dimension. that seems like a nice incremental enhancement.
<fantasai> fremy: When you start to do zooming, dimensions of pixel changes over time
<fantasai> fremy: with dynamic contents
<fantasai> fremy: it's easier to adapt yourself as zooming
<fantasai> fremy: than to back-propagate to elements shown
<fantasai> fremy: For use case for maps, highlly unlikely anyone would prefer CSS solution
<fantasai> fremy: but other cases
<fantasai> fremy: if you include PDF or SVG documents
<fantasai> fremy: Sometimes it's really nice to be able to zoom on them
<fantasai> fremy: and these contents do not react ot gestures
<fantasai> fremy: so more reasonable use case to focus on these
<fantasai> fremy: so you really want to provide view on fixed contents
<fantasai> fremy: for maps, need to know where user is, fetching data to get new info, new tiles, etc.
<fantasai> fremy: I'm not sure it would work for a world map to fit in CSS pixels
<fantasai> fremy: At least for dynamic use cases, seems highly unlikely for CSS to provide a solution
<fantasai> fremy: so if working on that, suggest to focus on static use cases
<fantasai> fremy: you want the effect, but don't want to include an entire library
<astearns> ack chrishtr
<fantasai> chrishtr: In addition to what fremy said, about dynamic scenario, where devs want top optimize pixels at different zoom levels and to ??
<dbaron> Example of the pan/zoom results being in the URL (in a very common format for putting it in the URL): https://www.openstreetmap.org/#map=18/43.45397/-80.49867
<fantasai> chrishtr: a lot of maps these days are drawn with WebGL anyway, I don't think this approach would work with WebGL
<Rossen__> q/
<Rossen__> q?
<fantasai> AmeliaBR: Could work if you have a top-level DIV which maintains interaction with users, and passes things as events
<fantasai> AmeliaBR: just as many things use position on scrollbar to trigger things, but still using normal scrolling
<fantasai> chrishtr: common gestures across sites would be useful
<astearns> ack Rossen__
<fantasai> Rossen__: I think most everything was said already
<fantasai> Rossen__: Want to underline here, still the motivation or use case here involves quite a bit of complexity
<fantasai> Rossen__: zoom as a gesture, as a capability of a presentation system, is a very overloaded behavior, that has many differnet parts that more often than not apps are hooking into
<fantasai> Rossen__: initiation of zoom vs when it is already steady
<fantasai> Rossen__: visual zoom, layout-affecting zoom
<fantasai> Rossen__: semantic zoom, going through levels of different categories of app logic
<fantasai> Rossen__: loads / unloads resources
<fantasai> Rossen__: so this is also a type of zoom
<fantasai> Rossen__: Tab summarized as, what are primitives we're missing here?
<fantasai> Rossen__: when we thought about this in the past
<fantasai> Rossen__: and attempted to extend zoom for application developers
<fantasai> Rossen__: the main point was, what are the primitives that a platform such as CSS needs to provide
<fantasai> Rossen__: capabilities and observability of it
<fantasai> Rossen__: to extend that we do to today, quite far on this journey
<fantasai> Rossen__: can observe types of transforms applied, etc.
<fantasai> Rossen__: but you also mentioned something interesting
<fantasai> Rossen__: around a11y, and this being able to help a11y
<TabAtkins> q+
<fantasai> Rossen__: whad do you mean by that?
<astearns> ack Rossen__
<fantasai> Rossen__: is that the ability to use zoom as an intent? or are there experiences broken today?
<TabAtkins> ack Rossen_
<tantek> gmap-pedometer is also horizontal pan, vertical scroll
<fantasai> AmeliaBR: Specifically about existing author code and libraries aren't necessarily working well with different ways ppl interact with their computer
<fantasai> AmeliaBR: pan and zoom via keyboard controls is very opaque and inconsistent from site to site
<fantasai> AmeliaBR: just by standardizing it, we could move towards the browser controlling that standard interaction
<fantasai> AmeliaBR: and making sure it's accessible for all the ways ppl interact with their computre
<fantasai> AmeliaBR: being able to zoom is also an a11y issue, but not necessarily going to line up with author's decision that certain content is zoomable
<fantasai> Rossen__: Usually what we specify here, we specify differnet modality of input
<fantasai> Rossen__: and capability of apps to provide the zoom experience or bheavior
<fantasai> Rossen__: whether [lists input actions]
<fantasai> Rossen__: reality is these will differ vastly between capabilities of device
<fantasai> Rossen__: and standardizing on these types of gestures or higher-level intents
<tantek> really dislikes when you're using scroling gestures to scroll down a page, then you run into a full width map and your page stops scrolling, and the map starts zooming. What a disaster of overloading the same gesture to do/mean different things. Can we fix that? or at least not create something (primitives) that would make it worse?
<fantasai> Rossen__: will be more troublesome than if we identify primitives missing in the platform to allow awareness of this thing happening
<fantasai> TabAtkins: I strongly agree with idea that this would be really useful for a11y, because not relying on bespoke and incomplete JS solutions to a very common modality
<fantasai> TabAtkins: as something becomes common, if it's addressed by random JS solution, won't be accessible to ppl not having same setup as JS author
<fantasai> TabAtkins: captureing, as Amelia says, that this is meant to be a pan/zoom thing
<fantasai> TabAtkins: As dbaron says, we need a concrete proposal
<fantasai> TabAtkins: I propose two parts
<tantek> +1 TabAtkins
<fantasai> TabAtkins: 1. new generic property that switches into pan+zoom mode
<fantasai> TabAtkins: 2. A gesture-based JS approach, so you can hook that, and then roll your own pan+ zoom
<fantasai> TabAtkins: so you can roll your own, as ppl currently do with 'overflow: hidden'
<fantasai> TabAtkins: So, CSS side that does things automatically as long as you do simple thing
<smfr> not scroll events (they are reactive, not prescriptive)
<fantasai> TabAtkins: and JS side that standardizes pan+zoom gestures, and lets ppl hook that to do further stuff
<smfr> say input events like mouse and wheel
<fantasai> TabAtkins: build on top of mouse/keyboard gestures etc.
<astearns> ack TabAtkins
<fantasai> AmeliaBR: That's a useful breakdown
<tantek> was tilt explicitly excluded or just not brought up?
<TabAtkins> not brought up
<tantek> e.g. Google maps on a touch device, two finger up/down gesture will tilt the map up/down
<fantasai> AmeliaBR: Also need to take into consideration what exactly overflow would look like, all dfiferent things that need to be considered.
<fantasai> AmeliaBR: semantic gesture events, for pan+zoom
<fantasai> AmeliaBR: One question, who would be standardizing group for adding new events?
<fantasai> TabAtkins: I think WICG until it gets picked up by DOM?
<fantasai> fremy: But we do have CSS events too
<fantasai> smfr: We also have scroll events
<fantasai> smfr: You want all the input events to be defined in the same place
<TabAtkins> ScribeNick: TabAtkins
<astearns> ack fantasai
<TabAtkins> fantasai: In addition to capturing the gestures tha twoudl correspond to pan/zoom/etc
<tantek> pan / zoom / tilt / rotate
<TabAtkins> fantasai: I think ifyou had pan/zoom mode for overflow, we have currently a lot of APIs for getting the current scroll position and direct a widget to go to that position
<TabAtkins> fantasai: You'd probably want analogous methods that would let you do all the manipulations in scripts
<tantek> +1 fantasai, zoom level equivalents to scroll settings
<tantek> The only request I have is to include tilt & rotate in pan & zoom considerations / designs
<fantasai> s/do all the manipulations in script/do the same things in pan+zoom, e.g. get zoom level or set zoom level/
<fantasai> ScribeNick: fantasai

[jakearchibald]

We're implementing panning and pinch-zooming for squoosh.app, and as such we don't have platform physics. We also have a split viewport, where half the screen shows one thing, and the other half shows another, but it acts as one surface. I guess we'd need events from one pinch-zoom surface that we can replicate elsewhere.

Users have also asked for looping scrolling to implement carousels.

[prushforth]

I gave a presentation on the key features of our proposal for panning and zooming in Web maps (MapML), which currently hinge on what we call "tiled coordinate reference systems". (coordinate reference systems are coordinate systems that are referenced to the surface of the earth, roughly). The reason that we think the browser should understand TCRS is so that the browser rendering engine can support rendering of map primitives (layers, tiles, features, images and regular HTML). So as not to pollute this issue too much, please comment over on WICG.

[Malvoz]

• accessibility: a first-class interaction model on the web should be usable for everyone, including when using assistive technologies. I don't know what the current situation is like or what the challenges are. But custom JS-based implementations might neglect this aspect, potentially leaving web maps inaccessible.

Generally, pan and zoom in JS-based maps is not meaningful to non-visual users: Maps4HTML/MapML.js#396.

[rodighiero]

It might be useful to take a look at https://github.com/davidfig/pixi-viewport. It is an interface for zooming created for pixi.js, a library for video games that I use to boost my network visualizations in WebGL. There is a demo that features all the functions of viewport at https://davidfig.github.io/pixi-viewport/

[prushforth]

The mapml polyfill implements a "2D form"-like control that is the heart of how Web maps work, from Google Maps to every other library. The <map-extent> and related child element roles are described here:

https://maps4html.org/web-map-doc/docs/elements/extent/

and <map-input>,

https://maps4html.org/web-map-doc/docs/elements/input/