"canvas" source type

[lucaswoj]

Creating a custom layer API that allows developers to draw directly into the map's canvas is a huge project.

A simpler solution (and perhaps a solution more in line with our users' needs) is to create a canvas source type which copies the contents of an external <canvas> element onto the map.

The implementation of this source type would be almost identical to the implementation of the video source type, which copies the contents of a <video> element onto the map.

We'll need to include an example of using this source type along with the implementation.

[lucaswoj]

I have some time to play with this today

[lucaswoj]

Quick note: performance ought to be good when using this with a 2D canvas context but not with a 3D (WebGL canvas context)

[lbud]

In progress in https://github.com/mapbox/mapbox-gl-js/tree/canvas_type (using 2D canvas context).
Example: http://bl.ocks.org/lbud/a1f6e07528cc62317309c0da5f00f3a2

[mourner]

@lbud wow, that looks great!

[mourner]

Quick note: performance ought to be good when using this with a 2D canvas context but not with a 3D (WebGL canvas context)

BTW, I just realized that while readPixels would hit hard, accepting a WebGL framebuffer texture instead should enable using WebGL here without the performance penalty. What do you think @lbud @ansis @lucaswoj?

[ansis]

accepting a WebGL framebuffer texture instead should enable using WebGL here without the performance penalty

This would be the performant way to do it. We'd have to expose our gl context and make sure we unset and reset all state before and after they render.

[lbud]

We'd have to expose our gl context and make sure we unset and reset all state before and after they render.

I'm not sure it's worth going down this road, at least at this point. I think the intent of the canvas source type proposal was to quickly enable users familiar with common HTML drawing techniques like HTML canvas to attach their graphics to the map, and IMO it may make most sense here to at least start with a 2D canvas context iteration. If people are really dying to attach their own WebGL framebuffers to the map, we can consider going down that road as well (and I'm certainly open to being convinced by some tangible use cases), but particularly at a time when we've also been talking more and more about how much we'd like to revisit custom shaders #281 — AND before we nail down more robust maintenance of our GL state #145 — it seems to me we should leave the 3D webgl framebuffer transference on the shelf.

[core44]

Great work! This feature could definitely open up a lot of possibilities for us. We've been playing around with trying to get live weather overlay which seems to work nicely. Although we're struggling with performance a little, the canvas animation seems to be running at a lot less fps than it should do. Would the correct approach for something like this be splitting up the animation into multiple canvas sources, rather than a single overlay?

[mourner]

@core44 slightly unrelated to the issue (canvas source type), but I've been recently working on a purely WebGL implementation of the wind animation. My goal is to design the custom layers API #281 so that it can accommodate such animations directly (drawing with the same GL context), without loss of performance, although it may take a while to get there.

[core44]

@mourner Incredible... cant stop staring at it! Certainly fits the 'Design and Publish Beautiful Maps' message. Repository watched. Thanks.

[core44]

Attached screenshot is the sort of performance loss we're seeing in comparison to the external canvas animation. It seems to be heavily downsampled. The animation should also be running at 20fps but it looks like we're getting more like 2 or 3fps. Is there any particular reason for this... and is there anything we might be able to do to squeeze out a few more frames per second?

[kronick]

A thought after trying out @lbud's implementation: I want to do custom rendering of data in my canvas layer from data in a vector tile source. I can use querySourceFeatures() to figure out what data is currently in the viewport, but only if I tie that source to an invisible layer or else the data never loads.

Can we add something like a data-source property to layer type=source? So I could do:

 map.addSource('canvas', {
  type: 'canvas',
  canvas: 'canvas',
  "coordinates": [
          [-81.490, 46.437],
          [-72.582, 46.437],
          [-72.582, 38.907],
          [-81.490, 38.907]
      ],
  "dimensions": [0, 0, 800, 655]
});

map.addSource('my-dataset', {
  type: 'vector',
  url: 'mapbox://me.a84f9d9'
});

map.addLayer({
  id: 'canvas',
  type: 'raster',
  source: 'canvas',
  data-source: 'my-dataset',
});

Instead of my workaround right now which has me adding an invisible layer like so:

map.addLayer({
    'id': 'my-dataset',
    'type': 'circle',
    'source': 'my-dataset',
    'paint': {
      'circle-color': 'rgba(255,255,255,0)'
    }
  }); 

[lbud]

@kronick that goes beyond the scope/intention of a canvas source type (but will be possible with the later-down-the-road custom layer type). The canvas source type should and will be pretty naïve, just copying pixels like the image and video source types.

[ibgreen]

@lucaswoj Continuing discussion from #3745 which was closed and redirected here. This post is mainly to restart our engagement.

Handling perspective seems to involve a bunch of tricky math (as usual). Assume we'll need to calculate a canvas size using a perspective enable fitBounds operation with both long/lats and pixel sizes set to cover the entire screen.

There are a number of implementation decisions we haven't made yet that'll affect your ability to render pitched geometries. We may decide to:
expose a transformation matrix to custom layers which encode the camera position
allow the canvas to be rendered unpitched and covering the whole viewport

The second option is actually quite attractive - and on the surface looks like the simplest to implement.
Since we already render deck.gl layers in a transparent canvas that we overlay on top of the mapbox map, and we already have a camera system that matches the map even when pitched. So with this simple addition/option to the canvas style, we'd at least be able to render a mix of 2D layers and styles:

• the base map (i.e some styles)
• some deck.gl layers (in a canvas)
• some more styles
• some additional layers (in a second canvas)

Thus providing basic interleaving of layers and styles.

This simple addition of a 'screen coordinate' option for the canvas style could be a big enabler for us, until we can get WebGL context access.