Skip to content

Terrain 3D#165

Closed
prozessor13 wants to merge 113 commits intomaplibre:mainfrom
prozessor13:terrain3d
Closed

Terrain 3D#165
prozessor13 wants to merge 113 commits intomaplibre:mainfrom
prozessor13:terrain3d

Conversation

@prozessor13
Copy link
Copy Markdown
Contributor

@prozessor13 prozessor13 commented Jun 2, 2021

A working prototype, which adds 3d-terrain to maplibre-gl with the following features:

Already done:

  • create a regular 3d-mesh.
  • render all layers, except symbols, into a framebuffer(s), which textures later used on the terrain-mesh.
  • add the third dimension to all symbols/circle/fill-extrusion layers and the collision index.
  • implentation, based on a coords-framebuffer, to unproject screen to mercator coordinates.
  • Elevate camera over terrain.
  • Correct lngLat coordinates in all mouse-events.
  • hide symbols behind mountains.
  • enable/disable terrain mode

Still open:

  • Correct calculation of coveringTiles in transform.js.
  • Test/Improve performance.
  • Write a document how the code works in detail.
  • document any changes to public APIs

Nice to have, but not on my agenda:

  • tilt more than 60 degrees & implement a sky.
  • add 3d dimension to showTileBoundaries & showCollisionBoxes.
  • ray/triangle calculation for pixel -> mercator projection. If this this done, the coords-framebuffer can be easily transformed to a depth-framebuffer.
  • 3D Camera animations.

Need Help:

  • confirm all changes do not include backports from Mapbox projects. I think this is already guaranteed, but it needs a confirmation!
  • correct calculation of transform.coveringTiles, because i do not have any math background in this!
  • testing & writing tests.
  • post benchmark scores

This code is already a work in progress! There are a some of FIXME-3D in code, also this is my first experience in webgl, so i am looking forward for feedback.

Also this code is not a copy of mapbox-gl 2.0, i initially looked onto the first 3 commits of mapbox-gl 2.0 at 8.Dec.2020, and than i started my work with a very different approach of implementation. For example:

  • all 3d is done with 3d-coordinates. mapbox uses 2d coordinates and calculates the 3d dimension in the shader.
  • unproject from screen to mercator coordinates is done via a coordinate-framebuffer, instead of ray/triangle calculation.

…ing the

development i did no intermediate commits, so in this first commit all the
following functionality is included:

* allow MTK terrainRGB-tiles encoding with parameters:
  [6553.6, 25.6, 0.03, 10000.0]. In our opinion 0.1 for height-steps is to
  rough.
* create a TerrainSourceCache.js class which is similar to SourceCache.js and
  holds all the terrain-tiles used for the 3D-mesh. If a terrainRGB tile is
  used in both, terrain & hillshading, it is loaded twice. This makes the
  whole process much more easy.
* create a 3d-mesh in raster_dem_tile_worker_source.js with the martini
  library.
* rewrite the draw logic to render all layers, except symbols, into a
  framebuffer. This framebuffer a later used as a texture onto the 3d-mesh.
* rewrite symbol rendering to use 3d-coordinates. This is done with an extra
  a_ele shader parameter, because the z-value of the a_pos variable is already
  used for other things.
* add the third dimension into the collision index.
* create a terrain.html test-page.
This is done with an extra framebuffer in which all tile coordinates are
rendered in an encoded rgb value. Dont know why, but this framebuffer looks
very inperformant. As advantage, grabbing coordinates from screen
is very fast. It may help to render the framebuffer only every 100ms instead
of every requestAnimationFrame?
This logic is currently used in
* map.project
* map.unproject.
Other usecases are:
* elevate camera over terrain
* use in mouse-events
* use in queryRenderedFeatures
@prozessor13 prozessor13 changed the title Terrain3d Terrain 3d Jun 2, 2021
@prozessor13 prozessor13 changed the title Terrain 3d Terrain 3D Jun 2, 2021
@davenquinn
Copy link
Copy Markdown

This is an excellent thing to see here. I want to mention that the aims and structure of this implementation are similar to that of the Cesium-Martini project, which has a working but prototypical implementation at Macrostrat. Some of the problems and potential solutions might be shared, especially as one of the major goals of the Cesium-based work is to get Mapbox vector maps working atop the platform.

@kylebarron
Copy link
Copy Markdown
Contributor

I agree with Daven that this is exciting to see. I haven't looked through this PR's changes yet, but one note is that it might be worthwhile to keep the mesh generation and mesh rendering loosely coupled if possible. Especially for high DPI screens with many tiles rendered, calling Martini on every tile can use significant CPU. Additionally, this approach uses extra network bandwidth because it requires downloading the heavier raster DEM files to create the mesh on the client.

If mesh generation and mesh rendering are loosely coupled, support for something like Quantized Mesh could also be implemented, where the mesh is generated on the server side, which avoids extra CPU usage on the client and allows for smaller downloads. I have a demo of rendering these quantized mesh tiles in deck.gl here if anyone's interested.

@AbelVM
Copy link
Copy Markdown

AbelVM commented Jun 6, 2021

Once this PR #30 is merged, we could use the same logic to load tiled DEMs 🚀

@HarelM
Copy link
Copy Markdown
Collaborator

HarelM commented Jun 6, 2021

Cool!!! I'd be happy to take if for a spin somehow and help testing this... But I need to migrate to maplibre first first, which requires #30... :-)
Does it use the RGBTerrain source to calculate the 3D?

@prozessor13
Copy link
Copy Markdown
Contributor Author

A few explanations:

  • this implementation uses custom terreinRGB tiles with the unpack-vector of [6553.6, 25.6, 0.03, 10000.0]. With this change we get much better results on the hillshading layer. But also mapbox & terrarium unpack-vectors can be used.
  • the hillshading is currently done via the normal hillshading layer, so the martini-tiles can have a relativ rough mesh, which is good for performance i think.
  • i know cesium-martini, and it was a great inspiration! I am looking forward to share knowlege.
  • i also looked at qantized-mesh, and after all major problems solved, i will take a deeper look onto it!

And my plan is to have a usable beta ready in the next few weeks.

@aakaydin
Copy link
Copy Markdown

Hi, great job with the terrain! I've looked at your code and for the moment all layers except the Symbol are drawn over terrain as a texture, is this going to be how those layers are rendered or do you want to in the future render them over the terrain directly, not just using a texture?

@prozessor13
Copy link
Copy Markdown
Contributor Author

Hi,

background, fill, linestring, raster & raster-dem render to texture, symbols, circle & fill-extrude are render ontop. I think this is the way to go.

@aakaydin
Copy link
Copy Markdown

Yeah, you are right, its also those layers that are rendered over the terrain in mapbox I think, can't wait to see the final results!

@HarelM
Copy link
Copy Markdown
Collaborator

HarelM commented Jun 20, 2021

OK! I've migrated my project to maplibre :-) (Although not yet published to production)
#30 was merged to main too so this is not blocking anymore.
I'd be happy to help testing this.
We have a map we are maintaining with a RGB terrain tiles which can be seen here:
https://israelhiking.osm.org.il/
I'd be happy to know if I can use the terrain RGB tiles I already have to render 3D terrain when using this feature (preferably when dragging the two fingers to change camera angle).
In any case, this is super exciting, let me know how I can help in order to test this, I'll be happy to introduce this feature into my app.

@prozessor13
Copy link
Copy Markdown
Contributor Author

I'd be happy to know if I can use the terrain RGB tiles I already have to render 3D terrain when using this feature (preferably when dragging the two fingers to change camera angle).

Yes, you can use any kind of raster-dem source. Simply add a useForTerrain: true (= maybe changed in future) into the source-definition.

In any case, this is super exciting, let me know how I can help in order to test this, I'll be happy to introduce this feature into my app.

If the early-alpha stage, as it is now, is completed it will be great to create some unit-tests :D

@prozessor13
Copy link
Copy Markdown
Contributor Author

prozessor13 commented Jun 25, 2021

So the 3D Code is currently already useable, but because it is my first experience in programming computer graphics, i am not sure if the implementation is state of the art.
So, at this point i think i need help from someone, who is familiar with either maplibre or webgl code, for making a review (best together with me).

Some of the things to validate/discuss:

  • render-to-texture pipeline
  • a coords-framebuffer is used instead of a depth-framebuffer. This framebufer contains for every pixel an encoded coordinate instead of the depth. I use this for 2 things: symbol-visiblity and as a replacement for a raycast to get the geographical coordinate.
  • covering tiles for the visible area is very hacky implemented
  • elevate camera over terrain, and set correct zoomlevel
  • ...

After the code is approved, i will start on performance and testing.

@HarelM
Copy link
Copy Markdown
Collaborator

HarelM commented Jun 25, 2021

I'd love to help, but I don't have the knowledge in either... :-( I can help test it though, probably :-) is there anything special I need to do in order to see 3D with the code in your branch?

@prozessor13
Copy link
Copy Markdown
Contributor Author

Simply add a terrain-dem source with the useForTerrain flag into a mapbox-gl style.json file: e.g.:

     "terrain": {
        "type": "raster-dem",
        "tileSize": 512,
        "tiles": ["https://vtc-cdn.maptoolkit.net/terrainrgb/{z}/{x}/{y}.png"],
        "encoding": "mtk",
        "useForTerrain": true,
        "maxzoom": 15
     },

@HarelM
Copy link
Copy Markdown
Collaborator

HarelM commented Jun 26, 2021

First and foremost, thanks a lot for all the effort invested in this, please don't take my criticism negatively, I just want to help push this, nothing more. I can't stress this enough...

I've created the following stackblitz:
https://stackblitz.com/edit/maplibre-3d-test?file=index.js
What I did to create it:

  1. Compile the branch locally
  2. Copy the css and minified js into the stackblitz files
  3. Added a style.json we are using in our site, and added the useForTerrain flag

The following is my observations:

  1. When I moved the map/zoom pan it seems to not draw the see of Galilee from time to time (the lake near the initial position in the linked stackblitz). Below is a screenshot of another location it happened:
    image

  2. The hill shading seems to stop at what seems to be zoom 12 which is the source of the tiles' max zoom, but in the regular version the hillshading it would continue to the next zoom levels.

  3. I couldn't see the 3D effect strongly, I'm not sure this is because there aren't big mountains around this area or that I needed to do something else (maybe related to our map definition of hillshade-exaggeration of 0.33)

I left the link to maplibre-gl.js in the index.html file so one can easily switch between the versions.

Thanks a lot again!
Let me know if there are more tests I can do.

@HarelM
Copy link
Copy Markdown
Collaborator

HarelM commented Jun 26, 2021

Here's how our initial testing looks :-)

3d-test-dead-sea.mp4

@prozessor13
Copy link
Copy Markdown
Contributor Author

Hello Harel,

thanks for your tests!

1 & 2: I will have a look into your stylesheet whats going wrong.
3: The exaggeration is currently not implemented, but this should be a small issue.

And great to see, that it basically works on mobile devices!

@HarelM
Copy link
Copy Markdown
Collaborator

HarelM commented Feb 9, 2022

As suggested in the meeting today:

  • Create a new pull request to allow review etc since this PR is too long.
  • Make sure the tests pass
  • Make sure to document how to use the feature - i.e. get started steps
  • After the PR is merged open the relevant issues in the repo describing in details what the problem is, where in the code you think this is coming from and hopefully someone will be able to pick this up.

Thanks again @prozessor13 !!

@acalcutt
Copy link
Copy Markdown
Contributor

acalcutt commented Feb 9, 2022

Just an FYI for merging this into main, so I don't clutter up a new PR.

I think this still breaks heatmap. I commented on it here ( prozessor13#46 ). I was able to get it working with terrain disabled in that thread, but the elevation wasn't right with terrain enabled.

@HarelM
Copy link
Copy Markdown
Collaborator

HarelM commented Feb 10, 2022

@prozessor13 please resolve conflict so the tests can run.

@HarelM
Copy link
Copy Markdown
Collaborator

HarelM commented Feb 13, 2022

@prozessor13 do you want to move the code in this PR to a branch under this repo? This way the guys here can help with tasks such as merging, conflict resolution etc. What do you say?

@wipfli
Copy link
Copy Markdown
Contributor

wipfli commented Feb 24, 2022

@prozessor13 I created a terrain3d branch in this repo to coordinate the merge process: https://github.com/maplibre/maplibre-gl-js/tree/terrain3d

@wipfli wipfli mentioned this pull request Feb 24, 2022
1 task
@wipfli
Copy link
Copy Markdown
Contributor

wipfli commented Feb 24, 2022

Closing in favor of #1022

@wipfli wipfli closed this Feb 24, 2022
@prozessor13
Copy link
Copy Markdown
Contributor Author

Great, Thx!

I currently working on performance to render the world with mapPitch=85. This is necessary to render tiles in respect to earth-radius. Without this you see Dubai from the peak of a mountain in Austria, which does not make any sense.
Additionally tomorrow we have a telco with the TU-Wien (Technical University Vienna), maybe they can help us to improve the camera-issues. I will report our progress in the new created Branch, or directly create Tickets for new Features.

Regards!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.