Huge slowdown when raycasting large HeightmapCollider

[Zylann]

Godot 3.0.6
Godot 3.1 alpha3

This is a very weird one found by Zylann/godot_heightmap_plugin#57
My terrain plugin uses PhysicsServer to build a Bullet HeightmapShape of size 2049x2049, and apparently moving the mouse over it causes a massive slowdown, to 1 fps or lower, even when the shape is set to not be pickable. This makes collision on terrains completely unusable.
It should also be noted that my plugin creates this collider also in the editor, but for some reason, it doesn't cause any slowdown there. Also, I tested collisions just fine in a 513x513 terrain this morning, but perhaps it was small enough to not be noticed?

• Something must be wrong with the way picking actually triggers (only in 3.0.6)
• Something must be making raycasts very slow with HeightmapShapes (and by extent, probably any collision detection)

I made a minimal project showing the issue:
HeightmapColliderSlowdown.zip
It just creates a completely flat HeightmapShape with the origin corner centered at the world origin, and an AABB spanning -1 to 1 in height (so it definitely encloses the heights).

It also reproduces in Godot 3.1 alpha3, but only if actually set to be pickable:
HeightmapColliderSlowdown31.zip

Play the main scene, and look at the console printing every frame.
Move the mouse above the area with a plane mesh: notice the massive slowdown.

Edit: just in case, I tested with power of two sizes: 2048 has no effect, and 1024 only halves the slowdown, but we are talking about 1 -> 2 fps here.

[Zylann]

From what I gather from debugging, I see that in order to achieve picking, Godot is casting a ray with a distance of 10000. But Bullet actually builds an AABB enclosing that ray, finds the heightfield, and ends up processing ALL triangles of it within this AABB, which a crazy amount for a raycast of this length. I really wonder why it works that way... even my stupid-naive raycast in GDScript can achieve better results than this by relying on nothing but the heights themselves. This is from my rough interpretation in one debug session though...

I assume turning pickable off may make the shape usable again, since objects usually colliding with the terrain don't have such a bigass bounding box, but there is surely room for improvement regarding raycasts.
The heightfield could be chunked into many smaller ones, but that means processing the entire image into sparse blocks sharing their max edge, which is inconvenient to use, JUST for the sake of raycasts.
Or the raycast could be chunked into small AABBs, which would more accurately represent the area to check for hits, but also inconvenient to use so would have to be done internally, at a stage where we know that we are raycasting a heightfield.
Or the heights could be wrapped in multiple AABBs enclosing chunks rather than directly checking the whole grid, but that requires baking them each time the grid changes.

However there is another weird fact I noticed during my test:
When I look down, the ray may be 10000 along the Y axis (in Godot coordinate system), but its AABB would actually only span a few units in X and Z, which is supposed to match only a few cells of the grid. And it still slows down the game tremendously... what the heck?
I mean, just looking at the variables here, it doesn't look right:

And when it finally gets to checking triangles of the grid:

[Zireael07]

Tagging the resident Bullet wizard @AndreaCatania

[akien-mga]

Reopening as #25543 was reverted.

[akien-mga]

Fixed again as #25543 is being reinstated by #28928.