Implementation of new Spatial Camera type: Oblique

[qaptoR]

allows the specification of a world plane for near clipping;
applies matrix manipulation to generate oblique near plane.

A partial solution to this proposal: godotengine/godot-proposals#2713
**Note: It is by no means a complete solution and should not necessarily be taken as a reason to close the proposal. This PR is merely a straightforward solution for implementing the need for oblique clipping planes to implement a portal mechanic.

Not exactly what is requested, but this implementation covers essentially the most common case the exposed camera matrix would be used for according to the community. At least until more common uses of an exposed matrix are determined to push for a more broad solution.

Thus, this feature is necessary for implementing a 'portal' effect based on the most popular feature tutorials. For example, this often cited video by Sebastian Lague:
https://www.youtube.com/watch?v=cWpFZbjtSQg

Benefits of this solution...
Uses the function 'set_oblique_plane_from_transform' to convert the origin and forward vector into a camera-space Plane normal and distance-offset. Or alternatively, the position and normal can be provided independently of each other.

Exposes member variables in the editor for the plane normal, position, and offset data which supports editing of the plane in world space providing an easy way to visualize the clipping plane by making adjustments in the inspector.

Portal Effect using perspective camera...

Each portal has a camera which displays on the alternate portal's display. The perspective camera can't see through the obstacle its parent portal is placed on (its position is relative to the player's position and the other portal).

Portal Effect using Oblique camera...

The oblique camera uses the portal's global transform to adjust the camera's near clipping plane to be parallel with the portals facing direction. Thus, obstacles between the camera and portal are no longer visible.

Oblique Camera Member Variables...

Simple Video Demonstration

**Note: If it is not clear using some metrics I'm not familiar with, this is the master branch target for previously made 3.x PR #56664

• Bugsquad edit: This closes Add angled/oblique near and far camera planes godot-proposals#501.

[mrjustaguy]

the two images are the same...

[Zireael07]

Yeah, I think someone made a mistake when uploading pictures

[qaptoR]

I am correcting these errors. mostly just name changes that slipped past me

[fire]

There are some documentation errors.

ERROR: Camera3D.xml: Empty argument name in method "set_oblique_plane_from_transform" at position 0.

$ git diff
diff --git a/doc/classes/Camera3D.xml b/doc/classes/Camera3D.xml
index 0a6c658f0e..211c0972ff 100644
--- a/doc/classes/Camera3D.xml
+++ b/doc/classes/Camera3D.xml
@@ -133,7 +133,7 @@
                </method>
                <method name="set_oblique_plane_from_transform">
                        <return type="void" />
-                       <param index="0" name="_unnamed_arg0" type="Transform3D" />
+                       <param index="0" name="oblique_transform" type="Transform3D" />
                        <description>
                                Sets the Oblique_normal and oblique_position values of an oblique projection camera using a transform origin and forward vector.
                        </description>

[qaptoR]

Do you think it would be a ok enhancement to add a mirror boolean for mirrors?

If there is a reason this would be helpful then I am sure it would not be difficult.
However, this PR only facilitates the culling of all objects 'between the camera and the surface it is meant to look through'. Specifically it recreates the effect coined 'banana juice' by the team at valve who first developed the techniques used most often in portals today. This effects is more technically referred to as 'oblique near plane clipping'.

This is a talk given at Harvard by members of the valve team. This timestamp is where they describe 'banana juice'.
https://www.youtube.com/watch?v=ivyseNMVt-4&t=1064s

That is, based on my experience with creating portals, creating a mirror vs a portal would be dependent upon:

• the code written for how the camera tracks the player
• the use of a shader to cut from the camera's view just what needs to be seen on the mirror/portal surface (screen-space projection? I forget what this is called)
• the horizontal flip of the camera image (to my estimation.. I haven't actually tried making a mirror yet)

[fire]

I am having trouble making a mirror. The mirror shows on the editor, but not the played game.

Mirror Game Project.zip

Technical notes:

1. the mirroring effect is done by using a double sided texture and then having the plane mesh face backwards.

[fire]

I think it's something to do with the oblique normal defaulting to Vector3(0,0,0) which isn't a valid normal.

Edited:

Trying Vector3 oblique_normal = Vector3(0, 0, -1);

[fire]

Please remove commented codes.

[qaptoR]

✔
Thanks. Wasn't sure if that function call was necessary anymore in 4.0

[fire]

Initialize to Vector3(0, 0, -1) (forward) and don't initialize twice in the initializer and the constructor. ( Godot initializes in the class definition. )

[fire]

Swapping between camera projection perspective and oblique fixes the load state, and I wonder how it's related to the not playing oblique camera bug.

Hacky workaround on the camera node:

extends Camera3D


# Called when the node enters the scene tree for the first time.
func _ready():
	projection = PROJECTION_PERSPECTIVE
	projection = PROJECTION_OBLIQUE

[fire]

I extracted mirror nodes to a scene and made a smoother video.

Edited:

Any ideas on how to fix the force set projection bug?

2022-08-26.12-29-56_vsekai_mirrors.mp4

Can someone help me test performance? Use this branch of godot 4. #64876 and my demo https://github.com/fire/openxr-demo Made for #godotvr and V-Sekai

[qaptoR]

@fire
Thank-you for the work you're doing to help test this PR. It was originally written to work with 3.4, and did so without any hiccups (that I came across).

I've been playing with your XR demo, trying to get things working as intended, and there are a few notes I need to make:

• the mirror should only be 1 sided, otherwise there are certain positions the camera may be in where z-fighting becomes an issue and what should be the non-visible side becomes a visual anomaly. (or what I can only describe with my vocabulary as z-fighting)
  

• If the mirror is intended to move at all, a process function like in this image should be used to update the position (or could be done using signals for better performance, ie. 'on position changed')
  

I've had to use the two separate calls, rather than the 'from transform', as based on the rotation of the mesh, the forward normal is actually pointing downwards. Thus, it may be that the function needs to be reworked. It was meant to be the 'most general use-case' for applying the oblique data, as opposed to separate calls. Thus, the interface for this feature definitely needs work.

and lastly... I do need to update the docs for this feature as well. There are no entries for set_oblique_*, for position, offset, or normal, which should be visible in roughly this part of the reference:
EDIT: just got back from running an errand and I realized that because they are just setter/getter methods they don't require an entry, iirc

[aaronfranke]

@qaptoR Any update? This would be a really nice feature to have so that we can implement portals.

[ghost]

Hello,

I pulled master today and implemented the changes from this PR to test it out because I wanted to try to get this working for my project.
One issue I've noticed is that the camera seems to be cutting off the shadows from my DirectionalLight3d node.

Here's the symptom. I noticed that it appears this is present in the screenshots on the original post for this PR.
https://user-images.githubusercontent.com/114332440/192134833-d3f44b26-b9ed-47b5-be67-f9f733321873.mp4

Here's how it looks using perspective projection.
https://user-images.githubusercontent.com/114332440/192134913-d0311c62-41a9-4081-b4e5-8af2ed077850.mp4

Does anyone with a better understanding know what could be causing this?

[qaptoR]

@aaronfranke It has been a few weeks since I have looked at this project. I have adhd, so I find it difficult to focus on one project for very long. What are you asking for an update in reference to?

@tk3615175 I have some theories as to what is happening in your first video, but I cannot be sure. Would you be able to zip up your project and post it so that I may take a closer look?

[ghost]

@qaptoR here is my project if you (or anyone else) could have a look at to try and figure out this shadow issue, that would be appreciated.
I cloned from master at SHA 13f0158 yesterday.
Then merged your changes.
Then built, so that's the version of godot I was using.

PORTALS.zip

[qaptoR]

@tk3615175
Thank you for bringing this to my attention. I had forgotten others were using this camera.

The fix for shadows will soon be pushed. It Is in the process of being tested.

Please take note however that while this PR represents at least a working prototype (as in can be used effectively as is, minus any other unforseen renderer pipeline hiccups), while coming up with a solution for the shadows, it became apparent that instead of adding a fourth camera type to the mix, oblique near plane clipping could instead be implemented as a separate calculation for all camera types: perspective, orthogonal, frustum.

This is because, according to the paper which outlines the mathematics behind this oblique camera, the math works irrespective of what the prior projection was, so long as it is invertible.

Thus, the editor implementation may change slightly in the final release (eg. selecting a checkbox for 'use oblique plane', instead of the current choosing Oblique camera type), but the code used to interface with the camera should remain the same, since the data being passed in at present is essential.
This is because right now the oblique camera is only implemented for perspective projection, and would not work with frustum camera projects for instance.

[qaptoR]

The fix for shadows will soon be pushed. It Is in the process of being tested.

Actually. Something has drastically broken since the last time I worked with this PR.
I can't seem to get a quick solution.
It seems like something has changed in the render pipeline that I'm just not aware of, because nothing in my test project has changed. The engine is compiling fine too, but the oblique frustum is not being calculated properly, even though nothing has changed in the algorithm I've used.

My apologies, but this is going to have to be tabled on my todo list.
The last time I built the engine and opened my project everything was working:
oblique frustum, shadows, etc.
but now it's going haywire, and I'm not actively working on that project.
I just don't have the patience right now for the PR breaking every time I turn around to focus on something else.

[ghost]

I just don't have the patience right now for the PR breaking every time I turn around to focus on something else.

@qaptoR That's fair!

Is your untested shadow fix available somewhere? I'll see if I can make it play nice with master

[qaptoR]

I'm going to close this PR, work on a better implementation that can be applied more seamlessly to the camera (that will perhaps work with orthogonal cameras as well, based on the paper my current solution is sourced from). I already have an idea how to do this, but my local repo is too messy that I need to do housekeeping to get a clear mind about what I'm doing moving forward. This project was the first open source contribution I attempted to make and I'm very proud that I had a rough working solution at the time but it was still not a good enough solution IMO for final acceptance.

Moving forward, for my own sanity I will only be targeting the Master branch since at some point 4.0 and master will begin to diverge and I do not intend to keep track of multiple versions since that was one of the major downfalls of this current solution, I felt overwhelmed by having to jump back and forth between master and 3.x.