Skip to content

Add custom projection matrix support for rendering #110850

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
huwpascoe wants to merge 3 commits into from
Closed

Add custom projection matrix support for rendering #110850

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cd402f7
is_frustum_symmetric
huwpascoe Sep 25, 2025
2bbb175
custom projection matrix
huwpascoe Sep 24, 2025
e23c009
asymmetric projection matrix
huwpascoe Sep 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions core/math/projection.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -876,6 +876,13 @@ bool Projection::is_orthogonal() const {
return columns[2][3] == 0.0;
}

bool Projection::is_frustum_symmetric() const {
return (columns[0][1] == 0.0f && columns[0][2] == 0.0f && columns[0][3] == 0.0f &&
Copy link
Copy Markdown
Contributor

@BastiaanOlij BastiaanOlij Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to find some time to look this up, but this check seems unlikely, this looks like it would be true only for identity matrices?

Copy link
Copy Markdown
Contributor Author

@huwpascoe huwpascoe Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what's meant by symmetry no?

[ * 0 0 0 ]
[ 0 * 0 0 ]
[ 0 0 * * ]
[ 0 0 * * ]

unslanted, unsheared, rectangular and unpanned.

Copy link
Copy Markdown
Contributor

@BastiaanOlij BastiaanOlij Sep 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to find some time to double check but laying it out like that, my gut feeling says yes. So I might have read the code wrong.

Copy link
Copy Markdown
Contributor

@BastiaanOlij BastiaanOlij Sep 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically it's when left FOV = right FOV, top FOV = bottom FOV, so indeed unslanted, unsheared, rectangular and unpanned.

columns[1][0] == 0.0f && columns[1][2] == 0.0f && columns[1][3] == 0.0f &&
columns[2][0] == 0.0f && columns[2][1] == 0.0f &&
columns[3][0] == 0.0f && columns[3][1] == 0.0f);
}

real_t Projection::get_fov() const {
// NOTE: This assumes a rectangular projection plane, i.e. that :
// - the matrix is a projection across z-axis (i.e. is invertible and columns[0][1], [0][3], [1][0] and [1][3] == 0)
Expand Down
1 change: 1 addition & 0 deletions core/math/projection.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ struct [[nodiscard]] Projection {
real_t get_aspect() const;
real_t get_fov() const;
bool is_orthogonal() const;
bool is_frustum_symmetric() const;

Vector<Plane> get_projection_planes(const Transform3D &p_transform) const;

Expand Down
1 change: 1 addition & 0 deletions core/variant/variant_call.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2442,6 +2442,7 @@ static void _register_variant_builtin_methods_misc() {
bind_method(Projection, get_aspect, sarray(), varray());
bind_method(Projection, get_fov, sarray(), varray());
bind_method(Projection, is_orthogonal, sarray(), varray());
bind_method(Projection, is_frustum_symmetric, sarray(), varray());

bind_method(Projection, get_viewport_half_extents, sarray(), varray());
bind_method(Projection, get_far_plane_half_extents, sarray(), varray());
Expand Down
6 changes: 6 additions & 0 deletions doc/classes/Projection.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,12 @@
Returns a [Projection] that performs the inverse of this [Projection]'s projective transformation.
</description>
</method>
<method name="is_frustum_symmetric" qualifiers="const">
<return type="bool" />
<description>
Returns [code]true[/code] if this [Projection] conforms to standard orthographic or perspective.
</description>
</method>
<method name="is_orthogonal" qualifiers="const">
<return type="bool" />
<description>
Expand Down
8 changes: 8 additions & 0 deletions doc/classes/RenderingServer.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,14 @@
Sets camera to use perspective projection. Objects on the screen becomes smaller when they are far away.
</description>
</method>
<method name="camera_set_projection">
<return type="void" />
<param index="0" name="camera" type="RID" />
<param index="1" name="projection" type="Projection" />
<description>
Sets a custom projection matrix.
</description>
</method>
<method name="camera_set_transform">
<return type="void" />
<param index="0" name="camera" type="RID" />
Expand Down
9 changes: 9 additions & 0 deletions drivers/gles3/rasterizer_scene_gles3.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -803,6 +803,11 @@ void RasterizerSceneGLES3::_draw_sky(RID p_env, const Projection &p_projection,
spec_constants |= SkyShaderGLES3::APPLY_TONEMAPPING;
}

const bool use_asymmetric_projection = (!p_use_multiview && !p_projection.is_frustum_symmetric());
if (use_asymmetric_projection) {
spec_constants |= SkyShaderGLES3::USE_ASYMMETRIC_PROJECTION;
}

RS::EnvironmentBG background = environment_get_background(p_env);

if (sky) {
Expand Down Expand Up @@ -873,6 +878,10 @@ void RasterizerSceneGLES3::_draw_sky(RID p_env, const Projection &p_projection,
material_storage->shaders.sky_shader.version_set_uniform(SkyShaderGLES3::FOG_SKY_AFFECT, environment_get_fog_sky_affect(p_env), shader_data->version, SkyShaderGLES3::MODE_BACKGROUND, spec_constants);
material_storage->shaders.sky_shader.version_set_uniform(SkyShaderGLES3::DIRECTIONAL_LIGHT_COUNT, sky_globals.directional_light_count, shader_data->version, SkyShaderGLES3::MODE_BACKGROUND, spec_constants);

if (use_asymmetric_projection) {
material_storage->shaders.sky_shader.version_set_uniform(SkyShaderGLES3::INV_PROJECTION_MATRIX, camera.inverse(), shader_data->version, SkyShaderGLES3::MODE_BACKGROUND, spec_constants);
}

if (p_use_multiview) {
glBindBufferBase(GL_UNIFORM_BUFFER, SKY_MULTIVIEW_UNIFORM_LOCATION, scene_state.multiview_buffer);
glBindBuffer(GL_UNIFORM_BUFFER, 0);
Expand Down
7 changes: 7 additions & 0 deletions drivers/gles3/shaders/sky.glsl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ mode_cubemap = #define USE_CUBEMAP_PASS
#[specializations]

USE_MULTIVIEW = false
USE_ASYMMETRIC_PROJECTION = false
USE_INVERTED_Y = true
APPLY_TONEMAPPING = true
USE_QUARTER_RES_PASS = false
Expand Down Expand Up @@ -120,6 +121,8 @@ layout(std140) uniform MultiviewData { // ubo:11
highp vec4 eye_offset[MAX_VIEWS];
}
multiview_data;
#elif defined(USE_ASYMMETRIC_PROJECTION)
uniform mat4 inv_projection_matrix;
#endif

layout(location = 0) out vec4 frag_color;
Expand Down Expand Up @@ -187,6 +190,10 @@ void main() {

// Unproject will give us the position between the eyes, need to re-offset.
cube_normal += multiview_data.eye_offset[ViewIndex].xyz;
#elif defined(USE_ASYMMETRIC_PROJECTION)
vec4 unproject = vec4(uv_interp.x, -uv_interp.y, -1.0, 1.0);
vec4 unprojected = inv_projection_matrix * unproject;
cube_normal = unprojected.xyz / unprojected.w;
#else
cube_normal.z = -1.0;
cube_normal.x = (uv_interp.x + projection.x) / projection.y;
Expand Down
2 changes: 2 additions & 0 deletions servers/rendering/renderer_rd/environment/sky.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1213,6 +1213,8 @@ void SkyRD::setup_sky(const RenderDataRD *p_render_data, const Size2i p_screen_s
// This is unused so just reset to identity.
Projection ident;
RendererRD::MaterialStorage::store_camera(ident, sky_scene_state.ubo.combined_reprojection[i]);

sky_scene_state.ubo.full_projection = !view_inv_projection.is_frustum_symmetric();
}

RendererRD::MaterialStorage::store_camera(view_inv_projection, sky_scene_state.ubo.view_inv_projections[i]);
Expand Down
2 changes: 1 addition & 1 deletion servers/rendering/renderer_rd/environment/sky.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ class SkyRD {

float z_far; // 4 - 340
uint32_t directional_light_count; // 4 - 344
uint32_t pad1; // 4 - 348
uint32_t full_projection; // 4 - 348
uint32_t pad2; // 4 - 352
};

Expand Down
14 changes: 10 additions & 4 deletions servers/rendering/renderer_rd/shaders/environment/sky.glsl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ layout(set = 0, binding = 2, std140) uniform SkySceneData {

float z_far; // 4 - 52
uint directional_light_count; // 4 - 56
uint pad1; // 4 - 60
bool full_projection; // 4 - 60
uint pad2; // 4 - 64
}
sky_scene_data;
Expand Down Expand Up @@ -209,9 +209,15 @@ void main() {
// Unproject will give us the position between the eyes, need to re-offset
cube_normal += sky_scene_data.view_eye_offsets[ViewIndex].xyz;
#else
cube_normal.z = -1.0;
cube_normal.x = (cube_normal.z * (-uv_interp.x - params.projection.x)) / params.projection.y;
cube_normal.y = -(cube_normal.z * (uv_interp.y - params.projection.z)) / params.projection.w;
if (sky_scene_data.full_projection) {
vec4 unproject = vec4(uv_interp, 0.0, 1.0);
vec4 unprojected = sky_scene_data.view_inv_projections[0] * unproject;
cube_normal = unprojected.xyz / unprojected.w;
} else {
cube_normal.z = -1.0;
cube_normal.x = (cube_normal.z * (-uv_interp.x - params.projection.x)) / params.projection.y;
cube_normal.y = -(cube_normal.z * (uv_interp.y - params.projection.z)) / params.projection.w;
}
#endif
cube_normal = mat3(params.orientation) * cube_normal;
cube_normal = normalize(cube_normal);
Expand Down
24 changes: 17 additions & 7 deletions servers/rendering/renderer_scene_cull.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,14 @@ void RendererSceneCull::camera_set_frustum(RID p_camera, float p_size, Vector2 p
camera->zfar = p_z_far;
}

void RendererSceneCull::camera_set_projection(RID p_camera, const Projection &p_projection) {
Camera *camera = camera_owner.get_or_null(p_camera);
ERR_FAIL_NULL(camera);

camera->type = Camera::CUSTOM;
camera->projection = p_projection;
}

void RendererSceneCull::camera_set_transform(RID p_camera, const Transform3D &p_transform) {
Camera *camera = camera_owner.get_or_null(p_camera);
ERR_FAIL_NULL(camera);
Expand Down Expand Up @@ -2618,15 +2626,12 @@ void RendererSceneCull::render_camera(const Ref<RenderSceneBuffers> &p_render_bu
// Setup Camera(s)
if (p_xr_interface.is_null()) {
// Normal camera
Transform3D transform = camera->transform;
Projection projection;
bool vaspect = camera->vaspect;
bool is_orthogonal = false;
bool is_frustum = false;

switch (camera->type) {
case Camera::ORTHOGONAL: {
projection.set_orthogonal(
camera->projection.set_orthogonal(
camera->size,
p_viewport_size.width / (float)p_viewport_size.height,
camera->znear,
Expand All @@ -2635,7 +2640,7 @@ void RendererSceneCull::render_camera(const Ref<RenderSceneBuffers> &p_render_bu
is_orthogonal = true;
} break;
case Camera::PERSPECTIVE: {
projection.set_perspective(
camera->projection.set_perspective(
camera->fov,
p_viewport_size.width / (float)p_viewport_size.height,
camera->znear,
Expand All @@ -2644,7 +2649,7 @@ void RendererSceneCull::render_camera(const Ref<RenderSceneBuffers> &p_render_bu

} break;
case Camera::FRUSTUM: {
projection.set_frustum(
camera->projection.set_frustum(
camera->size,
p_viewport_size.width / (float)p_viewport_size.height,
camera->offset,
Expand All @@ -2653,9 +2658,14 @@ void RendererSceneCull::render_camera(const Ref<RenderSceneBuffers> &p_render_bu
camera->vaspect);
is_frustum = true;
} break;
case Camera::CUSTOM: {
is_orthogonal = camera->projection.is_orthogonal();
} break;
}

camera_data.set_camera(transform, projection, is_orthogonal, is_frustum, vaspect, jitter, taa_frame_count, camera->visible_layers);
camera_data.set_camera(camera->transform, camera->projection,
is_orthogonal, is_frustum, camera->vaspect, jitter, taa_frame_count, camera->visible_layers);

#ifndef XR_DISABLED
} else {
XRServer *xr_server = XRServer::get_singleton();
Expand Down
5 changes: 4 additions & 1 deletion servers/rendering/renderer_scene_cull.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,8 @@ class RendererSceneCull : public RenderingMethod {
enum Type {
PERSPECTIVE,
ORTHOGONAL,
FRUSTUM
FRUSTUM,
CUSTOM,
};
Type type;
float fov;
Expand All @@ -87,6 +88,7 @@ class RendererSceneCull : public RenderingMethod {
RID attributes;
RID compositor;

Projection projection;
Transform3D transform;

Camera() {
Expand All @@ -109,6 +111,7 @@ class RendererSceneCull : public RenderingMethod {
virtual void camera_set_perspective(RID p_camera, float p_fovy_degrees, float p_z_near, float p_z_far);
virtual void camera_set_orthogonal(RID p_camera, float p_size, float p_z_near, float p_z_far);
virtual void camera_set_frustum(RID p_camera, float p_size, Vector2 p_offset, float p_z_near, float p_z_far);
virtual void camera_set_projection(RID p_camera, const Projection &p_projection);
virtual void camera_set_transform(RID p_camera, const Transform3D &p_transform);
virtual void camera_set_cull_mask(RID p_camera, uint32_t p_layers);
virtual void camera_set_environment(RID p_camera, RID p_env);
Expand Down
1 change: 1 addition & 0 deletions servers/rendering/rendering_method.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ class RenderingMethod {
virtual void camera_set_perspective(RID p_camera, float p_fovy_degrees, float p_z_near, float p_z_far) = 0;
virtual void camera_set_orthogonal(RID p_camera, float p_size, float p_z_near, float p_z_far) = 0;
virtual void camera_set_frustum(RID p_camera, float p_size, Vector2 p_offset, float p_z_near, float p_z_far) = 0;
virtual void camera_set_projection(RID p_camera, const Projection &p_projection) = 0;
virtual void camera_set_transform(RID p_camera, const Transform3D &p_transform) = 0;
virtual void camera_set_cull_mask(RID p_camera, uint32_t p_layers) = 0;
virtual void camera_set_environment(RID p_camera, RID p_env) = 0;
Expand Down
1 change: 1 addition & 0 deletions servers/rendering/rendering_server.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2811,6 +2811,7 @@ void RenderingServer::_bind_methods() {
ClassDB::bind_method(D_METHOD("camera_set_perspective", "camera", "fovy_degrees", "z_near", "z_far"), &RenderingServer::camera_set_perspective);
ClassDB::bind_method(D_METHOD("camera_set_orthogonal", "camera", "size", "z_near", "z_far"), &RenderingServer::camera_set_orthogonal);
ClassDB::bind_method(D_METHOD("camera_set_frustum", "camera", "size", "offset", "z_near", "z_far"), &RenderingServer::camera_set_frustum);
ClassDB::bind_method(D_METHOD("camera_set_projection", "camera", "projection"), &RenderingServer::camera_set_projection);
ClassDB::bind_method(D_METHOD("camera_set_transform", "camera", "transform"), &RenderingServer::camera_set_transform);
ClassDB::bind_method(D_METHOD("camera_set_cull_mask", "camera", "layers"), &RenderingServer::camera_set_cull_mask);
ClassDB::bind_method(D_METHOD("camera_set_environment", "camera", "env"), &RenderingServer::camera_set_environment);
Expand Down
1 change: 1 addition & 0 deletions servers/rendering/rendering_server.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -913,6 +913,7 @@ class RenderingServer : public Object {
virtual void camera_set_perspective(RID p_camera, float p_fovy_degrees, float p_z_near, float p_z_far) = 0;
virtual void camera_set_orthogonal(RID p_camera, float p_size, float p_z_near, float p_z_far) = 0;
virtual void camera_set_frustum(RID p_camera, float p_size, Vector2 p_offset, float p_z_near, float p_z_far) = 0;
virtual void camera_set_projection(RID p_camera, const Projection &p_projection) = 0;
virtual void camera_set_transform(RID p_camera, const Transform3D &p_transform) = 0;
virtual void camera_set_cull_mask(RID p_camera, uint32_t p_layers) = 0;
virtual void camera_set_environment(RID p_camera, RID p_env) = 0;
Expand Down
1 change: 1 addition & 0 deletions servers/rendering/rendering_server_default.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -672,6 +672,7 @@ class RenderingServerDefault : public RenderingServer {
FUNC4(camera_set_perspective, RID, float, float, float)
FUNC4(camera_set_orthogonal, RID, float, float, float)
FUNC5(camera_set_frustum, RID, float, Vector2, float, float)
FUNC2(camera_set_projection, RID, const Projection &)
FUNC2(camera_set_transform, RID, const Transform3D &)
FUNC2(camera_set_cull_mask, RID, uint32_t)
FUNC2(camera_set_environment, RID, RID)
Expand Down