Skip to content

Commit

Permalink
Implement omnidirectional cameras for real-time reflection probes.
Browse files Browse the repository at this point in the history
This commit introduces a new type of camera, the *omnidirectional*
camera. These cameras render to a cubemap texture, and as such extract
into six different cameras in the render world, one for each side. The
cubemap texture can be attached to a reflection probe as usual, which
allows reflections to contain moving objects. To use an omnidirectional
camera, create an [`OmnidirectionalCameraBundle`].

Because omnidirectional cameras extract to six different sub-cameras in
the render world, render world extraction code that targets components
present on cameras now needs to be aware of this fact and extract
components to the individual sub-cameras, not the root camera component.
They also need to run after omnidirectional camera extraction, as only
then will the sub-cameras be present in the render world. New plugins,
`ExtractCameraComponentPlugin` and `ExtractCameraInstancesPlugin`, are
available to assist with this.

Each side of an omnidirectional camera can be individually marked as
active via the `ActiveCubemapSides` bitfield. This allows for the common
technique of rendering only one (or two, or three) sides of the cubemap
per frame, to reduce rendering overhead. It also allows for on-demand
rendering, so that an application that wishes to optimize further can
choose sides to refresh. For example, an application might wish to only
rerender sides whose frusta contain moving entities.

In addition to real-time reflection probes, this patch introduces much
of the infrastructure necessary to support baking reflection probes from
within Bevy as opposed to in an external program such as Blender, which
has been the status quo up to this point. Even with this patch, there
are still missing pieces needed to make this truly convenient, however:

1. Baking a reflection probe requires more than just saving a cubemap:
   it requires pre-filtering the cubemap into diffuse and specular parts
   in the same way that the [glTF IBL Sampler] does. This is not yet
   implemented in Bevy; see bevyengine#9414 for a previous attempt.

2. The cubemap needs to be saved in `.ktx2` format, as that's the only
   format that Bevy presently knows how to load. There's no
   comprehensive Rust crate for this, though note that my [glTF IBL
   Sampler UI] has code to do it for the specific case of cubemaps.

3. An editor UI is necessary for convenience, as otherwise every
   application will have to create some sort of bespoke tool that
   arranges scenes and saves the reflection cubemaps.

The `reflection_probes` example has been updated in order to add an
option to enable dynamic reflection probes, as well as an option to spin
the cubes so that the impact of the dynamic reflection probes is
visible. Additionally, the static reflection probe, which was previously
rendered in Blender, has been changed to one rendered in Bevy. This
results in a change in appearance, as Blender and Bevy render somewhat
differently.

Partially addresses bevyengine#12233.

[glTF IBL Sampler]: https://github.com/KhronosGroup/glTF-IBL-Sampler

[glTF IBL Sampler UI]: https://github.com/pcwalton/gltf-ibl-sampler-egui
  • Loading branch information
pcwalton committed Jun 14, 2024
1 parent 2cffd14 commit f0d12c5
Show file tree
Hide file tree
Showing 19 changed files with 1,744 additions and 456 deletions.
Binary file not shown.
Binary file modified assets/models/cubes/Cubes.glb
Binary file not shown.
1 change: 1 addition & 0 deletions crates/bevy_core_pipeline/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ radsort = "0.1"
nonmax = "0.5"
smallvec = "1"
thiserror = "1.0"
arrayvec = "0.7"

[lints]
workspace = true
Expand Down
Loading

0 comments on commit f0d12c5

Please sign in to comment.