Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pick instance feature #479

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

thatcomputerguy0101
Copy link
Contributor

This adds pick_instance and ray_intersect_instance functions to allow detecting which geometry instance a ray intersects with. There is a new example picking_instances that demonstrates this capability for basic geometry, instanced geometry, and imported models. The additional example has yet to be added to the examples README. This closes #467.

Additionally, the shader IDs have been separated out to simplify keeping track of which ID's were used, and a few ID overlaps were corrected. The shader IDs internally make use of open_enum types, which could be changed to also be the public type signature in a breaking release. The shader ID changes can be seen individually at https://github.com/thatcomputerguy0101/three-d/tree/shader_ID_refactor, and I can create a separate pull request for that or remove those changes if desired.

@asny asny mentioned this pull request Sep 17, 2024
@asny
Copy link
Owner

asny commented Sep 17, 2024

@thatcomputerguy0101 I made a separate PR for the shader ID refactor to make it easier to review and test that change separately. I'll update this PR after that is merged.

@thatcomputerguy0101
Copy link
Contributor Author

This PR has now been updated to be based on the current master branch.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super nice with an example, but maybe we can just extend the current picking example to also change the color of the selected mesh/instance.

let mut sphere_instances = Instances {
transformations: [
Mat4::from_translation(vec3(2.0, 0.0, 0.0)),
Mat4::from_translation(vec3(4.0, 0.0, 0.0)),
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice with a lot of instances.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am confused if this is a suggestion for a change or just a comment.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a suggestion, currently, there's only two instances, it's better to test with 100 instances for example.

///
/// Representation of a specific geometry instance
///
pub struct GeometryInstance {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I like the name (naming is hard). It's not a geometry instance, it's not even necessarily an picking result containing an instance. Maybe PickResult or something?

Copy link
Contributor Author

@thatcomputerguy0101 thatcomputerguy0101 Sep 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it used to fit the GeometryInstance name better, but has since been adjusted away from that. However, ray_intersect_instance also returns that type, so maybe IntersectionResult would be good?

/// and (viewport.x + viewport.width, viewport.y + viewport.height) indicate the top right corner.
/// Returns ```None``` if no geometry was hit between the near (`z_near`) and far (`z_far`) plane for this camera.
///
pub fn pick_instance(
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These functions are almost identical to pick and ray_cast, maybe just update those return more data. It's a breaking change, but an easy one to understand and fix. Also, it shouldn't affect performance, so even if you only want the picked position, the new functionality should still be good.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll probably change that with my next set of edits. (Technically it is a couple of instructions slower due to moving more data around, but that is probably a negligible difference especially once compiler optimizations are applied.)

src/renderer.rs Outdated
texture.as_color_target(None),
depth_texture.as_depth_target(),
)
.clear(ClearState::color_and_depth(-1.0, -1.0, -1.0, -1.0, 1.0))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The clear values are clamped between 0 and 1, so this doesn't make sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The platform that I was testing on doesn't seem to be clamping it, which was why I thought it was fine. I'll try and find an alternative solution to have the intended behavior on all platforms.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. That's weird, but yeah, you never know how OpenGL specs are actually implemented. I guess you can use 0 for no hit and just offset the ID result with -1.

#[derive(Default, Clone)]
pub struct GeometryInstanceMaterial {
/// Geometry ID
pub id: usize,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I generally try to use u32 or u16 to make it explicit. usize varies with operating system and is usually u64 which can't fit in a f32 anyway.

src/renderer/material/geometry_instance_material.rs Outdated Show resolved Hide resolved
src/renderer/material.rs Outdated Show resolved Hide resolved
src/renderer/geometry/shaders/mesh.vert Outdated Show resolved Hide resolved
@thatcomputerguy0101
Copy link
Contributor Author

I'm looking at using ints the whole way through the GPU side, although that requires using an integer output texture which restricts support to OpenGL 3.0 or greater and WebGL 2. Would these be acceptable versions to still replace the original pick, or should pick_instance remain separate to support older versions that only need pick?

@asny
Copy link
Owner

asny commented Sep 25, 2024

I'm looking at using ints the whole way through the GPU side, although that requires using an integer output texture which restricts support to OpenGL 3.0 or greater and WebGL 2. Would these be acceptable versions to still replace the original pick, or should pick_instance remain separate to support older versions that only need pick?

Yes, that would be ok. Would you write the depth to an integer then? You need to write depth (f32) and ID (u32) to a render target, so you have to choose either a float or integer target. I'm not sure it makes a big difference whether you choose one over the other.

@thatcomputerguy0101
Copy link
Contributor Author

Part of that change would be reading the depth from the depth buffer (supported in WebGL 2 if I interpreted it correctly), which is always a float data type.

@thatcomputerguy0101 thatcomputerguy0101 force-pushed the pick_instance_feature branch 2 times, most recently from 867cc4b to d5cf6c2 Compare September 27, 2024 05:31
@thatcomputerguy0101
Copy link
Contributor Author

Seems like I misinterpreted being able to read from the depth texture on web. I still have some ideas to work around that though.

@thatcomputerguy0101
Copy link
Contributor Author

GL's floatBitsToInt function ended up being useful. It is fully working on both MacOS and web with an integer output texture, and the floating point depth value encoded in one of the color channels. I'll probably create a separate PR for integer texture support, as I made some significant changes to get that working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pick geometry instance from coordinate
2 participants