bevy_reflect: Better proxies #6971
Conversation
MrGVSV
added
C-Usability
|
@soqb can I get your review here? |
Keep in mind I still need to run some benchmarks 😅 |
|
im still not convinced this is better than returning the proxied this would avoid polluting the trait space with too many reflect-adjacent traits (limited to frankly, imo, i like the idea of changing the signature to return i definitely won't block approving this PR on the "dynamic |
Adding
|
|
other options i can see for
|
I don't think 2 is feasible (again, due to the static-ness of And 3 actually sounds pretty reasonable. We don't have Edit: Actually just looked at the code. Because we don't have I’d like to go with 3 and remove |
is there a good use case for this? |
For example, preventing Dynamics from being serialized and deserialized as mentioned in this comment (serialization is actually preventable if we add |
|
im confused as to what you mean. struct MyStruct {
dynamic_struct: DynamicStruct,
}why would you want to prevent (de)serialization? {
"my_crate::MyStruct": (
dynamic_struct: (
foo: 123,
),
),
}seems perfectly deserializable. |
In that example, what type does |
|
good point. i wonder if we should choose a canonical representation for each of serde's primitives e.g. all numbers are for now i think adding an |
Yeah that could be an option. It might be a little confusing or too implicit and would need proper documentation. I'm also not sure how it would work for non-primitive types. But I think that should be explored in a separate ticket.
Since the Dynamic types aren't themselves registered, I'm not sure how we'd store/access data like Maybe we just make deserialization error out if there's no registration? If not that, we either need to add a new mechanism to store |
if they aren't register how are they deserialized? surely they go through
this is not possible using |
|
we currently cannot directly serialize a #[derive(Reflect)]
struct ContainsDyn(DynamicStruct);
let mut registry = TypeRegistry::new();
registry.register::<ContainsDyn>();
let registration = registry.get(TypeId::of::<ContainsDyn>()).unwrap();
let contains_dyn = ContainsDyn(DynamicStruct::default());
let reflect_ser = ReflectSerializer::new(&contains_dyn, ®istry);
let ron = ron::ser::to_string(&reflect_ser); // Err(Message("no registration found for dynamic type with name ")) |
| self.represented_type | ||
| .map(|info| info.type_name()) | ||
| .unwrap_or_default() |
There was a problem hiding this comment.
should this be something like:
| self.represented_type | |
| .map(|info| info.type_name()) | |
| .unwrap_or_default() | |
| self.represented_type | |
| .map(|info| info.type_name()) | |
| .unwrap_or_else(|| std::any::type_name::<Self>()) |
to aid with debugging and error messages? (likewise for other dynamics).
There was a problem hiding this comment.
oops! i see that i accidentally approved somehow. ignore that 😅.
There was a problem hiding this comment.
I'm not sure I love this solution, but I do agree it may be better than simply returning an empty string. I'll make the change!
Yeah, as I mentioned that code in the deserializer isn't doing anything because we don't (and can't) register those types. And with that being the one place we need to statically access whether a type is dynamic or not, we should be okay to remove |
|
Here's another question: what does a Dynamic type's Should I don't think doing so will have any major consequences apart from needing to unwrap or guard against the |
|
I would absolutely use an Option here. |
|
three options i can think of for opaques (for scripting):
|
|
can you provide an example of a partially initialised dynamic - this sounds like an antipattern so maybe it should be removed if Bevy has this capability. |
A good example is actually the reflection example. Besides being proxies, the Dynamics were also intended to be used to apply "patches" to other values (including other Dynamics). We also partially construct them when deserializing reflected data, only finalizing them after all data is deserialized. And the deserialized data itself may be missing fields, especially if they're marked with I don't think this is behavior we want to remove. If we want to create separate |
|
i wouldn't call those partially constructed - hence my confusion but i wouldn't say they had a defined represented type either; they are "partially representative". unless we want a new |
Sorry, my terminology was a bit confusing 😅
Yeah, I'd like to avoid doing the whole |
|
i'm not sure how a dynamic can not have a |
Again, this is in the case where a user does something like: let mut dyn_struct = DynamicStruct::default();
dyn_struct.insert("foo", 123_u32);
let type_info = dyn_struct.represented_type_info();
assert_eq!(???, type_info.type_id());
Hm, that's a possibility. The next question then would be: how do we handle the required data in But I can see now why we would need |
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
# Objective > This PR is based on discussion from #6601 The Dynamic types (e.g. `DynamicStruct`, `DynamicList`, etc.) act as both: 1. Dynamic containers which may hold any arbitrary data 2. Proxy types which may represent any other type Currently, the only way we can represent the proxy-ness of a Dynamic is by giving it a name. ```rust // This is just a dynamic container let mut data = DynamicStruct::default(); // This is a "proxy" data.set_name(std::any::type_name::<Foo>()); ``` This type name is the only way we check that the given Dynamic is a proxy of some other type. When we need to "assert the type" of a `dyn Reflect`, we call `Reflect::type_name` on it. However, because we're only using a string to denote the type, we run into a few gotchas and limitations. For example, hashing a Dynamic proxy may work differently than the type it proxies: ```rust #[derive(Reflect, Hash)] #[reflect(Hash)] struct Foo(i32); let concrete = Foo(123); let dynamic = concrete.clone_dynamic(); let concrete_hash = concrete.reflect_hash(); let dynamic_hash = dynamic.reflect_hash(); // The hashes are not equal because `concrete` uses its own `Hash` impl // while `dynamic` uses a reflection-based hashing algorithm assert_ne!(concrete_hash, dynamic_hash); ``` Because the Dynamic proxy only knows about the name of the type, it's unaware of any other information about it. This means it also differs on `Reflect::reflect_partial_eq`, and may include ignored or skipped fields in places the concrete type wouldn't. ## Solution Rather than having Dynamics pass along just the type name of proxied types, we can instead have them pass around the `TypeInfo`. Now all Dynamic types contain an `Option<&'static TypeInfo>` rather than a `String`: ```diff pub struct DynamicTupleStruct { - type_name: String, + represented_type: Option<&'static TypeInfo>, fields: Vec<Box<dyn Reflect>>, } ``` By changing `Reflect::get_type_info` to `Reflect::represented_type_info`, hopefully we make this behavior a little clearer. And to account for `None` values on these dynamic types, `Reflect::represented_type_info` now returns `Option<&'static TypeInfo>`. ```rust let mut data = DynamicTupleStruct::default(); // Not proxying any specific type assert!(dyn_tuple_struct.represented_type_info().is_none()); let type_info = <Foo as Typed>::type_info(); dyn_tuple_struct.set_represented_type(Some(type_info)); // Alternatively: // let dyn_tuple_struct = foo.clone_dynamic(); // Now we're proxying `Foo` assert!(dyn_tuple_struct.represented_type_info().is_some()); ``` This means that we can have full access to all the static type information for the proxied type. Future work would include transitioning more static type information (trait impls, attributes, etc.) over to the `TypeInfo` so it can actually be utilized by Dynamic proxies. ### Alternatives & Rationale > **Note** > These alternatives were written when this PR was first made using a `Proxy` trait. This trait has since been removed. <details> <summary>View</summary> #### Alternative: The `Proxy<T>` Approach I had considered adding something like a `Proxy<T>` type where `T` would be the Dynamic and would contain the proxied type information. This was nice in that it allows us to explicitly determine whether something is a proxy or not at a type level. `Proxy<DynamicStruct>` proxies a struct. Makes sense. The reason I didn't go with this approach is because (1) tuples, (2) complexity, and (3) `PartialReflect`. The `DynamicTuple` struct allows us to represent tuples at runtime. It also allows us to do something you normally can't with tuples: add new fields. Because of this, adding a field immediately invalidates the proxy (e.g. our info for `(i32, i32)` doesn't apply to `(i32, i32, NewField)`). By going with this PR's approach, we can just remove the type info on `DynamicTuple` when that happens. However, with the `Proxy<T>` approach, it becomes difficult to represent this behavior— we'd have to completely control how we access data for `T` for each `T`. Secondly, it introduces some added complexities (aside from the manual impls for each `T`). Does `Proxy<T>` impl `Reflect`? Likely yes, if we want to represent it as `dyn Reflect`. What `TypeInfo` do we give it? How would we forward reflection methods to the inner type (remember, we don't have specialization)? How do we separate this from Dynamic types? And finally, how do all this in a way that's both logical and intuitive for users? Lastly, introducing a `Proxy` trait rather than a `Proxy<T>` struct is actually more inline with the [Unique Reflect RFC](bevyengine/rfcs#56). In a way, the `Proxy` trait is really one part of the `PartialReflect` trait introduced in that RFC (it's technically not in that RFC but it fits well with it), where the `PartialReflect` serves as a way for proxies to work _like_ concrete types without having full access to everything a concrete `Reflect` type can do. This would help bridge the gap between the current state of the crate and the implementation of that RFC. All that said, this is still a viable solution. If the community believes this is the better path forward, then we can do that instead. These were just my reasons for not initially going with it in this PR. #### Alternative: The Type Registry Approach The `Proxy` trait is great and all, but how does it solve the original problem? Well, it doesn't— yet! The goal would be to start moving information from the derive macro and its attributes to the generated `TypeInfo` since these are known statically and shouldn't change. For example, adding `ignored: bool` to `[Un]NamedField` or a list of impls. However, there is another way of storing this information. This is, of course, one of the uses of the `TypeRegistry`. If we're worried about Dynamic proxies not aligning with their concrete counterparts, we could move more type information to the registry and require its usage. For example, we could replace `Reflect::reflect_hash(&self)` with `Reflect::reflect_hash(&self, registry: &TypeRegistry)`. That's not the _worst_ thing in the world, but it is an ergonomics loss. Additionally, other attributes may have their own requirements, further restricting what's possible without the registry. The `Reflect::apply` method will require the registry as well now. Why? Well because the `map_apply` function used for the `Reflect::apply` impls on `Map` types depends on `Map::insert_boxed`, which (at least for `DynamicMap`) requires `Reflect::reflect_hash`. The same would apply when adding support for reflection-based diffing, which will require `Reflect::reflect_partial_eq`. Again, this is a totally viable alternative. I just chose not to go with it for the reasons above. If we want to go with it, then we can close this PR and we can pursue this alternative instead. #### Downsides Just to highlight a quick potential downside (likely needs more investigation): retrieving the `TypeInfo` requires acquiring a lock on the `GenericTypeInfoCell` used by the `Typed` impls for generic types (non-generic types use a `OnceBox which should be faster). I am not sure how much of a performance hit that is and will need to run some benchmarks to compare against. </details> ### Open Questions 1. Should we use `Cow<'static, TypeInfo>` instead? I think that might be easier for modding? Perhaps, in that case, we need to update `Typed::type_info` and friends as well? 2. Are the alternatives better than the approach this PR takes? Are there other alternatives? --- ## Changelog ### Changed - `Reflect::get_type_info` has been renamed to `Reflect::represented_type_info` - This method now returns `Option<&'static TypeInfo>` rather than just `&'static TypeInfo` ### Added - Added `Reflect::is_dynamic` method to indicate when a type is dynamic - Added a `set_represented_type` method on all dynamic types ### Removed - Removed `TypeInfo::Dynamic` (use `Reflect::is_dynamic` instead) - Removed `Typed` impls for all dynamic types ## Migration Guide - The Dynamic types no longer take a string type name. Instead, they require a static reference to `TypeInfo`: ```rust #[derive(Reflect)] struct MyTupleStruct(f32, f32); let mut dyn_tuple_struct = DynamicTupleStruct::default(); dyn_tuple_struct.insert(1.23_f32); dyn_tuple_struct.insert(3.21_f32); // BEFORE: let type_name = std::any::type_name::<MyTupleStruct>(); dyn_tuple_struct.set_name(type_name); // AFTER: let type_info = <MyTupleStruct as Typed>::type_info(); dyn_tuple_struct.set_represented_type(Some(type_info)); ``` - `Reflect::get_type_info` has been renamed to `Reflect::represented_type_info` and now also returns an `Option<&'static TypeInfo>` (instead of just `&'static TypeInfo`): ```rust // BEFORE: let info: &'static TypeInfo = value.get_type_info(); // AFTER: let info: &'static TypeInfo = value.represented_type_info().unwrap(); ``` - `TypeInfo::Dynamic` and `DynamicInfo` has been removed. Use `Reflect::is_dynamic` instead: ```rust // BEFORE: if matches!(value.get_type_info(), TypeInfo::Dynamic) { // ... } // AFTER: if value.is_dynamic() { // ... } ``` --------- Co-authored-by: radiish <[email protected]>
|
@MrGVSV can you take a look at why this is failing to compile in CI and rebase if needed? |
Co-authored-by: radiish <[email protected]>
MrGVSV
force-pushed
the
reflect-better-proxies
branch
from
99f0128
to
69106c3
Compare
|
@alice-i-cecile should be rebased and ready to go now! |
Selene-Amanita
added
the
M-Needs-Migration-Guide
# Objective
Right now, `TypeInfo` can be accessed directly from a type using either
`Typed::type_info` or `Reflect::get_represented_type_info`.
However, once that `TypeInfo` is accessed, any nested types must be
accessed via the `TypeRegistry`.
```rust
#[derive(Reflect)]
struct Foo {
bar: usize
}
let registry = TypeRegistry::default();
let TypeInfo::Struct(type_info) = Foo::type_info() else {
panic!("expected struct info");
};
let field = type_info.field("bar").unwrap();
let field_info = registry.get_type_info(field.type_id()).unwrap();
assert!(field_info.is::<usize>());;
```
## Solution
Enable nested types within a `TypeInfo` to be retrieved directly.
```rust
#[derive(Reflect)]
struct Foo {
bar: usize
}
let TypeInfo::Struct(type_info) = Foo::type_info() else {
panic!("expected struct info");
};
let field = type_info.field("bar").unwrap();
let field_info = field.type_info().unwrap();
assert!(field_info.is::<usize>());;
```
The particular implementation was chosen for two reasons.
Firstly, we can't just store `TypeInfo` inside another `TypeInfo`
directly. This is because some types are recursive and would result in a
deadlock when trying to create the `TypeInfo` (i.e. it has to create the
`TypeInfo` before it can use it, but it also needs the `TypeInfo` before
it can create it). Therefore, we must instead store the function so it
can be retrieved lazily.
I had considered also using a `OnceLock` or something to lazily cache
the info, but I figured we can look into optimizations later. The API
should remain the same with or without the `OnceLock`.
Secondly, a new wrapper trait had to be introduced: `MaybeTyped`. Like
`RegisterForReflection`, this trait is `#[doc(hidden)]` and only exists
so that we can properly handle dynamic type fields without requiring
them to implement `Typed`. We don't want dynamic types to implement
`Typed` due to the fact that it would make the return type
`Option<&'static TypeInfo>` for all types even though only the dynamic
types ever need to return `None` (see #6971 for details).
Users should never have to interact with this trait as it has a blanket
impl for all `Typed` types. And `Typed` is automatically implemented
when deriving `Reflect` (as it is required).
The one downside is we do need to return `Option<&'static TypeInfo>`
from all these new methods so that we can handle the dynamic cases. If
we didn't have to, we'd be able to get rid of the `Option` entirely. But
I think that's an okay tradeoff for this one part of the API, and keeps
the other APIs intact.
## Testing
This PR contains tests to verify everything works as expected. You can
test locally by running:
```
cargo test --package bevy_reflect
```
---
## Changelog
### Public Changes
- Added `ArrayInfo::item_info` method
- Added `NamedField::type_info` method
- Added `UnnamedField::type_info` method
- Added `ListInfo::item_info` method
- Added `MapInfo::key_info` method
- Added `MapInfo::value_info` method
- All active fields now have a `Typed` bound (remember that this is
automatically satisfied for all types that derive `Reflect`)
### Internal Changes
- Added `MaybeTyped` trait
## Migration Guide
All active fields for reflected types (including lists, maps, tuples,
etc.), must implement `Typed`. For the majority of users this won't have
any visible impact.
However, users implementing `Reflect` manually may need to update their
types to implement `Typed` if they weren't already.
Additionally, custom dynamic types will need to implement the new hidden
`MaybeTyped` trait.
Objective
The Dynamic types (e.g.
DynamicStruct,DynamicList, etc.) act as both:Currently, the only way we can represent the proxy-ness of a Dynamic is by giving it a name.
This type name is the only way we check that the given Dynamic is a proxy of some other type. When we need to "assert the type" of a
dyn Reflect, we callReflect::type_nameon it. However, because we're only using a string to denote the type, we run into a few gotchas and limitations.For example, hashing a Dynamic proxy may work differently than the type it proxies:
Because the Dynamic proxy only knows about the name of the type, it's unaware of any other information about it. This means it also differs on
Reflect::reflect_partial_eq, and may include ignored or skipped fields in places the concrete type wouldn't.Solution
Rather than having Dynamics pass along just the type name of proxied types, we can instead have them pass around the
TypeInfo.Now all Dynamic types contain an
Option<&'static TypeInfo>rather than aString:pub struct DynamicTupleStruct { - type_name: String, + represented_type: Option<&'static TypeInfo>, fields: Vec<Box<dyn Reflect>>, }By changing
Reflect::get_type_infotoReflect::represented_type_info, hopefully we make this behavior a little clearer. And to account forNonevalues on these dynamic types,Reflect::represented_type_infonow returnsOption<&'static TypeInfo>.This means that we can have full access to all the static type information for the proxied type. Future work would include transitioning more static type information (trait impls, attributes, etc.) over to the
TypeInfoso it can actually be utilized by Dynamic proxies.Alternatives & Rationale
View
Alternative: The
Proxy<T>ApproachI had considered adding something like a
Proxy<T>type whereTwould be the Dynamic and would contain the proxied type information.This was nice in that it allows us to explicitly determine whether something is a proxy or not at a type level.
Proxy<DynamicStruct>proxies a struct. Makes sense.The reason I didn't go with this approach is because (1) tuples, (2) complexity, and (3)
PartialReflect.The
DynamicTuplestruct allows us to represent tuples at runtime. It also allows us to do something you normally can't with tuples: add new fields. Because of this, adding a field immediately invalidates the proxy (e.g. our info for(i32, i32)doesn't apply to(i32, i32, NewField)). By going with this PR's approach, we can just remove the type info onDynamicTuplewhen that happens. However, with theProxy<T>approach, it becomes difficult to represent this behavior— we'd have to completely control how we access data forTfor eachT.Secondly, it introduces some added complexities (aside from the manual impls for each
T). DoesProxy<T>implReflect? Likely yes, if we want to represent it asdyn Reflect. WhatTypeInfodo we give it? How would we forward reflection methods to the inner type (remember, we don't have specialization)? How do we separate this from Dynamic types? And finally, how do all this in a way that's both logical and intuitive for users?Lastly, introducing a
Proxytrait rather than aProxy<T>struct is actually more inline with the Unique Reflect RFC. In a way, theProxytrait is really one part of thePartialReflecttrait introduced in that RFC (it's technically not in that RFC but it fits well with it), where thePartialReflectserves as a way for proxies to work like concrete types without having full access to everything a concreteReflecttype can do. This would help bridge the gap between the current state of the crate and the implementation of that RFC.All that said, this is still a viable solution. If the community believes this is the better path forward, then we can do that instead. These were just my reasons for not initially going with it in this PR.
Alternative: The Type Registry Approach
The
Proxytrait is great and all, but how does it solve the original problem? Well, it doesn't— yet!The goal would be to start moving information from the derive macro and its attributes to the generated
TypeInfosince these are known statically and shouldn't change. For example, addingignored: boolto[Un]NamedFieldor a list of impls.However, there is another way of storing this information. This is, of course, one of the uses of the
TypeRegistry. If we're worried about Dynamic proxies not aligning with their concrete counterparts, we could move more type information to the registry and require its usage.For example, we could replace
Reflect::reflect_hash(&self)withReflect::reflect_hash(&self, registry: &TypeRegistry).That's not the worst thing in the world, but it is an ergonomics loss.
Additionally, other attributes may have their own requirements, further restricting what's possible without the registry. The
Reflect::applymethod will require the registry as well now. Why? Well because themap_applyfunction used for theReflect::applyimpls onMaptypes depends onMap::insert_boxed, which (at least forDynamicMap) requiresReflect::reflect_hash. The same would apply when adding support for reflection-based diffing, which will requireReflect::reflect_partial_eq.Again, this is a totally viable alternative. I just chose not to go with it for the reasons above. If we want to go with it, then we can close this PR and we can pursue this alternative instead.
Downsides
Just to highlight a quick potential downside (likely needs more investigation): retrieving the
TypeInforequires acquiring a lock on theGenericTypeInfoCellused by theTypedimpls for generic types (non-generic types use a `OnceBox which should be faster). I am not sure how much of a performance hit that is and will need to run some benchmarks to compare against.Open Questions
Cow<'static, TypeInfo>instead? I think that might be easier for modding? Perhaps, in that case, we need to updateTyped::type_infoand friends as well?Changelog
Changed
Reflect::get_type_infohas been renamed toReflect::represented_type_infoOption<&'static TypeInfo>rather than just&'static TypeInfoAdded
Reflect::is_dynamicmethod to indicate when a type is dynamicset_represented_typemethod on all dynamic typesRemoved
TypeInfo::Dynamic(useReflect::is_dynamicinstead)Typedimpls for all dynamic typesMigration Guide
The Dynamic types no longer take a string type name. Instead, they require a static reference to
TypeInfo:Reflect::get_type_infohas been renamed toReflect::represented_type_infoand now also returns anOption<&'static TypeInfo>(instead of just&'static TypeInfo):TypeInfo::DynamicandDynamicInfohas been removed. UseReflect::is_dynamicinstead: