Selectable should compare field values, not call getters for initialized collections
#11160
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mpdude
changed the title
Selectable should not call getters for initialized collectionsSelectable should compare field values, not call getters for initialized collections
greg0ire
reviewed
Jan 13, 2024
Member
That would be the case if any of the getter/isser has side effects. I don't know if we have documentation where we forbid that. |
Member
You can do a breaking change only on 3.x, and you have until Feb 1st to do that. Even after Feb 1st, it will still be possible (and encouraged) to add deprecation layers to 2.x. |
Member
If feels like a |
mpdude
force-pushed
the
gh3591-still-issue
branch
2 times, most recently
from
a28cf80 to
c37e686
Compare
Contributor
Author
|
Let's ignore for the time being that I have no idea what the deprecation strategy/BC might look like and how to fix this as a cross-library issue... In order to deal with private fields, you need to know the class in which the field is declared. For the ORM, field names are currently unique, so for a given field name, we can figure that out... as long as we have access to the ClassMetadata or the ClassMetadataFactory. We probably can get hold of that in But, what about entity classes that have just been created, where users assigned |
mpdude
force-pushed
the
gh3591-still-issue
branch
from
c37e686 to
cbde02f
Compare
Contributor
|
There hasn't been any activity on this pull request in the past 90 days, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 7 days. |
Contributor
|
This pull request was closed due to inactivity. |
3 tasks
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 8, 2025
#### Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine#149, especially see this [comment](doctrine#149 (comment)) * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. #### Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). #### Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata.
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 8, 2025
# Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine#149, especially see this [comment](doctrine#149 (comment)) * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. # Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). # Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata. # Please enter the commit message for your changes. Lines starting # with '#' will be kept; you may remove them yourself if you want to. # An empty message aborts the commit. # # Date: Wed Oct 8 16:51:55 2025 +0200 # # On branch access-fields-directly # Your branch and 'origin/access-fields-directly' have diverged, # and have 3 and 9 different commits each, respectively. # # Changes to be committed: # modified: docs/en/index.rst # modified: src/ArrayCollection.php # modified: src/Criteria.php # modified: src/Expr/ClosureExpressionVisitor.php # modified: tests/ArrayCollectionTestCase.php # modified: tests/ClosureExpressionVisitorTest.php # modified: tests/CollectionTest.php # modified: tests/CollectionTestCase.php # modified: tests/CriteriaTest.php # new file: tests/TestObjectPrivatePropertyOnly.php # new file: tests/TestObjectPropertyHook.php # # Untracked files: # phpunit.xml #
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 8, 2025
# Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine#149, especially see this [comment](doctrine#149 (comment)) * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. # Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). # Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata. # Please enter the commit message for your changes. Lines starting # with '#' will be kept; you may remove them yourself if you want to. # An empty message aborts the commit. # # Date: Wed Oct 8 16:51:55 2025 +0200 # # On branch access-fields-directly # Your branch and 'origin/access-fields-directly' have diverged, # and have 3 and 9 different commits each, respectively. # # Changes to be committed: # modified: docs/en/index.rst # modified: src/ArrayCollection.php # modified: src/Criteria.php # modified: src/Expr/ClosureExpressionVisitor.php # modified: tests/ArrayCollectionTestCase.php # modified: tests/ClosureExpressionVisitorTest.php # modified: tests/CollectionTest.php # modified: tests/CollectionTestCase.php # modified: tests/CriteriaTest.php # new file: tests/TestObjectPrivatePropertyOnly.php # new file: tests/TestObjectPropertyHook.php # # Untracked files: # phpunit.xml #
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 8, 2025
# Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine#149, especially see this [comment](doctrine#149 (comment)) * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. # Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). # Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata.
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 9, 2025
# Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine#149, especially see this [comment](doctrine#149 (comment)) * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. # Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). # Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata.
Contributor
Author
|
I think doctrine/collections#472 can fix this. |
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 22, 2025
# Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine#149, especially see this [comment](doctrine#149 (comment)) * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. # Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). # Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata.
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 23, 2025
# Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. # Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This is also what @Ocramius already suggested in [this comment](doctrine#149 (comment)). For reference, here is a list of discussions around which style of accessors, getters, issers, public access etc. should be used or not used – in the future, the answer would be "only direct state (raw property value) matters". * doctrine#276 * doctrine#263 * doctrine#149 * doctrine#135 * doctrine#134 * doctrine#95 * doctrine#62 # Migration path This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). # Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata.
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 24, 2025
# Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. # Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This is also what @Ocramius already suggested in [this comment](doctrine#149 (comment)). For reference, here is a list of discussions around which style of accessors, getters, issers, public access etc. should be used or not used – in the future, the answer would be "only direct state (raw property value) matters". * doctrine#276 * doctrine#263 * doctrine#149 * doctrine#135 * doctrine#134 * doctrine#95 * doctrine#62 # Migration path This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). # Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata.
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 24, 2025
# Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. # Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This is also what @Ocramius already suggested in [this comment](doctrine#149 (comment)). For reference, here is a list of discussions around which style of accessors, getters, issers, public access etc. should be used or not used – in the future, the answer would be "only direct state (raw property value) matters". * doctrine#276 * doctrine#263 * doctrine#149 * doctrine#135 * doctrine#134 * doctrine#95 * doctrine#62 # Migration path This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). # Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata.
mpdude
added a commit
to mpdude/doctrine-collections
that referenced
this pull request
Oct 24, 2025
# Motivation There are a few issues about differences in behaviour when using the collection filtering API (the `Selectable` interface) against collections that are database-backed (in ORM, these are `PersistentCollections`) vs. memory-based collections using `ArrayCollection` from this package here. For example: * doctrine/orm#11160 * doctrine#170 * doctrine/orm#3591 * Maybe doctrine/orm#11021 Database-based matching can work on the raw field values only, as those values are persisted to the database and there is no PHP code involved when filtering at the database level. Memory-based matching currently tries a [series of access methods on the objects](https://www.doctrine-project.org/projects/doctrine-collections/en/2.3/index.html#selectable-methods:~:text=For%20collections%20that%20contain%20objects). The effects of this may be surprising. For example, with the ORM, it may be fine to filter entities based on the value of `private` or `protected` fields that have no getters. This works as long as a persistent collection is uninitialized. But as soon as it gets initialized, the `ArrayCollection` will require a getter method to be available. Another (more rare) example is a getter method that does some type of type conversion, like having a `string` field with values like `'y'|'n'` internally but returning a `bool` value from the getter; or, more generally, every type mismatch between the return value of the getter and the field value. Yet another example may be getters that cause side effects 🙈. # Proposed solution I discussed with @beberlei at the Doctrine hackathon that the primary use case for this library here was supporting the ORM/ODM use cases. This can be seen in places as `ClosureExpressionVisitor::getObjectFieldValue()` that take a `$field` parameter. So, although this library here has nothing to do with ORM/ODM mapping, I want to add a migration path here that moves the `doctrine/collection` behaviour closer to the implementation realities of ORM/ODM. This means to ultimately use direct (reflection-based) field access only. This is also what @Ocramius already suggested in [this comment](doctrine#149 (comment)). For reference, here is a list of discussions around which style of accessors, getters, issers, public access etc. should be used or not used – in the future, the answer would be "only direct state (raw property value) matters". * doctrine#276 * doctrine#263 * doctrine#149 * doctrine#135 * doctrine#134 * doctrine#95 * doctrine#62 # Migration path This feature is opt-in and will be activated by passing `accessRawFieldValues: true` to the `Criteria` constructor. The `Criteria` object is what is typically constructed by users in preparation for calling the `Selectable` API, so it seems to be a good fit. By opting in through this flag, memory-based comparisons and sorting will use direct field access only. Not activating the feature triggers a deprecation notice. In the next major version, direct field access will be the only (default) behaviour. The `$accessRawFieldValues` can be removed in the next major version (or, possibly, go through another round of deprecations in case when it is still passed, before being eventually removed). # Remaining edge case Given an inheritance hierarchy of classes where a multiple classes feature a `private` field of the same name, the downmost field will be picked. This may differ from Doctrine ORM behaviour when this field is not mapped at all in the ORM and another field (higher up the class hierarchy) is used as the mapped field instead. We cannot solve this without having access to ORM/ODM mapping metadata at hand, which is not possible from within an `ArrayCollection` that is typically created as [a newable type](https://testing.googleblog.com/2008/10/to-new-or-not-to-new.html). We rather plan to discourage or even prevent this kind of setup (entity class hierarchy with different classes having fields of the same name) at some point when loading and validating metadata.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This revisits #3591 which I think is not fixed correctly.
When using the
SelectableAPI to filter elements from a collection, the result depends on the initialization state of the collection.When the collection is uninitialized, the filtering happens in the database at the SQL level. The given field name is translated to a database column name and a database column-based lookup (
WHERE) is performed.When the collection is initialized, we go through hoops to derive potential getter/isser names and/or access
$object->$fieldas a last resort, but completely fail on private properties without getter method access.To the user, it should not make a difference whether a collection is initialized. We should use plain, direct field access (through reflection?) since that is what most closely resembles the database-based lookup. I think this is what Marco refers to in doctrine/collections#149 (comment) as "rely only on state for lookups".
I am not aware of any other places where the ORM would be concerned with deriving accessor names, which makes this an additional inconsistency.
Obviously, preferring code (getter based) access is not an option for uninitialized collections.
Note that even for plain "state" lookups, there is a remaining chance for issues when DBAL types come into play that do database-to-PHP value conversion. A SQL level column-value comparison might not come to the same result as a PHP-level comparison for certain DBAL type implementations. Anyways.
Suggestions needed
SelectableAPI lives in doctrine/collections and uses static method calls? Is this an ORM issue or a collections issue?Related: