Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: add new filter operator "$contains" #79

Merged
merged 2 commits into from
Feb 27, 2024

Conversation

LovaArutinovi
Copy link
Contributor

@LovaArutinovi LovaArutinovi commented Feb 23, 2024

What kind of change does this PR introduce?

Feature

What is the current behavior?

I need to be able to filter vectors by tag but I can't because there is no way to filter array metadata

What is the new behavior?

I added the new operator "$contains"
now we can have an array in metadata like ['sales', "marketing", "development"]

collection.query(
            data=[...],
            include_metadata=True,
            include_value=True,
            filters={"tags": {"$contains": "development"}})

@@ -885,6 +885,10 @@ def build_filters(json_col: Column, filters: Dict):
contains_value = [cast(elem, postgresql.JSONB) for elem in clause]
return json_col.op("->")(key).in_(contains_value)

if operator == "$contains":
contains_value = cast(clause, postgresql.JSONB)
return json_col.op("->")(key).contains(contains_value)
Copy link
Collaborator

@olirice olirice Feb 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self:
somehow this renders as

SELECT vecs.bar.id
FROM vecs.bar
WHERE (vecs.bar.metadata -> 'a') @> CAST('1' AS JSONB)
ORDER BY vecs.bar.vec <=> '[0.0, 0.0, 0.0, 0.0, 0.0]'
LIMIT 3

which is exactly correct, but not documented?

Copy link
Collaborator

@olirice olirice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice PR, thank you

I'm on the fence about what the right behavior for the contains operator on objects and scalars. Could you update the filter to enforce that the metadata field be an json_typeof 'array'? I think that will lead to the least confusion and address the issue that was initially raised.

We can potentially make it broader in the future but narrowing would be difficult to do safely.

src/tests/test_collection.py Outdated Show resolved Hide resolved
@olirice
Copy link
Collaborator

olirice commented Feb 23, 2024

update the filter to enforce that the metadata field be an json_typeof 'array'

if you have any trouble with that lmk and I'm happy to do it

@LovaArutinovi
Copy link
Contributor Author

update the filter to enforce that the metadata field be an json_typeof 'array'

if you have any trouble with that lmk and I'm happy to do it

I will be very grateful if you help🙏

@olirice olirice merged commit 8c2d87f into supabase:main Feb 27, 2024
10 checks passed
@jeanmaried
Copy link
Contributor

@olirice thank you for merging this! Would you be able to publish a new version soon? We had another PR for deleting by filter that is still unpublished. The last time vecs was published was mid december I believe.

@olirice
Copy link
Collaborator

olirice commented Feb 28, 2024

Sure, just published 0.4.3 with current master including this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants