Implement search for rich text fields and use it for notes #7953
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
This pull request implements search functionality for rich text fields, focusing on enhancing search capabilities for notes. Here are the key changes:
- Added support for searching rich text fields in the
is-searchable-field.util.tsfile - Implemented a new search vector field for notes in
note.workspace-entity.ts - Updated
CommandMenu.tsxto use the newuseSearchRecordshook for notes search - Modified
get-ts-vector-column-expression.util.tsto handle rich text fields using JSON path queries - Adjusted migration sorting logic in
workspace-migration.service.tsto prioritize TS_VECTOR field creation - Added a new 'searchVector' field ID in
standard-field-ids.tsfor notes
These changes collectively improve the search experience for notes, particularly those with rich text content.
7 file(s) reviewed, 10 comment(s)
Edit PR Review Bot Settings | Greptile
...twenty-server/src/engine/metadata-modules/workspace-migration/workspace-migration.service.ts
Outdated
Show resolved
Hide resolved
...twenty-server/src/engine/metadata-modules/workspace-migration/workspace-migration.service.ts
Outdated
Show resolved
Hide resolved
...twenty-server/src/engine/metadata-modules/workspace-migration/workspace-migration.service.ts
Outdated
Show resolved
Hide resolved
...twenty-server/src/engine/metadata-modules/workspace-migration/workspace-migration.service.ts
Outdated
Show resolved
Hide resolved
...workspace-manager/workspace-migration-builder/factories/workspace-migration-field.factory.ts
Outdated
Show resolved
Hide resolved
...gine/workspace-manager/workspace-sync-metadata/utils/get-ts-vector-column-expression.util.ts
Outdated
Show resolved
Hide resolved
packages/twenty-server/src/modules/note/standard-objects/note.workspace-entity.ts
Show resolved
Hide resolved
packages/twenty-server/src/modules/note/standard-objects/note.workspace-entity.ts
Show resolved
Hide resolved
Weiko
reviewed
Oct 23, 2024
| if (fieldMetadata.type === FieldMetadataType.RELATION) { | ||
| continue; | ||
| } | ||
| const fieldMetadataCollectionGroupByObjectMetadataId = |
There was a problem hiding this comment.
grouping field migrations by object so we can have a unique migration per object with all fields keeping order of insertion. This is helping because searchVector column needs to be created after the ones that are used in the expression so we need to have a reliable way to keep the correct order.
charlesBochet
pushed a commit
that referenced
this pull request
Nov 8, 2024
Search vector fields based on `RICH_TEXT` fields were generated using a treatment on `RICH_TEXT` fields's `body` column, to only extract and index the core text. ([PR](#7953)) Actually our RICH_TEXT fields are of datatype `text` in our database, allowing users to insert non-json values, which breaks the search vector generation (as it expects json values). Our vision is unclear for now: for instance we may want to turn RICH_TEXT into a composite field where the plain text would be stored in a different column. So for now, we will (1) treat rich_text data as text, and (2) update the search vector expressions for the existing workspaces.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.