Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unique indexes and indexes for composite types #7162

Merged
merged 43 commits into from
Oct 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
05c3ff1
Add unique indexes
FelixMalfait Sep 19, 2024
b668d45
Linter
FelixMalfait Sep 19, 2024
3d5e149
Improve frontend error messages
FelixMalfait Sep 19, 2024
4e79c01
Merge branch 'main' into add-unique-indexes
FelixMalfait Sep 27, 2024
5d265fb
Begin migration command
FelixMalfait Oct 1, 2024
3255104
Merge main and attempt to fix huge conflicts
FelixMalfait Oct 6, 2024
3fdffce
Add additional constraints
FelixMalfait Oct 6, 2024
3b7d6cd
Error handling
FelixMalfait Oct 7, 2024
5cfa045
Merge branch 'main' into add-unique-indexes
FelixMalfait Oct 7, 2024
d125250
Remove dead code
FelixMalfait Oct 7, 2024
33bfe5c
Partial index for empty string + rename workspacindexcolumn
FelixMalfait Oct 7, 2024
fede32a
Dont overload cache
FelixMalfait Oct 7, 2024
e03888e
Fixes
FelixMalfait Oct 7, 2024
43727a3
Drop composite column
FelixMalfait Oct 7, 2024
58522fb
Display indexes on the frontend
FelixMalfait Oct 8, 2024
42d0dcd
Improve settings display, dont hide ts vector
FelixMalfait Oct 8, 2024
a740bc3
Style update
FelixMalfait Oct 8, 2024
2b7e2b3
Merge main
FelixMalfait Oct 8, 2024
5605c00
Improve display
FelixMalfait Oct 9, 2024
4d4a368
Merge main
FelixMalfait Oct 9, 2024
fb2c6a4
begin test fix
FelixMalfait Oct 9, 2024
b4f1b5a
Lint
FelixMalfait Oct 9, 2024
6c147f9
Fix some tests
FelixMalfait Oct 10, 2024
8d33bbb
IsUnique decorator and property on fields
FelixMalfait Oct 10, 2024
6573253
Merge main
FelixMalfait Oct 10, 2024
f80414a
Progress on WorkspaceUnique
FelixMalfait Oct 10, 2024
8f4471e
Enable workflow by default in local env
FelixMalfait Oct 10, 2024
5540edb
Fix a test
FelixMalfait Oct 10, 2024
ef7ecfc
Disable create index
FelixMalfait Oct 10, 2024
d84812f
Fix
FelixMalfait Oct 10, 2024
1d3a5a7
Fix a test unrelated to this PR changes
FelixMalfait Oct 11, 2024
794e0d6
Not possible to remove ts_vector from sort/filter
FelixMalfait Oct 11, 2024
6920a70
Remove console log
FelixMalfait Oct 11, 2024
9beeaf6
Merge branch 'main' into add-unique-indexes
FelixMalfait Oct 11, 2024
3410bf9
Fix a few tests
FelixMalfait Oct 11, 2024
3f54928
Improve command
FelixMalfait Oct 11, 2024
886a7e9
Add feature flag
FelixMalfait Oct 11, 2024
75ce755
Fix lessThan operator not applying column case
charlesBochet Oct 12, 2024
a7ba38e
Feature flag
FelixMalfait Oct 13, 2024
d537979
Merge branch 'main' into add-unique-indexes
charlesBochet Oct 13, 2024
cbb1eeb
Setup 0.32 column folder
charlesBochet Oct 13, 2024
77b719c
Add tests to pass coverage
charlesBochet Oct 13, 2024
249da24
Fix tests
charlesBochet Oct 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/twenty-front/src/generated-metadata/gql.ts

Large diffs are not rendered by default.

130 changes: 128 additions & 2 deletions packages/twenty-front/src/generated-metadata/graphql.ts

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ describe('useCommandMenu', () => {
'ab7901eb-43e1-4dc7-8f3b-cdee2857eb9a',
imageIdentifierFieldMetadataId: null,
fields: [],
indexMetadatas: [],
},
]);
});
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import React, { useCallback, useEffect } from 'react';
import { useCallback, useEffect } from 'react';

import { ObjectMetadataItemNotFoundError } from '@/object-metadata/errors/ObjectMetadataNotFoundError';
import { SnackBarVariant } from '@/ui/feedback/snack-bar-manager/components/SnackBar';
Expand All @@ -20,7 +20,7 @@ export const PromiseRejectionEffect = () => {
},
);
} else {
enqueueSnackBar(`Error: ${event.reason}`, {
enqueueSnackBar(`${error.message}`, {
variant: SnackBarVariant.Error,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

red color and message usually already convey the fact that it's an error, and there's limited space

});
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,38 +47,36 @@ export const initialFavorites = [
},
];

export const sortedFavorites = [
{
"avatarType": "rounded",
"avatarUrl": "",
"id": "1",
"labelIdentifier": " ",
"link": "/object/person/1",
"position": 0,
"recordId": "1",
"workspaceMemberId": undefined,
},
{
"avatarType": "rounded",
"avatarUrl": "",
"id": "2",
"labelIdentifier": " ",
"link": "/object/person/3",
"position": 1,
"recordId": "3",
"workspaceMemberId": undefined,
},
{
"avatarType": "squared",
"avatarUrl": "example.com",
"id": "3",
"key": "8f3b2121-f194-4ba4-9fbf-2d5a37126806",
"labelIdentifier": "favoriteLabel",
"link": "example.com",
"position": 2,
"recordId": "1",
},
]
export const sortedFavorites = [
{
id: '1',
recordId: '2',
position: 0,
avatarType: 'squared',
avatarUrl: undefined,
labelIdentifier: 'ABC Corp',
link: '/object/company/2',
},
{
id: '2',
recordId: '4',
position: 1,
avatarType: 'squared',
avatarUrl: undefined,
labelIdentifier: 'Company Test',
link: '/object/company/4',
},
{
id: '3',
position: 2,
key: '8f3b2121-f194-4ba4-9fbf-2d5a37126806',
labelIdentifier: 'favoriteLabel',
avatarUrl: 'example.com',
avatarType: 'squared',
link: 'example.com',
recordId: '1',
},
];

export const mocks = [
{
Expand Down Expand Up @@ -343,8 +341,8 @@ export const mocks = [
mutation DeleteOneFavorite($idToDelete: ID!) {
deleteFavorite(id: $idToDelete) {
__typename
deletedAt
id
deletedAt
}
}
`,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { findAvailableTimeZoneOption } from '@/localization/utils/findAvailableTimeZoneOption';

describe('findAvailableTimeZoneOption', () => {
it('should find the matching available IANA time zone select option from a given IANA time zone', () => {
const ianaTimeZone = 'Europe/Paris';
const expectedOption = {
label: '(GMT+02:00) Central European Summer Time - Paris',
value: 'Europe/Paris',
};

const option = findAvailableTimeZoneOption(ianaTimeZone);

expect(option).toEqual(expectedOption);
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,11 @@ import { currentUserState } from '@/auth/states/currentUserState';
import { currentWorkspaceState } from '@/auth/states/currentWorkspaceState';
import { useFindManyObjectMetadataItems } from '@/object-metadata/hooks/useFindManyObjectMetadataItems';
import { objectMetadataItemsState } from '@/object-metadata/states/objectMetadataItemsState';
import { ObjectMetadataItem } from '@/object-metadata/types/ObjectMetadataItem';
import { FieldMetadataType } from '~/generated-metadata/graphql';
import { WorkspaceActivationStatus } from '~/generated/graphql';
import { generatedMockObjectMetadataItems } from '~/testing/mock-data/generatedMockObjectMetadataItems';
import { isDeeplyEqual } from '~/utils/isDeeplyEqual';
import { isUndefinedOrNull } from '~/utils/isUndefinedOrNull';

const filterTsVectorFields = (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to keep that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see it seems you handled it differently 👍

objectMetadataItems: ObjectMetadataItem[],
): ObjectMetadataItem[] => {
return objectMetadataItems.map((item) => ({
...item,
fields: item.fields.filter(
(field) => field.type !== FieldMetadataType.TsVector,
),
}));
};

export const ObjectMetadataItemsLoadEffect = () => {
const currentUser = useRecoilValue(currentUserState);
const currentWorkspace = useRecoilValue(currentWorkspaceState);
Expand All @@ -37,13 +24,12 @@ export const ObjectMetadataItemsLoadEffect = () => {
const updateObjectMetadataItems = useRecoilCallback(
({ set, snapshot }) =>
() => {
const filteredFields = filterTsVectorFields(newObjectMetadataItems);
const toSetObjectMetadataItems =
isUndefinedOrNull(currentUser) ||
currentWorkspace?.activationStatus !==
WorkspaceActivationStatus.Active
? generatedMockObjectMetadataItems
: filteredFields;
: newObjectMetadataItems;

if (
!isDeeplyEqual(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,30 @@ export const FIND_MANY_OBJECT_METADATA_ITEMS = gql`
updatedAt
labelIdentifierFieldMetadataId
imageIdentifierFieldMetadataId
indexMetadatas(paging: { first: 100 }) {
edges {
node {
id
createdAt
updatedAt
name
indexWhereClause
indexType
isUnique
indexFieldMetadatas(paging: { first: 100 }) {
edges {
node {
id
createdAt
updatedAt
order
fieldMetadataId
}
}
}
}
}
}
fields(paging: { first: 1000 }, filter: $fieldFilter) {
edges {
node {
Expand All @@ -37,6 +61,7 @@ export const FIND_MANY_OBJECT_METADATA_ITEMS = gql`
isActive
isSystem
isNullable
isUnique
createdAt
updatedAt
defaultValue
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { useMemo } from 'react';
import { useQuery } from '@apollo/client';
import { useMemo } from 'react';

Comment on lines 1 to +2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Import order changed. Consider using a consistent import order throughout the codebase.

import { SnackBarVariant } from '@/ui/feedback/snack-bar-manager/components/SnackBar';
import { useSnackBar } from '@/ui/feedback/snack-bar-manager/hooks/useSnackBar';
Expand Down Expand Up @@ -41,12 +41,9 @@ export const useFindManyObjectMetadataItems = ({
skip: skip || !apolloMetadataClient,
onError: (error) => {
logError('useFindManyObjectMetadataItems error : ' + error);
enqueueSnackBar(
`Error during useFindManyObjectMetadataItems, ${error.message}`,
{
variant: SnackBarVariant.Error,
},
);
enqueueSnackBar(`${error.message}`, {
variant: SnackBarVariant.Error,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was too dev-oriented

});
},
});

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { IndexField as GeneratedIndexField } from '~/generated-metadata/graphql';

export type IndexFieldMetadataItem = Omit<GeneratedIndexField, '__typename'> & {
__typename?: string;
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import { IndexFieldMetadataItem } from '@/object-metadata/types/IndexFieldMetadataItem';
import { Index as GeneratedIndex } from '~/generated-metadata/graphql';

export type IndexMetadataItem = Omit<
GeneratedIndex,
'__typename' | 'indexFieldMetadatas' | 'objectMetadata'
> & {
__typename?: string;
indexFieldMetadatas: IndexFieldMetadataItem[];
};
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
import { Object as GeneratedObject } from '~/generated-metadata/graphql';

import { IndexMetadataItem } from '@/object-metadata/types/IndexMetadataItem';
import { FieldMetadataItem } from './FieldMetadataItem';

export type ObjectMetadataItem = Omit<
GeneratedObject,
'__typename' | 'fields' | 'dataSourceId'
'__typename' | 'fields' | 'dataSourceId' | 'indexMetadatas'
> & {
__typename?: string;
fields: FieldMetadataItem[];
indexMetadatas: IndexMetadataItem[];
};
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ export const mapPaginatedObjectMetadataItemsToObjectMetadataItems = ({
pagedObjectMetadataItems?.objects.edges.map((object) => ({
...object.node,
fields: object.node.fields.edges.map((field) => field.node),
indexMetadatas: object.node.indexMetadatas.edges.map((index) => ({
...index.node,
indexFieldMetadatas: index.node.indexFieldMetadatas?.edges.map(
(indexField) => indexField.node,
),
})),
})) ?? [];

return formattedObjects;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ export const fieldMetadataItemSchema = (existingLabels?: string[]) => {
isActive: z.boolean(),
isCustom: z.boolean(),
isNullable: z.boolean(),
isUnique: z.boolean(),
isSystem: z.boolean(),
label: metadataLabelSchema(existingLabels),
name: camelCaseStringSchema,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { z } from 'zod';

import { IndexFieldMetadataItem } from '@/object-metadata/types/IndexFieldMetadataItem';

export const indexFieldMetadataItemSchema = z.object({
__typename: z.literal('indexField'),
fieldMetadataId: z.string().uuid(),
id: z.string(),
createdAt: z.string(),
updatedAt: z.string(),
order: z.number(),
}) satisfies z.ZodType<IndexFieldMetadataItem>;
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import { z } from 'zod';

import { IndexMetadataItem } from '@/object-metadata/types/IndexMetadataItem';
import { indexFieldMetadataItemSchema } from '@/object-metadata/validation-schemas/indexFieldMetadataItemSchema';
import { IndexType } from '~/generated-metadata/graphql';

export const indexMetadataItemSchema = z.object({
__typename: z.literal('index'),
id: z.string().uuid(),
name: z.string(),
indexFieldMetadatas: z.array(indexFieldMetadataItemSchema),
createdAt: z.string(),
updatedAt: z.string(),
indexType: z.nativeEnum(IndexType),
indexWhereClause: z.string().nullable(),
isUnique: z.boolean(),
objectMetadata: z.any(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: objectMetadata: z.any() is too permissive. Define a more specific schema

}) satisfies z.ZodType<IndexMetadataItem>;
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { z } from 'zod';

import { ObjectMetadataItem } from '@/object-metadata/types/ObjectMetadataItem';
import { fieldMetadataItemSchema } from '@/object-metadata/validation-schemas/fieldMetadataItemSchema';
import { indexMetadataItemSchema } from '@/object-metadata/validation-schemas/indexMetadataItemSchema';
import { metadataLabelSchema } from '@/object-metadata/validation-schemas/metadataLabelSchema';
import { camelCaseStringSchema } from '~/utils/validation-schemas/camelCaseStringSchema';

Expand All @@ -11,6 +12,7 @@ export const objectMetadataItemSchema = z.object({
dataSourceId: z.string().uuid(),
description: z.string().trim().nullable().optional(),
fields: z.array(fieldMetadataItemSchema()),
indexMetadatas: z.array(indexMetadataItemSchema),
icon: z.string().startsWith('Icon').trim(),
id: z.string().uuid(),
imageIdentifierFieldMetadataId: z.string().uuid().nullable(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { ObjectRecord } from '@/object-record/types/ObjectRecord';
import { getDeleteManyRecordsMutationResponseField } from '@/object-record/utils/getDeleteManyRecordsMutationResponseField';
import { useRecoilValue } from 'recoil';
import { isDefined } from '~/utils/isDefined';
import { isUndefinedOrNull } from '~/utils/isUndefinedOrNull';
import { sleep } from '~/utils/sleep';
import { capitalize } from '~/utils/string/capitalize';

Expand Down Expand Up @@ -132,7 +133,7 @@ export const useDeleteManyRecords = ({
})
.catch((error: Error) => {
cachedRecords.forEach((cachedRecord) => {
if (!cachedRecord) {
if (isUndefinedOrNull(cachedRecord?.id)) {
return;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,9 @@ export const useFindDuplicateRecords = <T extends ObjectRecord = ObjectRecord>({
`useFindDuplicateRecords for "${objectMetadataItem.nameSingular}" error : ` +
error,
);
enqueueSnackBar(
`Error during useFindDuplicateRecords for "${objectMetadataItem.nameSingular}", ${error.message}`,
{
variant: SnackBarVariant.Error,
},
);
enqueueSnackBar(`Error finding duplicates:", ${error.message}`, {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was too dev-oriented

variant: SnackBarVariant.Error,
});
Comment on lines +56 to +58
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Error message simplified, but consider including the object name for context

},
},
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,9 @@ export const useHandleFindManyRecordsError = ({
`useFindManyRecords for "${objectMetadataItem.namePlural}" error : ` +
error,
);
enqueueSnackBar(
`Error during useFindManyRecords for "${objectMetadataItem.namePlural}", ${error.message}`,
{
variant: SnackBarVariant.Error,
},
);
enqueueSnackBar(`${error.message}`, {
variant: SnackBarVariant.Error,
});
Comment on lines +22 to +24
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider adding a generic prefix to error messages for consistency

handleError?.(error);
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import { useUpdateOneRecordMutation } from '@/object-record/hooks/useUpdateOneRe
import { ObjectRecord } from '@/object-record/types/ObjectRecord';
import { getUpdateOneRecordMutationResponseField } from '@/object-record/utils/getUpdateOneRecordMutationResponseField';
import { sanitizeRecordInput } from '@/object-record/utils/sanitizeRecordInput';
import { isUndefinedOrNull } from '~/utils/isUndefinedOrNull';
import { capitalize } from '~/utils/string/capitalize';

type useUpdateOneRecordProps = {
Expand Down Expand Up @@ -130,7 +131,7 @@ export const useUpdateOneRecord = <
},
})
.catch((error: Error) => {
if (!cachedRecord) {
if (isUndefinedOrNull(cachedRecord?.id)) {
throw error;
}
updateRecordFromCache({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ const sortDefinition: SortDefinition = {
const objectMetadataItem: ObjectMetadataItem = {
id: 'object1',
fields: [],
indexMetadatas: [],
createdAt: '2021-01-01',
updatedAt: '2021-01-01',
nameSingular: 'object1',
Expand Down
Loading
Loading