Feat/twenty orm

[magrinj]

Introduction

This PR introduces "TwentyORM," a custom ORM module designed to streamline database interactions within our workspace schema, reducing the need for raw SQL queries. The API mirrors TypeORM's to provide a familiar interface while integrating enhancements specific to our project's needs.

To facilitate this integration, new decorators prefixed with Workspace have been implemented. These decorators are used to define entity metadata more explicitly and are critical in constructing our schema dynamically.

New Features

• Custom ORM System: Named "TwentyORM," which aligns closely with TypeORM for ease of use but is tailored to our application's specific requirements.
• Decorator-Driven Configuration: Entities are now configured with Workspace-prefixed decorators that clearly define schema mappings and relationships directly within the entity classes.
• Injectable Repositories: Repositories can be injected similarly to TypeORM, allowing for flexible and straightforward data management.

Example Implementations

Decorated Entity Definitions

Entities are defined with new decorators that outline table and field metadata, relationships, and constraints. Here are examples of these implementations:

Company Metadata Object

@WorkspaceObject({
  standardId: STANDARD_OBJECT_IDS.company,
  namePlural: 'companies',
  labelSingular: 'Company',
  labelPlural: 'Companies',
  description: 'A company',
  icon: 'IconBuildingSkyscraper',
})
export class CompanyObjectMetadata extends BaseObjectMetadata {
  @WorkspaceField({
    standardId: COMPANY_STANDARD_FIELD_IDS.name,
    type: FieldMetadataType.TEXT,
    label: 'Name',
    description: 'The company name',
    icon: 'IconBuildingSkyscraper',
  })
  name: string;

  @WorkspaceField({
    standardId: COMPANY_STANDARD_FIELD_IDS.xLink,
    type: FieldMetadataType.LINK,
    label: 'X',
    description: 'The company Twitter/X account',
    icon: 'IconBrandX',
  })
  @WorkspaceIsNullable()
  xLink: LinkMetadata;

  @WorkspaceField({
    standardId: COMPANY_STANDARD_FIELD_IDS.position,
    type: FieldMetadataType.POSITION,
    label: 'Position',
    description: 'Company record position',
    icon: 'IconHierarchy2',
  })
  @WorkspaceIsSystem()
  @WorkspaceIsNullable()
  position: number;

  @WorkspaceRelation({
    standardId: COMPANY_STANDARD_FIELD_IDS.accountOwner,
    label: 'Account Owner',
    description: 'Your team member responsible for managing the company account',
    type: RelationMetadataType.MANY_TO_ONE,
    inverseSideTarget: () => WorkspaceMemberObjectMetadata,
    inverseSideFieldKey: 'accountOwnerForCompanies',
    onDelete: RelationOnDeleteAction.SET_NULL,
  })
  @WorkspaceIsNullable()
  accountOwner: WorkspaceMemberObjectMetadata;
}

Workspace Member Metadata Object

@WorkspaceObject({
  standardId: STANDARD_OBJECT_IDS.workspaceMember,
  namePlural: 'workspaceMembers',
  labelSingular: 'Workspace Member',
  labelPlural: 'Workspace Members',
  description: 'A workspace member',
  icon: 'IconUserCircle',
})
@WorkspaceIsSystem()
@WorkspaceIsNotAuditLogged()
export class WorkspaceMemberObjectMetadata extends BaseObjectMetadata {
  @WorkspaceField({
    standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.name,
    type: FieldMetadataType.FULL_NAME,
    label: 'Name',
    description: 'Workspace member name',
    icon: 'IconCircleUser',
  })
  name: FullNameMetadata;

  @WorkspaceRelation({
    standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.accountOwnerForCompanies,
    label: 'Account Owner For Companies',
    description: 'Account owner for companies',
    icon: 'IconBriefcase',
    type: RelationMetadataType.ONE_TO_MANY,
    inverseSideTarget: () => CompanyObjectMetadata,
    inverseSideFieldKey: 'accountOwner',
    onDelete: RelationOnDeleteAction.SET_NULL,
  })
  accountOwnerForCompanies: Relation

<CompanyObjectMetadata[]>;
}

Injectable Repository Usage

Repositories can be directly injected into services, allowing for streamlined query operations:

export class CompanyService {
  constructor(
    @InjectWorkspaceRepository(CompanyObjectMetadata)
    private readonly companyObjectMetadataRepository: WorkspaceRepository<CompanyObjectMetadata>,
  ) {}

  async companies(): Promise<CompanyObjectMetadata[]> {
    // Example queries demonstrating simple and relation-loaded operations
    const simpleCompanies = await this.companyObjectMetadataRepository.find({});
    const companiesWithOwners = await this.companyObjectMetadataRepository.find({
      relations: ['accountOwner'],
    });
    const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
      where: { xLinkLabel: 'MyLabel' },
    });

    return companiesFilteredByLinkLabel;
  }
}

Conclusions

This PR sets the foundation for a decorator-driven ORM layer that simplifies data interactions and supports complex entity relationships while maintaining clean and manageable code architecture. This is not finished yet, and should be extended.
All the standard objects needs to be migrated and all the module using the old decorators too.

[magrinj]

DataSource it automatically created based on each request, a storage is system is in place to avoid creating too many data sources.

[magrinj]

Loggin all for now, as this shouldn't be used in production yet

[Weiko]

nit: I feel like this might get complex over time if this can be used with either FieldMetadata or string which are quite different 🤔

[magrinj]

It's like that for now since some previous PR, because I'm not yet sure on which constructor I should choose

[Weiko]

I'd prefer pure functions if possible to avoid side effects and will be easier to debug, wdyt?

[magrinj]

Yes you're right, I've done that at first to create a quick class

[Weiko]

same here

[Weiko]

What's the impact on performance if we have many datasources with opened connections? Should we provide TTL and destroy the datasource after some time?

[magrinj]

Actually it's storing data source the same way it was done before, but we can of course optimize this with TTL if needed, but I'm not sure it's really needed

[Weiko]

Sure, let's discuss later if we need to improve that part 👍

[Weiko]

Are we keeping those names or is it temporary before removing ObjectMetadata decorator?

[magrinj]

It's a proposal of new name, as params will now differ over what really ObjectMetadata entities are

[Weiko]

Yes makes sense 👍

[Weiko]

Maybe we can add some tests + 1 test using 1 repository? I'm ok to merge it like this since it's still not fully rolled-out

[Weiko]

Should we call it ObjectMetadataClass?

[Weiko]

Maybe we can add some tests + 1 test using 1 repository? I'm ok to merge it like this since it's still not fully rolled-out

[Weiko]

typo

[Weiko]

LGTM, great job @magrinj! 👏