Data explorer #6159

ad-elias · 2024-07-08T10:43:02Z

Building report and chart creation.

greptile-apps

PR Summary

Added new routes for Reports, Charts, and ChartEditor in /packages/twenty-front/src/App.tsx
Introduced AnalyticsQueryEditor, AnalyticsQueryFilter, and AnalyticsQueryFilters components in /packages/twenty-front/src/modules/activities/reports/components/
Added Chart, ChartConfig, PageAddReportButton, ReportGroup, ReportGroups, ReportRow, and ReportsLayout components in /packages/twenty-front/src/modules/activities/reports/components/
Defined new types and constants for reports and charts in /packages/twenty-front/src/modules/activities/reports/types/ and /packages/twenty-front/src/modules/activities/reports/constants/ReportGroupTimeSpans.ts
Introduced IS_REPORTS_ENABLED feature flag and updated related files in /packages/twenty-server/src/engine/core-modules/feature-flag/feature-flag.entity.ts and /packages/twenty-front/src/modules/workspace/types/FeatureFlagKey.ts

_{30 file(s) reviewed, 16 comment(s)}
_{Edit PR Review Bot Settings}

packages/twenty-front/src/modules/activities/reports/components/AnalyticsQueryEditor.tsx

packages/twenty-front/src/modules/activities/reports/components/AnalyticsQueryFilter.tsx

packages/twenty-front/src/modules/activities/reports/components/AnalyticsQueryFilters.tsx

packages/twenty-front/src/modules/activities/reports/components/Chart.tsx

packages/twenty-front/src/modules/activities/reports/constants/ReportGroupTimeSpans.ts

packages/twenty-front/src/modules/activities/reports/types/AnalyticsQuery.ts

packages/twenty-front/src/modules/activities/reports/types/Report.ts

packages/twenty-front/src/modules/navigation/hooks/useIsReportsPage.ts

greptile-apps

PR Summary

(updates since last review)

Removed a console.log statement used for debugging in /packages/twenty-server/src/engine/workspace-manager/workspace-sync-metadata/services/workspace-sync-relation-metadata.service.ts

No major changes found since last review.

_{1 file(s) reviewed, no comment(s)}
_{Edit PR Review Bot Settings}

FelixMalfait

Great work. Good that you opened the PR because I think there might be some wasted work, but it's the right direction!

packages/twenty-server/src/modules/reports/standard-objects/report.workspace-entity.ts

packages/twenty-server/src/modules/reports/standard-objects/chart.workspace-entity.ts

packages/twenty-server/src/modules/reports/standard-objects/analytics-query.workspace-entity.ts

FelixMalfait · 2024-07-08T11:43:09Z

packages/twenty-server/src/modules/reports/standard-objects/analytics-query.workspace-entity.ts

+    description: 'Singular name of the source object',
+    icon: 'IconAbc',
+  })
+  sourceObjectNameSingular: string;


It's better to point to the objectMetadataId instead, but you don't even need this (cf comment below, fieldMetadataId should be enough to retrieve everything)

FelixMalfait · 2024-07-08T11:44:26Z

packages/twenty-server/src/modules/reports/standard-objects/analytics-query.workspace-entity.ts

+    description: 'Dot-separated path to the source field.',
+    icon: 'IconForms',
+  })
+  fieldPath: string;


use fieldMetadataId instead and retrieve the truth from the source in metadata

I think something like fieldPath is required to be able to do joins - do we want to do joins? Do you have another idea how to do joins?

FelixMalfait · 2024-07-08T11:48:56Z

packages/twenty-front/src/pages/reports/Charts.tsx

+    objectNameSingular: CoreObjectNameSingular.Chart,
+  });
+
+  return (


We don't want to create a page dedicated to Charts ideally, instead we would want to try leveraging the generic /charts view that is generated from Charts being a standard object. Otherwise we're creating more tech debt. It might require a few adjustments (e.g. hijack + button?) but it should be doable

(I know we didn't it for task but we actually plan to migrate tasks whenever we can and we regret creating a dedicated page for it)

packages/twenty-front/src/pages/reports/Reports.tsx

packages/twenty-front/src/modules/activities/reports/types/AnalyticsQuery.ts

...wenty-server/src/modules/reports/standard-objects/analytics-query-filter.workspace-entity.ts

FelixMalfait · 2024-07-08T11:53:21Z

...wenty-server/src/modules/reports/standard-objects/analytics-query-filter.workspace-entity.ts

+import { WorkspaceJoinColumn } from 'src/engine/twenty-orm/decorators/workspace-join-column.decorator';
+
+@WorkspaceEntity({
+  standardId: STANDARD_OBJECT_IDS.analyticsQueryFilter,


I wonder how much of this is the same as viewFilter and should be 100% shared? let's keep them as close as possible?

I think this also depends on whether we want to be able to create JOIN queries or not. If not, I think we can use view-filter, otherwise a separate entity is required.

packages/twenty-server/src/modules/reports/standard-objects/analytics-query.workspace-entity.ts

FelixMalfait · 2024-07-08T12:02:07Z

packages/twenty-front/src/pages/reports/Charts.tsx

+export const Charts = () => {
+  const reportId = useParams().reportId ?? '';
+
+  const navigate = useNavigate();


FYI even if this page goes away, let's try to avoid using useNavigate() when we can and use instead which are better for accessibility and other reasons

What should we use instead? import { Link } from 'react-router-dom'; ?

"The link on the bottom left should be seen on hover"
Also, open in new tab should work!

greptile-apps

PR Summary

(updates since last review)

Added AnalyticsQueryModule to packages/twenty-server/src/engine/core-modules/core-engine.module.ts
Introduced AnalyticsQueryService in packages/twenty-server/src/engine/core-modules/analytics-query/analytics-query.service.ts
Created AnalyticsQueryResolver in packages/twenty-server/src/engine/core-modules/analytics-query/analytics-query.resolver.ts
Added GraphQL mutation RUN_ANALYTICS_QUERY in packages/twenty-front/src/modules/analytics-query/graphql/queries/runAnalyticsQuery.ts
Implemented UI components for analytics queries in packages/twenty-front/src/modules/activities/reports/components/

_{11 file(s) reviewed, 7 comment(s)}
_{Edit PR Review Bot Settings}

packages/twenty-front/src/modules/activities/reports/components/AnalyticsQueryFilter.tsx

packages/twenty-front/src/modules/activities/reports/components/AnalyticsQueryFilters.tsx

packages/twenty-front/src/modules/activities/reports/components/ReportRow.tsx

packages/twenty-server/src/engine/core-modules/analytics-query/analytics-query.service.ts

greptile-apps

PR Summary

(updates since last review)

Removed routes for ChartEditor, Charts, and Reports pages (packages/twenty-front/src/App.tsx)
Deleted multiple report-related components (PageAddReportButton.tsx, ReportGroup.tsx, ReportGroups.tsx, ReportRow.tsx, ReportsLayout.tsx, Reports.tsx)
Removed 'report' and 'reportId' properties from Chart interface (packages/twenty-front/src/modules/activities/reports/types/Chart.ts)
Renamed feature flag from 'IS_REPORTS_ENABLED' to 'IS_CHARTS_ENABLED' (packages/twenty-server/src/engine/core-modules/feature-flag/feature-flag.entity.ts, add-standard-id.command.ts)
Removed ReportWorkspaceEntity and related fields (chart.workspace-entity.ts, report.workspace-entity.ts)

_{22 file(s) reviewed, 2 comment(s)}
_{Edit PR Review Bot Settings}

packages/twenty-front/src/pages/reports/Charts.tsx

greptile-apps

PR Summary

(updates since last review)

Moved AnalyticsQueryEditor.tsx, AnalyticsQueryFilter.tsx, AnalyticsQueryFilters.tsx, Chart.tsx, ChartConfig.tsx, ReportGroupTimeSpans.ts, AnalyticsQuery.ts, AnalyticsQueryFilter.ts, and Chart.ts from reports to charts directory (packages/twenty-front/src/modules/activities/charts/components/)
Deleted useIsReportsPage.ts (packages/twenty-front/src/modules/navigation/hooks/)
Updated import paths in analytics-query.service.ts (packages/twenty-server/src/engine/core-modules/analytics-query/analytics-query.service.ts)
Updated import paths in index.ts (packages/twenty-server/src/engine/workspace-manager/workspace-sync-metadata/standard-objects/index.ts)
Moved analytics-query-filter.workspace-entity.ts, analytics-query.workspace-entity.ts, and chart.workspace-entity.ts from reports to charts module (packages/twenty-server/src/modules/charts/standard-objects/)

_{15 file(s) reviewed, 9 comment(s)}
_{Edit PR Review Bot Settings}

packages/twenty-front/src/modules/activities/charts/components/AnalyticsQueryEditor.tsx

packages/twenty-front/src/modules/activities/charts/components/AnalyticsQueryFilter.tsx

packages/twenty-front/src/modules/activities/charts/constants/ReportGroupTimeSpans.ts

packages/twenty-server/src/engine/core-modules/analytics-query/analytics-query.service.ts

greptile-apps

PR Summary

(updates since last review)

Updated import paths in packages/twenty-front/src/modules/activities/charts/components/AnalyticsQueryEditor.tsx
Added state management and mutation for analytics queries in packages/twenty-front/src/modules/activities/charts/components/AnalyticsQueryEditor.tsx
Renamed AnalyticsQueryFilterWorkspaceEntity to ChartFilterWorkspaceEntity in packages/twenty-server/src/modules/charts/standard-objects/chart-filter.workspace-entity.ts
Removed AnalyticsQueryWorkspaceEntity and added ChartFilterWorkspaceEntity in packages/twenty-server/src/engine/workspace-manager/workspace-sync-metadata/standard-objects/index.ts
Enhanced ChartWorkspaceEntity with new fields and relationships in packages/twenty-server/src/modules/charts/standard-objects/chart.workspace-entity.ts

_{13 file(s) reviewed, 7 comment(s)}
_{Edit PR Review Bot Settings}

packages/twenty-front/src/modules/activities/charts/components/AnalyticsQueryFilter.tsx

packages/twenty-front/src/modules/activities/charts/components/Chart.tsx

packages/twenty-front/src/pages/reports/Charts.tsx

packages/twenty-server/src/engine/core-modules/analytics-query/analytics-query.service.ts

packages/twenty-server/src/modules/charts/standard-objects/chart-filter.workspace-entity.ts

greptile-apps

PR Summary

(updates since last review)

Updated AnalyticsQueryEditor.tsx to use Chart type instead of AnalyticsQuery
Renamed AnalyticsQueryFilter.tsx to ChartFilter.tsx and updated props
Refactored AnalyticsQueryFilters.tsx to align with new Chart and ChartFilter types
Integrated useFindOneRecord hook in Chart.tsx for data fetching and loading state
Added support for displaying charts in RecordShowContainer.tsx and ShowPageRightContainer.tsx

_{9 file(s) reviewed, 4 comment(s)}
_{Edit PR Review Bot Settings}

packages/twenty-front/src/modules/activities/charts/components/AnalyticsQueryEditor.tsx

packages/twenty-front/src/modules/activities/charts/components/AnalyticsQueryFilter.tsx

greptile-apps

PR Summary

(updates since last review)

Removed runAnalyticsQuery mutation and button in packages/twenty-front/src/modules/activities/charts/components/AnalyticsQueryEditor.tsx
Added button to run chart query in packages/twenty-front/src/modules/activities/charts/components/Chart.tsx
Renamed mutation from RunAnalyticsQuery to RunChartQuery in packages/twenty-front/src/modules/analytics-query/graphql/queries/runAnalyticsQuery.ts
Deleted ChartEditor.tsx and Charts.tsx in packages/twenty-front/src/pages/reports
Updated UUIDs in CHART_STANDARD_FIELD_IDS in packages/twenty-server/src/engine/workspace-manager/workspace-sync-metadata/constants/standard-field-ids.ts

_{9 file(s) reviewed, 5 comment(s)}
_{Edit PR Review Bot Settings}

packages/twenty-front/src/modules/activities/charts/components/AnalyticsQueryEditor.tsx

packages/twenty-server/src/engine/core-modules/analytics-query/analytics-query.service.ts

greptile-apps · 2024-07-14T23:07:53Z

packages/twenty-server/src/engine/core-modules/analytics-query/analytics-query.service.ts

+      JSON.stringify(result, undefined, 2),
+    );
+
+    return { chartResult: JSON.stringify(result) };


Logic: Replace placeholder return value with actual query result processing.

packages/twenty-server/src/modules/charts/standard-objects/chart.workspace-entity.ts

charlesBochet · 2024-08-23T10:12:03Z

I'm going to close this PR as it's not ready to merge and we want to keep a clean history but let's keep going on it in September!

ad-elias added 10 commits July 2, 2024 00:31

Add reports page

21cbb4f

Reports data model draft

a5d97fb

Disable chart entity to make workspace:sync-metadata work

0b51b48

Add chart entity

80ae167

Fix report standardId

ee43744

Add relation between chart and analyticsQuery

388a062

Add create report button and load reports

0c6e7a4

Draft reports UI

398598c

Draft chart editor UI

d9dee57

Remove debug print

1e42afe

greptile-apps bot reviewed Jul 8, 2024

View reviewed changes

FelixMalfait reviewed Jul 8, 2024

View reviewed changes

packages/twenty-server/src/modules/reports/standard-objects/analytics-query.workspace-entity.ts Outdated Show resolved Hide resolved

FelixMalfait reviewed Jul 8, 2024

View reviewed changes

packages/twenty-server/src/modules/reports/standard-objects/analytics-query.workspace-entity.ts Outdated Show resolved Hide resolved

FelixMalfait reviewed Jul 8, 2024

View reviewed changes

FelixMalfait self-assigned this Jul 8, 2024

Create RunAnalyticsQuery mutation

165f508

greptile-apps bot reviewed Jul 8, 2024

View reviewed changes

Remove reports

2566c14

greptile-apps bot reviewed Jul 12, 2024

View reviewed changes

packages/twenty-front/src/pages/reports/Charts.tsx Outdated Show resolved Hide resolved

packages/twenty-front/src/pages/reports/Charts.tsx Outdated Show resolved Hide resolved

Rename reports to charts

f228dad

greptile-apps bot reviewed Jul 12, 2024

View reviewed changes

Merge AnalyticsQueryWorkspaceEntity to ChartWorkspaceEntity

3348509

greptile-apps bot reviewed Jul 13, 2024

View reviewed changes

Add chart tab to ShowPage

ce29c62

greptile-apps bot reviewed Jul 14, 2024

View reviewed changes

Run a simple hardcoded chart SQL query from frontend

22f1991

greptile-apps bot reviewed Jul 14, 2024

View reviewed changes

FelixMalfait reviewed Jul 15, 2024

View reviewed changes

packages/twenty-server/src/modules/charts/standard-objects/chart.workspace-entity.ts Show resolved Hide resolved

ad-elias and others added 28 commits August 10, 2024 14:41

Field path empty value

d2cf9d2

Field path join generation WiP

1f5c8eb

Update todo

957dd54

Fix warnings

66e3f9b

Add table aliases

1c637af

Merge branch 'main' into feat/data-explorer

c2cbf95

Refactor

5462df2

Group by WiP

2dd88c0

Merge branch 'main' into feat/data-explorer

d12c034

Rename chart.fieldPath to chart.target, fix empty group by

2dc3d89

Update todo

97368cd

Microfix import

199b4da

Fix getTableAliasAndColumn

94fa77a

Draft composite type handling

bc3963b

Merge branch 'main' into feat/data-explorer

d1442c3

Fix getCommonTableExpressionDefinitions return type

86deef6

Common table expressions for composite type handling WiP

efa4a24

Merge branch 'main' into feat/data-explorer

8aaaffc

Refactor to support common table expressions

dfb501a

Merge branch 'main' into feat/data-explorer

ffe3661

Merge remote-tracking branch 'upstream/main' into feat/data-explorer

3621a22

Add soft delete to chart workspace entity

19e10ad

Draft chart query data structure

e41dc25

Group by common table expressions support WiP

f8e591f

Group by with common table expressions and the source table works

f1fa4a2

Fix group by common table expression

aa651b1

Improve chart query format

9bce68b

Persist data explorer query

27bee2a

charlesBochet closed this Aug 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Data explorer #6159

Data explorer #6159

ad-elias commented Jul 8, 2024 •

edited

Loading

greptile-apps bot left a comment

greptile-apps bot left a comment

FelixMalfait left a comment

FelixMalfait Jul 8, 2024

FelixMalfait Jul 8, 2024

ad-elias Jul 20, 2024

FelixMalfait Jul 8, 2024

FelixMalfait Jul 8, 2024

FelixMalfait Jul 8, 2024

ad-elias Jul 21, 2024

FelixMalfait Jul 8, 2024

ad-elias Jul 21, 2024

ad-elias Jul 22, 2024 •

edited

Loading

greptile-apps bot left a comment

greptile-apps bot left a comment

greptile-apps bot left a comment

greptile-apps bot left a comment

greptile-apps bot left a comment

greptile-apps bot left a comment

greptile-apps bot Jul 14, 2024

charlesBochet commented Aug 23, 2024

Data explorer #6159

Data explorer #6159

Conversation

ad-elias commented Jul 8, 2024 • edited Loading

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

FelixMalfait left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ad-elias Jul 22, 2024 • edited Loading

Choose a reason for hiding this comment

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot Jul 14, 2024

Choose a reason for hiding this comment

charlesBochet commented Aug 23, 2024

ad-elias commented Jul 8, 2024 •

edited

Loading

ad-elias Jul 22, 2024 •

edited

Loading