You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+2-2
Original file line number
Diff line number
Diff line change
@@ -4,8 +4,8 @@ Represents the **NuGet** versions.
4
4
5
5
## v3.0.0
6
6
-*Enhancement:* Major **"Sidecar"** feature added based on feedback from the community.
7
-
- The existing implementation required all generated supporting database capabilities to be within the "main" database itself; a new option has been added to generate a separate "sidecar" database to manage. This minimizes the impact on the "main" database.
8
-
- The "main" database will still require the database CDC (change-data-capture) capabilities to be enabled.
7
+
- The existing implementation required all generated supporting database capabilities to be within the "source" database itself; a new option has been added to generate a separate "sidecar" database to manage. This minimizes the impact on the "source" database.
8
+
- The "source" database will still require the database CDC (change-data-capture) capabilities to be enabled.
9
9
- The new [`EntitySidecarOrchestratorBase`](./src/NTangle/Cdc/EntitySidecarOrchestratorBase.cs) will invoke a single statement to leverage the CDC capabilities and perform the resulting data selection; see the [`ContactExecuteBatch.sql`](./samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Resources/Generated/ContactExecuteBatch.sql).
10
10
- The required `NTangle` (and optional `Outbox`) schema(s), table(s) and stored procedures will be generated within the "sidecar" database.
11
11
- Note that there are _no_ cross database dependencies; as such, the "sidecar" database can be hosted separately, be on a difference versions, etc. as required. The .NET orchestrator logic will _require_ access to both databases to function.
Copy file name to clipboardExpand all lines: docs/SqlServer.md
+10-10
Original file line number
Diff line number
Diff line change
@@ -6,12 +6,12 @@ This documents the _nTangle_ [architecture](#Architecture) and corresponding [co
6
6
7
7
## Overview
8
8
9
-
At its core _nTangle_ is a database-driven code-generation solution, in that it leverages an existing database to infer its underlying schema (tables and columns). A YAML (or JSON) configuration file is required to define the CDC requirements, being the root and child tables, and their relationships, representing the entity (aggregate root). The _nTangle_ code-gen tool connects to the database, and using the YAML configuration generates the required database (main and sidecar), and .NET publishing runtime artefacts. This publishing runtime then uses the database's CDC capabilities to detect changes to the underlying tables and trigger the creation of the related entity events that are then published.
9
+
At its core _nTangle_ is a database-driven code-generation solution, in that it leverages an existing source database to infer its underlying schema (tables and columns). A YAML (or JSON) configuration file is required to define the CDC requirements, being the root and child tables, and their relationships, representing the entity (aggregate root). The _nTangle_ code-gen tool connects to the database, and using the YAML configuration generates the required database (source and sidecar), and .NET publishing runtime artefacts. This publishing runtime then uses the database's CDC capabilities to detect changes to the underlying tables and trigger the creation of the related entity events that are then published.
10
10
11
11
![Overview](../images/Overview.png"Overview")
12
12
13
13
The core components are:
14
-
-**Database** - the existing tables (main database), plus generated _nTangle_ runtime artefacts (including outbox) in separate sidecar database;
14
+
-**Database** - the existing tables (source database), plus generated _nTangle_ runtime artefacts (including outbox) in separate sidecar database;
15
15
-**Config** - the _nTangle_ code-gen configuration;
16
16
-**Tooling** - the _nTangle_ code generator and resulting generated publisher .NET runtime;
17
17
-**Events** - the resulting published entity event (depicted as a [CloudEvent](https://cloudevents.io/)).
@@ -44,9 +44,9 @@ Microsoft SQL Server provides two change tracking capabilities, namely CDC and [
44
44
45
45
## Sidecar database
46
46
47
-
As of version `3.0.0` the preferred (recommended and default) approach is to use a sidecar database to manage the _nTangle_ runtime artefacts. This is to ensure that the main database is not polluted with the _nTangle_ specific artefacts, and to ensure that the _nTangle_ runtime can be easily removed without impacting the main database as required.
47
+
As of version `3.0.0` the preferred (recommended and default) approach is to use a sidecar database to manage the _nTangle_ runtime artefacts. This is to ensure that the source database is not polluted with the _nTangle_ specific artefacts, and to ensure that the _nTangle_ runtime can be easily removed without impacting the source database as required.
48
48
49
-
Primarily, usage of a sidecar database will also limit impact (load and data) on the main database by minimizing access to the CDC and related entity data selection only. Otherwise, the required orchestration will occur against the sidecar database.
49
+
Primarily, usage of a sidecar database will also limit impact (load and data) on the source database by minimizing access to the CDC and related entity data selection only. Otherwise, the required orchestration will occur against the sidecar database.
50
50
51
51
Note that there are _no_ cross database dependencies; as such, the sidecar database can be hosted separately, be on a different version, etc. as required. The .NET orchestrator logic _will_ however require access to both databases to function.
52
52
@@ -61,7 +61,7 @@ The following represents the high-level conceptual run-time architecture for a s
61
61
</br>
62
62
63
63
The SQL Server databases are as follows:
64
-
-**Main** - the existing database schema (tables and columns) that is being monitored for changes (CDC).
64
+
-**Source** - the existing database schema (tables and columns) that is being monitored for changes (CDC).
65
65
-**Sidecar** - the generated _nTangle_ runtime artefacts (tables and stored procedures) that are used to orchestrate the CDC and event publishing.
66
66
67
67
The key .NET components are as follows.
@@ -94,7 +94,7 @@ An orchestrator is generated per [entity](../src/NTangle/IEntity.cs) (the aggreg
94
94
95
95
A batch check is performed against the sidecar database using the [`spXxxBatchExecute`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.SidecarDb/Schema/NTangle/Stored%20Procedures/Generated/spContactBatchExecute.sql) stored procedure to determine if there is already an incomplete Batch and attempt to reprocess; otherwise, determine current CDC position from last complete Batch to continue. The batch tracking is persisted in the [`XxxBatchTracking`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.SidecarDb/Migrations/20241024-233018-03-create-ntangle-contactbatchtracking-table.sql) table.
96
96
97
-
The orchestrator will the select (detect) the changes as enabled by the SQL Server CDC capabilities in the main database. This is achieved by invoking the [`XxxExecuteBatch`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Resources/Generated/ContactExecuteBatch.sql) SQL statement and updating the corresponding result sets into the .NET entity equivalents. The following steps are performed.
97
+
The orchestrator will the select (detect) the changes as enabled by the SQL Server CDC capabilities in the source database. This is achieved by invoking the [`XxxExecuteBatch`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Resources/Generated/ContactExecuteBatch.sql) SQL statement and updating the corresponding result sets into the .NET entity equivalents. The following steps are performed.
98
98
99
99
Step | Description
100
100
-|-
@@ -215,15 +215,15 @@ The hosting of the `XxxOrchestrator` to enable explicit execution is the respons
215
215
216
216
## Code-generation
217
217
218
-
The [code-generator](../tools/NTangle.Template/content/AppName.CodeGen/Program.cs) will leverage the [ntangle.yaml](../tools/NTangle.Template/content/AppName.CodeGen/ntangle.yaml) configuration to generate the [Main database](#Main-database), [Sidecar database](#Sidecar-database), and [.NET](#net) artefacts.
218
+
The [code-generator](../tools/NTangle.Template/content/AppName.CodeGen/Program.cs) will leverage the [ntangle.yaml](../tools/NTangle.Template/content/AppName.CodeGen/ntangle.yaml) configuration to generate the [source database](#Source-database), [Sidecar database](#Sidecar-database), and [.NET](#net) artefacts.
219
219
220
220
Where `Xxx` is referenced in the artefact name this is replaced with the name of the entity (root aggregate). Also, the artefact name represents the default, there are opportunities within the `ntangle.yaml` to change the behavior of these where applicable.
221
221
222
222
Finally, features such as _event outbox_ and _identity mapping_ are configurable; where not leveraged their respective artefacts will not be generated.
223
223
224
224
<br/>
225
225
226
-
### Main database
226
+
### Source database
227
227
228
228
The `AppName.Database` project generated artefacts are as follows.
229
229
@@ -267,6 +267,6 @@ Data | [`XxxOrchestrator`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.
267
267
Data | [`EventOutboxDequeue`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Data/Generated/EventOutboxDequeue.cs) | Event outbox dequeue.
268
268
Data | [`EventOutboxEnqueue`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Data/Generated/EventOutboxEnqueue.cs) | Event outbox enqueue.
269
269
Entities | [`XxxCdc`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Entities/Generated/ContactCdc.cs) | Entity (aggregate root) representation of database table(s) and relationships (per entity).
270
-
Resources | [`XxxExecuteBatch.sql`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Resources/Generated/ContactExecuteBatch.sql) | TSQL statement (main database) to execute the batch using CDC (per entity).
271
-
Resources | [`XxxExecuteExplicit.sql`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Resources/Generated/ContactExecuteExplicit.sql) | TSQL statement (main database) to execute explicitly without Batch and CDC (per entity).
270
+
Resources | [`XxxExecuteBatch.sql`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Resources/Generated/ContactExecuteBatch.sql) | TSQL statement (source database) to execute the batch using CDC (per entity).
271
+
Resources | [`XxxExecuteExplicit.sql`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Resources/Generated/ContactExecuteExplicit.sql) | TSQL statement (source database) to execute explicitly without Batch and CDC (per entity).
272
272
Services | [`XxxHostedService`](../samples/SqlServerSidecarDemo/SqlServerSidecarDemo.Publisher/Services/Generated/ContactHostedService.cs) | Timer-based host for the `XxxCdcOrchestrator`.
0 commit comments