Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
be68eb5
Created IExtracyAsync, ITransformAsync, and ILoadAsync
Chris-Wolfgang Jun 21, 2025
41c9f2d
Enhance ETL interfaces with improved documentation
Chris-Wolfgang Jun 21, 2025
5cc79d3
Fix project file and clean up unused references
Chris-Wolfgang Jun 22, 2025
2d5d1ea
Added
Chris-Wolfgang Jun 22, 2025
3587dd7
Fix filename typo and update namespaces in projects
Chris-Wolfgang Jun 22, 2025
a41427f
Refactor namespaces for consistency across files
Chris-Wolfgang Jun 22, 2025
5ab07d5
Refactor parameter names for clarity in ETL classes
Chris-Wolfgang Jun 23, 2025
dc7495a
Implement cancellation support in ETL process
Chris-Wolfgang Jun 23, 2025
8750b51
Rename classes for clarity and consistency
Chris-Wolfgang Jun 23, 2025
261a9ff
Add iterfaces with progress
Chris-Wolfgang Jun 23, 2025
66de359
Implement async ETL operations with cancellation support
Chris-Wolfgang Jun 24, 2025
3880b8c
Refactor namespaces and clean up using directives
Chris-Wolfgang Jun 24, 2025
7fb94ef
Add IExtractWithProgressAndCancellation, ITransformWithProgressAndCan…
Chris-Wolfgang Jun 24, 2025
67dbdf3
Add Example6 project with ETL functionality and enhancements
Chris-Wolfgang Jun 25, 2025
47b5ffd
Update parameter descriptions in LoadAsync methods
Chris-Wolfgang Jul 1, 2025
13f42b8
Created `ExtractorBase`, `LoaderBase`, and `TransformerBase` classes …
Chris-Wolfgang Jul 2, 2025
040ca4e
Refactor ETL classes for .NET compatibility and testing
Chris-Wolfgang Jul 2, 2025
b29eadd
Refactor test project configuration and cleanup
Chris-Wolfgang Jul 2, 2025
73c1ad4
Add [NotNull] attributes to method parameters
Chris-Wolfgang Jul 5, 2025
321abb6
Add ExcludeFromCodeCoverage and refactor test classes
Chris-Wolfgang Jul 5, 2025
aa3e59b
Removed interface tests in favor of tests on ExtractorBase, Transform…
Chris-Wolfgang Jul 5, 2025
4c4cb13
Enhance error handling and add cancellation support
Chris-Wolfgang Jul 5, 2025
4bbbb61
Update target frameworks in project file
Chris-Wolfgang Jul 5, 2025
cac2341
Add ETL.png and update project metadata for nuget upload
Chris-Wolfgang Jul 9, 2025
932e7b7
Add versioning to project files
Chris-Wolfgang Jul 9, 2025
58d5c0b
Remove JetBrains.Annotations dependency
Chris-Wolfgang Jul 9, 2025
76f904d
Refactor project dependencies and clean up code
Chris-Wolfgang Jul 9, 2025
4344fbd
Remove coverlet.collector from test project
Chris-Wolfgang Jul 9, 2025
19bb382
Update test SDK and xUnit runner package versions
Chris-Wolfgang Jul 9, 2025
d869ed0
Update Microsoft.NET.Test.Sdk package reference
Chris-Wolfgang Jul 9, 2025
30bfbf0
Update target frameworks and package versions
Chris-Wolfgang Jul 9, 2025
07bc856
Update package versions for improved compatibility
Chris-Wolfgang Jul 9, 2025
dd49b99
Merge branch 'main' into develop
Chris-Wolfgang Jul 11, 2025
2df3980
Change T to TSource on IExtractXXXAsync interfaces
Chris-Wolfgang Jul 11, 2025
dbe2106
Added Docs/Readme.md file to solution
Chris-Wolfgang Jul 11, 2025
ce1241e
Converted CurrentItemCount, SkipItemCount and MaxItemCount from int t…
Chris-Wolfgang Jul 13, 2025
343238d
Modified MaxItemCount on ExtractorBase, TransformerBase, and LoaderBa…
Chris-Wolfgang Jul 13, 2025
8fbdb5f
Modified
Chris-Wolfgang Jul 13, 2025
8962d58
Updated version to 1.0.0-rc2
Chris-Wolfgang Jul 13, 2025
8ede9ec
Converted properties like MaxItemCount, SkipItemCount, CurrentItemCou…
Chris-Wolfgang Jul 16, 2025
95b168f
Updated README.md to include link to wiki
Chris-Wolfgang Jul 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added ETL.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
33 changes: 31 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,31 @@
# ETL-Abstractions
Interfaces and base classes for creating ETLs
# Wolfgang.Etl.Abstractions

This package contains interfaces and base classes for building ETLs using a specific design pattern

The ETL design pattern is a common approach in data processing that involves three main stages:
- **Extract**: Retrieving data from various sources.
- **Transform**: Processing and transforming the extracted data into a desired format.
- **Load**: Storing the transformed data into a target system.

The abstractions in this package provide a way to define and implement these stages
in a flexible and reusable manner. Each stage can be implemented with or without
support for cancellation and progress reporting, allowing for greater control
over the ETL process.

To build an ETL using this package, you would typically need to create 5 classes:
- One class for each of the three stages: Extract, Transform, and Load.
- One class representing the source data.
- One class representing the target data.
- One class that acts as the ETL orchestrator, coordinating the execution of the three stages.

The design uses lazy loading and lazy evaluation to ensure that data is processed only when needed.
This allows for efficient memory usage and can handle large datasets without loading everything into memory at once.

The process uses a pull method rather than a push method to move data through the pipeline.
The process starts when the ETL orchestrator calls the `LoadAsyc` method of the `Loader` class.
The loader will start enumerating through the list of items passed into its `LoadAsync` method.
This will intern trigger the `TransformAsync` method of the `Transformer` class, which will process each item
and yield the transformed results. The process of transformation will also trigger the `ExtractAsync` method of the `Extractor` class,
which will retrieve the necessary data from the source.

For more information check out the [documentation](https://github.com/Chris-Wolfgang/ETL-Abstractions/wiki)
167 changes: 167 additions & 0 deletions Wolfgang.Etl.Abstractions.sln
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 17
VisualStudioVersion = 17.0.31903.59
MinimumVisualStudioVersion = 10.0.40219.1
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Wolfgang.Etl.Abstractions", "src\Wolfgang.Etl.Abstractions\Wolfgang.Etl.Abstractions.csproj", "{C4987BAD-4513-955F-B3C1-7563D0C1A7A3}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{02EA681E-C7D8-13C7-8484-4AC65E1B71E8}"
ProjectSection(SolutionItems) = preProject
README.md = README.md
EndProjectSection
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "tests", "tests", "{8220BC33-6632-4D4C-9A50-B7978141A4E3}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Wolfgang.Etl.Abstractions.Tests.Unit", "tests\Wolfgang.Etl.Abstractions.Tests.Unit\Wolfgang.Etl.Abstractions.Tests.Unit.csproj", "{B8558C7F-934B-3DC1-AAED-D668CF964C8E}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "examples", "examples", "{207971F1-432C-4AB4-9DC4-16A9E5ACF812}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example1-BasicETL", "examples\Net8.0\Example1-BasicETL\Example1-BasicETL.csproj", "{F1BA1AF5-9A71-421B-963C-E277610FBD40}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Net80", "Net80", "{3C48157B-5E90-489E-9444-E01F51D59F86}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Net48", "Net48", "{336D72A1-8E5E-49DE-83D9-DF6BE458BA24}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example1-BasicETL", "examples\Net4.8\Example1-BasicETL\Example1-BasicETL.csproj", "{B715D0C5-3F1A-485C-92D1-FFB87A801AC3}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example2-WithCancellationToken", "examples\Net8.0\Example2-WithCancellationToken\Example2-WithCancellationToken.csproj", "{F4A2F47C-9687-0ABE-FEAE-D07873E0133A}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example3-WithGracefulCancellation", "examples\Net8.0\Example3-WithGracefulCancellation\Example3-WithGracefulCancellation.csproj", "{733AD8E6-D170-789A-6F61-13C041011037}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example2-WithCancellationToken", "examples\Net4.8\Example2-WithCancellationToken\Example2-WithCancellationToken.csproj", "{2DC16382-F59F-4024-B560-D400E09EF91F}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example3-WithGracefulCancellation", "examples\Net4.8\Example3-WithGracefulCancellation\Example3-WithGracefulCancellation.csproj", "{5A7CFA9D-DCE7-43FD-AF89-7418C36AAE14}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example4a-WithExtractorProgress", "examples\Net8.0\Example4a-WithExtractorProgress\Example4a-WithExtractorProgress.csproj", "{483AE567-071E-4797-B13B-84B142D4ED44}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example4b-WithTransformerProgress", "examples\Net8.0\Example4b-WithTransformerProgress\Example4b-WithTransformerProgress.csproj", "{6C19204D-7CC8-48D2-A475-49CD98931105}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example4c-WithLoaderProgress", "examples\Net8.0\Example4c-WithLoaderProgress\Example4c-WithLoaderProgress.csproj", "{E858F22E-01D7-4BE9-B8DE-0E4AFBA5B33A}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example4a-WithExtractorProgress", "examples\Net4.8\Example4a-WithExtractorProgress\Example4a-WithExtractorProgress.csproj", "{8068B622-46A9-4ABA-BDA3-6AB259D1682C}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example4c-WithLoaderProgress", "examples\Net4.8\Example4c-WithLoaderProgress\Example4c-WithLoaderProgress.csproj", "{6DA63E99-692F-461C-983A-270E5CBE8D45}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example4b-WithTransformerProgress", "examples\Net4.8\Example4b-WithTransformerProgress\Example4b-WithTransformerProgress.csproj", "{6BD7828E-55B4-4213-8DF9-7BFBA891B8EE}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example5a-ExtractorWithProgressAndCancellation", "examples\Net8.0\Example5a-ExtractorWithProgressAndCancellation\Example5a-ExtractorWithProgressAndCancellation.csproj", "{2D14E222-4E44-40AB-82EF-1E8ABCED0476}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example5a-ExtractorWithProgressAndCancellation", "examples\Net4.8\Example5a-ExtractorWithProgressAndCancellation\Example5a-ExtractorWithProgressAndCancellation.csproj", "{861EA36D-970E-4CFE-9E72-D3D12F0BBB60}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example6-ReducingDuplicateCode", "examples\Net8.0\Example6-ReducingDuplicateCode\Example6-ReducingDuplicateCode.csproj", "{80E49C71-1073-4208-B48F-E0F399946B3B}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Example6-ReducingDuplicateCode", "examples\Net4.8\Example6-ReducingDuplicateCode\Example6-ReducingDuplicateCode.csproj", "{85A15A15-D528-4542-A546-63BE0EAED986}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Docs", "Docs", "{8EC462FD-D22E-90A8-E5CE-7E832BA40C5D}"
ProjectSection(SolutionItems) = preProject
docs\readme.md = docs\readme.md
EndProjectSection
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{C4987BAD-4513-955F-B3C1-7563D0C1A7A3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C4987BAD-4513-955F-B3C1-7563D0C1A7A3}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C4987BAD-4513-955F-B3C1-7563D0C1A7A3}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C4987BAD-4513-955F-B3C1-7563D0C1A7A3}.Release|Any CPU.Build.0 = Release|Any CPU
{B8558C7F-934B-3DC1-AAED-D668CF964C8E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{B8558C7F-934B-3DC1-AAED-D668CF964C8E}.Debug|Any CPU.Build.0 = Debug|Any CPU
{B8558C7F-934B-3DC1-AAED-D668CF964C8E}.Release|Any CPU.ActiveCfg = Release|Any CPU
{B8558C7F-934B-3DC1-AAED-D668CF964C8E}.Release|Any CPU.Build.0 = Release|Any CPU
{F1BA1AF5-9A71-421B-963C-E277610FBD40}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{F1BA1AF5-9A71-421B-963C-E277610FBD40}.Debug|Any CPU.Build.0 = Debug|Any CPU
{F1BA1AF5-9A71-421B-963C-E277610FBD40}.Release|Any CPU.ActiveCfg = Release|Any CPU
{F1BA1AF5-9A71-421B-963C-E277610FBD40}.Release|Any CPU.Build.0 = Release|Any CPU
{B715D0C5-3F1A-485C-92D1-FFB87A801AC3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{B715D0C5-3F1A-485C-92D1-FFB87A801AC3}.Debug|Any CPU.Build.0 = Debug|Any CPU
{B715D0C5-3F1A-485C-92D1-FFB87A801AC3}.Release|Any CPU.ActiveCfg = Release|Any CPU
{B715D0C5-3F1A-485C-92D1-FFB87A801AC3}.Release|Any CPU.Build.0 = Release|Any CPU
{F4A2F47C-9687-0ABE-FEAE-D07873E0133A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{F4A2F47C-9687-0ABE-FEAE-D07873E0133A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{F4A2F47C-9687-0ABE-FEAE-D07873E0133A}.Release|Any CPU.ActiveCfg = Release|Any CPU
{F4A2F47C-9687-0ABE-FEAE-D07873E0133A}.Release|Any CPU.Build.0 = Release|Any CPU
{733AD8E6-D170-789A-6F61-13C041011037}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{733AD8E6-D170-789A-6F61-13C041011037}.Debug|Any CPU.Build.0 = Debug|Any CPU
{733AD8E6-D170-789A-6F61-13C041011037}.Release|Any CPU.ActiveCfg = Release|Any CPU
{733AD8E6-D170-789A-6F61-13C041011037}.Release|Any CPU.Build.0 = Release|Any CPU
{2DC16382-F59F-4024-B560-D400E09EF91F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{2DC16382-F59F-4024-B560-D400E09EF91F}.Debug|Any CPU.Build.0 = Debug|Any CPU
{2DC16382-F59F-4024-B560-D400E09EF91F}.Release|Any CPU.ActiveCfg = Release|Any CPU
{2DC16382-F59F-4024-B560-D400E09EF91F}.Release|Any CPU.Build.0 = Release|Any CPU
{5A7CFA9D-DCE7-43FD-AF89-7418C36AAE14}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{5A7CFA9D-DCE7-43FD-AF89-7418C36AAE14}.Debug|Any CPU.Build.0 = Debug|Any CPU
{5A7CFA9D-DCE7-43FD-AF89-7418C36AAE14}.Release|Any CPU.ActiveCfg = Release|Any CPU
{5A7CFA9D-DCE7-43FD-AF89-7418C36AAE14}.Release|Any CPU.Build.0 = Release|Any CPU
{483AE567-071E-4797-B13B-84B142D4ED44}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{483AE567-071E-4797-B13B-84B142D4ED44}.Debug|Any CPU.Build.0 = Debug|Any CPU
{483AE567-071E-4797-B13B-84B142D4ED44}.Release|Any CPU.ActiveCfg = Release|Any CPU
{483AE567-071E-4797-B13B-84B142D4ED44}.Release|Any CPU.Build.0 = Release|Any CPU
{6C19204D-7CC8-48D2-A475-49CD98931105}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{6C19204D-7CC8-48D2-A475-49CD98931105}.Debug|Any CPU.Build.0 = Debug|Any CPU
{6C19204D-7CC8-48D2-A475-49CD98931105}.Release|Any CPU.ActiveCfg = Release|Any CPU
{6C19204D-7CC8-48D2-A475-49CD98931105}.Release|Any CPU.Build.0 = Release|Any CPU
{E858F22E-01D7-4BE9-B8DE-0E4AFBA5B33A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{E858F22E-01D7-4BE9-B8DE-0E4AFBA5B33A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{E858F22E-01D7-4BE9-B8DE-0E4AFBA5B33A}.Release|Any CPU.ActiveCfg = Release|Any CPU
{E858F22E-01D7-4BE9-B8DE-0E4AFBA5B33A}.Release|Any CPU.Build.0 = Release|Any CPU
{8068B622-46A9-4ABA-BDA3-6AB259D1682C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{8068B622-46A9-4ABA-BDA3-6AB259D1682C}.Debug|Any CPU.Build.0 = Debug|Any CPU
{8068B622-46A9-4ABA-BDA3-6AB259D1682C}.Release|Any CPU.ActiveCfg = Release|Any CPU
{8068B622-46A9-4ABA-BDA3-6AB259D1682C}.Release|Any CPU.Build.0 = Release|Any CPU
{6DA63E99-692F-461C-983A-270E5CBE8D45}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{6DA63E99-692F-461C-983A-270E5CBE8D45}.Debug|Any CPU.Build.0 = Debug|Any CPU
{6DA63E99-692F-461C-983A-270E5CBE8D45}.Release|Any CPU.ActiveCfg = Release|Any CPU
{6DA63E99-692F-461C-983A-270E5CBE8D45}.Release|Any CPU.Build.0 = Release|Any CPU
{6BD7828E-55B4-4213-8DF9-7BFBA891B8EE}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{6BD7828E-55B4-4213-8DF9-7BFBA891B8EE}.Debug|Any CPU.Build.0 = Debug|Any CPU
{6BD7828E-55B4-4213-8DF9-7BFBA891B8EE}.Release|Any CPU.ActiveCfg = Release|Any CPU
{6BD7828E-55B4-4213-8DF9-7BFBA891B8EE}.Release|Any CPU.Build.0 = Release|Any CPU
{2D14E222-4E44-40AB-82EF-1E8ABCED0476}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{2D14E222-4E44-40AB-82EF-1E8ABCED0476}.Debug|Any CPU.Build.0 = Debug|Any CPU
{2D14E222-4E44-40AB-82EF-1E8ABCED0476}.Release|Any CPU.ActiveCfg = Release|Any CPU
{2D14E222-4E44-40AB-82EF-1E8ABCED0476}.Release|Any CPU.Build.0 = Release|Any CPU
{861EA36D-970E-4CFE-9E72-D3D12F0BBB60}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{861EA36D-970E-4CFE-9E72-D3D12F0BBB60}.Debug|Any CPU.Build.0 = Debug|Any CPU
{861EA36D-970E-4CFE-9E72-D3D12F0BBB60}.Release|Any CPU.ActiveCfg = Release|Any CPU
{861EA36D-970E-4CFE-9E72-D3D12F0BBB60}.Release|Any CPU.Build.0 = Release|Any CPU
{80E49C71-1073-4208-B48F-E0F399946B3B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{80E49C71-1073-4208-B48F-E0F399946B3B}.Debug|Any CPU.Build.0 = Debug|Any CPU
{80E49C71-1073-4208-B48F-E0F399946B3B}.Release|Any CPU.ActiveCfg = Release|Any CPU
{80E49C71-1073-4208-B48F-E0F399946B3B}.Release|Any CPU.Build.0 = Release|Any CPU
{85A15A15-D528-4542-A546-63BE0EAED986}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{85A15A15-D528-4542-A546-63BE0EAED986}.Debug|Any CPU.Build.0 = Debug|Any CPU
{85A15A15-D528-4542-A546-63BE0EAED986}.Release|Any CPU.ActiveCfg = Release|Any CPU
{85A15A15-D528-4542-A546-63BE0EAED986}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(NestedProjects) = preSolution
{C4987BAD-4513-955F-B3C1-7563D0C1A7A3} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8}
{B8558C7F-934B-3DC1-AAED-D668CF964C8E} = {8220BC33-6632-4D4C-9A50-B7978141A4E3}
{F1BA1AF5-9A71-421B-963C-E277610FBD40} = {3C48157B-5E90-489E-9444-E01F51D59F86}
{3C48157B-5E90-489E-9444-E01F51D59F86} = {207971F1-432C-4AB4-9DC4-16A9E5ACF812}
{336D72A1-8E5E-49DE-83D9-DF6BE458BA24} = {207971F1-432C-4AB4-9DC4-16A9E5ACF812}
{B715D0C5-3F1A-485C-92D1-FFB87A801AC3} = {336D72A1-8E5E-49DE-83D9-DF6BE458BA24}
{F4A2F47C-9687-0ABE-FEAE-D07873E0133A} = {3C48157B-5E90-489E-9444-E01F51D59F86}
{733AD8E6-D170-789A-6F61-13C041011037} = {3C48157B-5E90-489E-9444-E01F51D59F86}
{2DC16382-F59F-4024-B560-D400E09EF91F} = {336D72A1-8E5E-49DE-83D9-DF6BE458BA24}
{5A7CFA9D-DCE7-43FD-AF89-7418C36AAE14} = {336D72A1-8E5E-49DE-83D9-DF6BE458BA24}
{483AE567-071E-4797-B13B-84B142D4ED44} = {3C48157B-5E90-489E-9444-E01F51D59F86}
{6C19204D-7CC8-48D2-A475-49CD98931105} = {3C48157B-5E90-489E-9444-E01F51D59F86}
{E858F22E-01D7-4BE9-B8DE-0E4AFBA5B33A} = {3C48157B-5E90-489E-9444-E01F51D59F86}
{8068B622-46A9-4ABA-BDA3-6AB259D1682C} = {336D72A1-8E5E-49DE-83D9-DF6BE458BA24}
{6DA63E99-692F-461C-983A-270E5CBE8D45} = {336D72A1-8E5E-49DE-83D9-DF6BE458BA24}
{6BD7828E-55B4-4213-8DF9-7BFBA891B8EE} = {336D72A1-8E5E-49DE-83D9-DF6BE458BA24}
{2D14E222-4E44-40AB-82EF-1E8ABCED0476} = {3C48157B-5E90-489E-9444-E01F51D59F86}
{861EA36D-970E-4CFE-9E72-D3D12F0BBB60} = {336D72A1-8E5E-49DE-83D9-DF6BE458BA24}
{80E49C71-1073-4208-B48F-E0F399946B3B} = {3C48157B-5E90-489E-9444-E01F51D59F86}
{85A15A15-D528-4542-A546-63BE0EAED986} = {336D72A1-8E5E-49DE-83D9-DF6BE458BA24}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {F673635D-58CE-48A5-9AE4-31F4484BED9E}
EndGlobalSection
EndGlobal
6 changes: 6 additions & 0 deletions examples/Net4.8/Example1-BasicETL/App.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.8" />
</startup>
</configuration>
23 changes: 23 additions & 0 deletions examples/Net4.8/Example1-BasicETL/ETL/ConsoleLoader.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Wolfgang.Etl.Abstractions;

namespace Example1_BasicETL.ETL
{
internal class ConsoleLoader : ILoadAsync<string>
{
public async Task LoadAsync(IAsyncEnumerable<string> items)
{
Console.WriteLine($"{ConsoleColors.Green}Loading{ConsoleColors.Reset} data to console asynchronously...\n");

await foreach (var item in items)
{
Console.WriteLine($"Loading item: {item}\n");
await Task.Delay(50); // Simulate some delay for loading
}

Console.WriteLine($"{ConsoleColors.Green}Loading{ConsoleColors.Reset} completed.\n");
}
}
}
29 changes: 29 additions & 0 deletions examples/Net4.8/Example1-BasicETL/ETL/FibonacciExtractor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Wolfgang.Etl.Abstractions;

namespace Example1_BasicETL.ETL
{
internal class FibonacciExtractor : IExtractAsync<int>
{
public async IAsyncEnumerable<int> ExtractAsync()
{
Console.WriteLine($"{ConsoleColors.Green}Extracting{ConsoleColors.Reset} Fibonacci numbers asynchronously...\n");

var current = 1;
var previous = 0;
for (var x = 0; x < 10; ++x)
{
Console.WriteLine($"Extracting Fibonacci number {x + 1}: {current}");
yield return current;
var temp = current;
current += previous;
previous = temp;
await Task.Yield(); // Simulate asynchronous operation
}

Console.WriteLine($"{ConsoleColors.Green}Extraction{ConsoleColors.Reset} completed.\n");
}
}
}
24 changes: 24 additions & 0 deletions examples/Net4.8/Example1-BasicETL/ETL/IntToStringTransformer.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Wolfgang.Etl.Abstractions;

namespace Example1_BasicETL.ETL
{
internal class IntToStringTransformer : ITransformAsync<int, string>
{
public async IAsyncEnumerable<string> TransformAsync(IAsyncEnumerable<int> items)
{
Console.WriteLine($"{ConsoleColors.Green}Transforming{ConsoleColors.Reset} integers to strings asynchronously...\n");

await foreach (var item in items)
{
Console.WriteLine($"Transforming integer {item} to string.");
await Task.Delay(50); // Simulate some delay for transformation
yield return item.ToString();
}

Console.WriteLine($"{ConsoleColors.Green}Transformation{ConsoleColors.Reset} completed.\n");
}
}
}
Loading