Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
02cd55e
Add ManagedType/SupportNull and extend value types
masesdevelopers Feb 17, 2026
5a8749a
Adds subtype for DateTime and DateTimeOffset to automatically support…
masesdevelopers Feb 17, 2026
dc1b833
#403: aligned configuration file to KNetReplicator obsolescence
masesdevelopers Feb 17, 2026
72fd5a3
Adds first complex test
masesdevelopers Feb 17, 2026
d4935f3
Merge branch 'master' into 410-extends-supported-object-and-nesting-l…
masesdevelopers Feb 17, 2026
149be9e
Update management of DateTimeOffset
masesdevelopers Feb 17, 2026
f3e9f67
Adds self compilation of avro schema
masesdevelopers Feb 17, 2026
bd4250b
Use configuration file to define backend engine and its configuration…
masesdevelopers Feb 17, 2026
8c4ef0c
Error fix
masesdevelopers Feb 17, 2026
4ca1873
Use KEFCore.SerDes.Avro.Compiler only in Debug
masesdevelopers Feb 17, 2026
ee8475b
Documentation review
masesdevelopers Feb 18, 2026
e99fd65
Merge branch 'master' into 410-extends-supported-object-and-nesting-l…
masesdevelopers Feb 18, 2026
62cfc5f
Adds test for NullableBooleanValue using new definition for topic name
masesdevelopers Feb 18, 2026
9b88094
Update howitworks.md
masesdevelopers Feb 18, 2026
806101a
Merge branch 'master' into 410-extends-supported-object-and-nesting-l…
masesdevelopers Feb 18, 2026
e00090d
Fix merge
masesdevelopers Feb 18, 2026
deecee5
Fix wrong table
masesdevelopers Feb 18, 2026
a70206f
Other updates
masesdevelopers Feb 18, 2026
4e76f87
https://github.com/masesgroup/KEFCore/pull/415#discussion_r2820193773
masesdevelopers Feb 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 69 additions & 23 deletions .github/workflows/build.yaml

Large diffs are not rendered by default.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"DatabaseName": "TestDBBenchmark",
"UseCompactedReplicator": true,
"BootstrapServers": "localhost:9092",
"NumberOfExecutions": 10,
"UseEnumeratorWithPrefetch": false
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"BootstrapServers": "localhost:9092",
"NumberOfExecutions": 10
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
{
"DatabaseName": "TestDBBenchmark",
"UseCompactedReplicator": false,
"BootstrapServers": "localhost:9092",
"NumberOfExecutions": 10,
"UseEnumeratorWithPrefetch": false
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
{
"DatabaseName": "TestDBBenchmark",
"UseCompactedReplicator": false,
"UseByteBufferDataTransfer": false,
"BootstrapServers": "localhost:9092",
"NumberOfExecutions": 10
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
{
"DatabaseName": "TestDBBenchmark",
"UseCompactedReplicator": false,
"UseByteBufferDataTransfer": false,
"BootstrapServers": "localhost:9092",
"NumberOfExecutions": 10,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
{
"DatabaseName": "TestDBBenchmark",
"UseCompactedReplicator": false,
"UseKNetStreams": false,
"BootstrapServers": "localhost:9092",
"NumberOfExecutions": 10
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
{
"DatabaseName": "TestDBBenchmark",
"UseCompactedReplicator": false,
"UseKNetStreams": false,
"UseByteBufferDataTransfer": false,
"BootstrapServers": "localhost:9092",
Expand Down
79 changes: 72 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# KEFCore: [Entity Framework Core](https://learn.microsoft.com/it-it/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/)
# KEFCore: [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/)

KEFCore is the [Entity Framework Core](https://learn.microsoft.com/it-it/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/).
KEFCore is the [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/).
Based on [KNet](https://github.com/masesgroup/KNet) it allows to use [Apache Kafka™](https://kafka.apache.org/) as a distributed database and more.

### Libraries and Tools
Expand All @@ -22,21 +22,86 @@ Its primary scope is to support other, public or internal, MASES Group projects:

The repository code and releases may contain bugs, the release cycle depends from critical discovered issues and/or enhancement requested from this or other projects.

Looking for the help of [Entity Framework Core](https://learn.microsoft.com/it-it/ef/core/) and [Apache Kafka™](https://kafka.apache.org/) experts? MASES Group can help you design, build, deploy, and manage [Entity Framework Core](https://learn.microsoft.com/it-it/ef/core/) and [Apache Kafka™](https://kafka.apache.org/) applications.
Looking for the help of [Entity Framework Core](https://learn.microsoft.com/ef/core/) and [Apache Kafka™](https://kafka.apache.org/) experts? MASES Group can help you design, build, deploy, and manage [Entity Framework Core](https://learn.microsoft.com/ef/core/) and [Apache Kafka™](https://kafka.apache.org/) applications.

---

## Scope of the project

This project aims to create a provider to access the information stored within an Apache Kafka™ cluster using the paradigm behind Entity Framework.
This project aims to create a provider able to save data to, and read data from, an Apache Kafka™ cluster using the paradigm behind Entity Framework.

Have you ever reached the [Entity Framework Core introduction page](https://learn.microsoft.com/ef/core/)?
The first example proposed is:

```c#
using System.Collections.Generic;
using Microsoft.EntityFrameworkCore;

namespace Intro;

public class BloggingContext : DbContext
{
public DbSet<Blog> Blogs { get; set; }
public DbSet<Post> Posts { get; set; }

protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer(
@"Server=(localdb)\mssqllocaldb;Database=Blogging;Trusted_Connection=True;ConnectRetryCount=0");
}
}

public class Blog
{
public int BlogId { get; set; }
public string Url { get; set; }
public int Rating { get; set; }
public List<Post> Posts { get; set; }
}

public class Post
{
public int PostId { get; set; }
public string Title { get; set; }
public string Content { get; set; }

public int BlogId { get; set; }
public Blog Blog { get; set; }
}
```
which uses a SQL Server instance to store and retrieve the data.

With [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/) any user can replace a database (the available providers always use a database) with an [Apache Kafka™](https://kafka.apache.org/) cluster.
So the following query:
```c#
using (var db = new BloggingContext())
{
var blogs = await db.Blogs
.Where(b => b.Rating > 3)
.OrderBy(b => b.Url)
.ToListAsync();
}
```
or the following operation:
```c#
using (var db = new BloggingContext())
{
var blog = new Blog { Url = "http://sample.com" };
db.Blogs.Add(blog);
await db.SaveChangesAsync();
}
```

can be done on a set of topics in an [Apache Kafka™](https://kafka.apache.org/) cluster.

The project is based on available information within the official [EntityFrameworkCore repository](https://github.com/dotnet/efcore), many classes was copied from there as reported in the official documentation within the Microsoft website at https://docs.microsoft.com/en-us/ef/core/providers/writing-a-provider.

Currently the project tries to support, at our best, the [supported Apache Kafka™ binary distribution](https://kafka.apache.org/downloads):
Currently the project tries to support, at our best, the [official supported Apache Kafka™ binary distribution](https://kafka.apache.org/downloads):

| KEFCore | State | Apache Kafka™ | .NET | JVM™ |
|:---: |:---: |:---: |:---: |:---: |
| 2.6.* | Active | 4.1.* | 8+ | 17+ |
| 2.5.* | Active | 3.9.* | 8+ | 11+ |
| 2.6.* | Active | 4.1.* | [![.NET 8+](https://img.shields.io/badge/.NET-8%2B-purple)](https://dotnet.microsoft.com/) | [![Java 17+](https://img.shields.io/badge/Java-17%2B-blue)](https://www.oracle.com/java/) |
| 2.5.* | Active | 3.9.* | [![.NET 8+](https://img.shields.io/badge/.NET-8%2B-purple)](https://dotnet.microsoft.com/) | [![Java 11+](https://img.shields.io/badge/Java-11%2B-blue)](https://www.oracle.com/java/) |

### Community and Contribution

Expand Down
6 changes: 3 additions & 3 deletions src/documentation/articles/externalapplication.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ _description: Describes how to use data managed by Entity Framework Core provide

# KEFCore: external application

[Entity Framework Core](https://learn.microsoft.com/it-it/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/) shall convert the entities used within the model in something viable from the backend.
Continuing from the concepts introduced in [serialization](serialization.md), an external application can use the data stored in the topics in a way it decides: [Entity Framework Core](https://learn.microsoft.com/it-it/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/) gives some helpers to get back the CLR Entity objects stored in the topics.
[Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/) shall convert the entities used within the model in something viable from the backend.
Continuing from the concepts introduced in [serialization](serialization.md), an external application can use the data stored in the topics in a way it decides: [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/) gives some helpers to get back the CLR Entity objects stored in the topics.

> IMPORTANT NOTE: till the first major version, all releases shall be considered not stable: this means the API public, or internal, can change without notice.

Expand Down Expand Up @@ -45,4 +45,4 @@ To work properly it needs, to be loaded in memory, at least:

## Possible usages

For possible usages of [Entity Framework Core](https://learn.microsoft.com/it-it/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/), and this feature, see [use cases](usecases.md)
For possible usages of [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/), and this feature, see [use cases](usecases.md)
2 changes: 1 addition & 1 deletion src/documentation/articles/gettingstarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ _description: Describes how to start to use Entity Framework Core provider for A

# KEFCore: Getting started

To use [Entity Framework Core](https://learn.microsoft.com/it-it/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/) you must have at least:
To use [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/) you must have at least:
- an installed JRE/JDK (11+)
- an accessible Apache Kafka™ broker (a full cluster or a local Dockerized version)

Expand Down
Loading
Loading