Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@
namespace CrestApps.Core.AI.Connections;

/// <summary>
/// Provides persisted storage for AI provider connections while preserving the standard
/// named-and-sourced catalog operations used by connection managers and editors.
/// Provides the merged runtime view of AI provider connections across all registered
/// binding sources while preserving the standard named-and-sourced catalog operations
/// used by connection managers and editors.
/// </summary>
public interface IAIProviderConnectionStore : INamedSourceCatalog<AIProviderConnection>
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@
namespace CrestApps.Core.AI.Deployments;

/// <summary>
/// Provides persisted storage for AI deployments while preserving the standard
/// named-and-sourced catalog operations used by deployment managers and editors.
/// Provides the merged runtime view of AI deployments across all registered binding
/// sources while preserving the standard named-and-sourced catalog operations used by
/// deployment managers and editors.
/// </summary>
public interface IAIDeploymentStore : INamedSourceCatalog<AIDeployment>
{
Expand Down
1 change: 1 addition & 0 deletions src/CrestApps.Core.Docs/docs/changelog/v1.0.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ description: Initial standalone release notes for the CrestApps.Core repository.
- includes a reference MVC host and an Aspire host for local composition and testing
- includes a dedicated `CrestApps.Core.Tests` project for framework validation
- publishes a framework-focused documentation site at [core.crestapps.com](https://core.crestapps.com)
- makes the generic AI deployment and connection catalog interfaces provider-backed database stores when YesSql or EntityCore is registered, while keeping `IAIDeploymentStore` and `IAIProviderConnectionStore` as the merged runtime views across configuration and database sources
- merges appsettings-backed and UI-managed AI provider connections and deployments through runtime catalogs, so MVC selectors and AI resolution stay current without rebuilding options or restarting the app
- exposes merged AI connection and deployment views through `INamedSourceCatalog<T>` registrations, adds generic `Add*DocumentCatalog<TModel, TIndex, T>()` helpers for custom catalog registration, keeps deterministic name conflict handling so UI-managed records override conflicting appsettings entries, and standardizes settings so connections and deployments are configured separately
- documents the default AI configuration sections (`CrestApps:AI:Connections` and `CrestApps:AI:Deployments`), tightens the quick-start path around Chat Interactions, and refreshes the docs navigation and landing page for faster onboarding
Expand Down
2 changes: 1 addition & 1 deletion src/CrestApps.Core.Docs/docs/core/ai-core.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ An **AI connection** stores credentials and endpoint information for a specific
| `INamedSourceCatalogSource<AIProviderConnection>` | `ConfigurationAIProviderConnectionSource` | Scoped | Reads connections from `appsettings.json` (Order 100) |
| `ITemplateService` | *(from AddCoreAITemplating)* | Scoped | Template rendering |

It also chains `AddCoreAITemplating()` and `AddCoreServices()` automatically, and forwards `IAIDeploymentStore` to `INamedSourceCatalog<AIDeployment>`, `INamedCatalog<AIDeployment>`, `ISourceCatalog<AIDeployment>`, and `ICatalog<AIDeployment>` (and the same forwarding for `AIProviderConnection`). This means you can inject any of these catalog interfaces and they will resolve through the multi-source store.
It also chains `AddCoreAITemplating()` and `AddCoreServices()` automatically. `IAIDeploymentStore` and `IAIProviderConnectionStore` are the merged runtime views across all registered binding sources. The generic catalog interfaces for those two models are intentionally left unbound by `AddCoreAIServices()` alone so the persistence packages can map `INamedSourceCatalog<T>`, `INamedCatalog<T>`, `ISourceCatalog<T>`, and `ICatalog<T>` to the concrete database-backed catalogs.

Optional format-specific packages stay opt-in. For example, Markdown-aware normalization lives in `CrestApps.Core.AI.Markdown`, so hosts that want Markdig-backed RAG normalization should register `AddCoreAIMarkdown()` explicitly instead of expecting `AddCoreAIServices()` to pull it in automatically.

Expand Down
13 changes: 10 additions & 3 deletions src/CrestApps.Core.Docs/docs/core/data-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ In `CrestApps.Core.Data.YesSql`, the shared convention is to keep each index typ
Each feature requires specific stores to be registered. The table below lists what each feature needs and the corresponding registration calls for YesSql and Entity Framework Core.

:::tip
`AddCoreAIServices()` registers the multi-source stores for `AIDeployment` and `AIProviderConnection` with appsettings-backed binding sources automatically. You only need to register the persistence-layer binding sources to enable database-backed storage.
`AddCoreAIServices()` registers the multi-source stores for `AIDeployment` and `AIProviderConnection` with appsettings-backed binding sources automatically. The persistence-layer packages then map the shared generic catalog interfaces to the concrete database-backed catalogs for those models.
:::

### Core AI (always required)
Expand All @@ -424,7 +424,7 @@ Registered automatically by `AddCoreAIServices()`:
| `AIDeployment` | `IAIDeploymentStore` | Auto-registered (multi-source, config source at Order 100) |
| `AIProviderConnection` | `IAIProviderConnectionStore` | Auto-registered (multi-source, config source at Order 100) |

Add a DB binding source if you want database-backed deployments and connections:
Add a persistence provider if you want the generic catalog interfaces for deployments and connections to resolve to the database-backed store:

```csharp
// Entity Framework Core (included in AddEntityCoreStores())
Expand All @@ -436,6 +436,13 @@ services.AddYesSqlNamedSourceBindingSource<AIDeployment, AIDeploymentIndex>();
services.AddYesSqlNamedSourceBindingSource<AIProviderConnection, AIProviderConnectionIndex>();
```

After a persistence provider is registered:

| Model | All sources | Database-backed catalog |
|-------|-------------|------------------------|
| `AIDeployment` | `IAIDeploymentStore` | `INamedSourceCatalog<AIDeployment>` / `ICatalog<AIDeployment>` |
| `AIProviderConnection` | `IAIProviderConnectionStore` | `INamedSourceCatalog<AIProviderConnection>` / `ICatalog<AIProviderConnection>` |

### AI Profiles

| Model | Catalog registration | EntityCore | YesSql |
Expand Down Expand Up @@ -691,7 +698,7 @@ The configuration sources read from `appsettings.json` using the sections config

Hosts can override those lists to focus only on the standalone `Connections` array or to append additional provider-grouped sections for compatibility scenarios.

When a persistence package (YesSql or EntityCore) is added, it registers an additional **writable** DB binding source at Order 0, so database entries take priority over `appsettings.json` entries and all write operations go to the database.
When a persistence package (YesSql or EntityCore) is added, it registers an additional **writable** DB binding source at Order 0, so database entries take priority over `appsettings.json` entries and all write operations go to the database. `IAIDeploymentStore` and `IAIProviderConnectionStore` still remain the merged runtime view; resolve the shared generic catalog interfaces when you specifically want the database-backed store.

### Registering DB binding sources

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
using CrestApps.Core.AI.Clients;
using CrestApps.Core.AI.Deployments;
using CrestApps.Core.AI.Indexing;
using CrestApps.Core.AI.Models;
using CrestApps.Core.Infrastructure;
using CrestApps.Core.Infrastructure.Indexing;
using CrestApps.Core.Infrastructure.Indexing.Models;
using CrestApps.Core.Services;
using Microsoft.Extensions.Logging;

namespace CrestApps.Core.AI.Documents.Indexing;
Expand All @@ -21,7 +20,7 @@ public sealed class AIDocumentSearchIndexProfileHandler : EmbeddingSearchIndexPr
/// <param name="aiClientFactory">The ai client factory.</param>
/// <param name="logger">The logger.</param>
public AIDocumentSearchIndexProfileHandler(
ICatalog<AIDeployment> deploymentCatalog,
IAIDeploymentStore deploymentCatalog,
IAIClientFactory aiClientFactory,
ILogger<AIDocumentSearchIndexProfileHandler> logger)
: base(IndexProfileTypes.AIDocuments, deploymentCatalog, aiClientFactory, logger)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
using CrestApps.Core.AI.Clients;
using CrestApps.Core.AI.Models;
using CrestApps.Core.AI.Deployments;
using CrestApps.Core.Infrastructure.Indexing;
using CrestApps.Core.Infrastructure.Indexing.Models;
using CrestApps.Core.Services;
using Microsoft.Extensions.Logging;

namespace CrestApps.Core.AI.Indexing;
Expand All @@ -27,7 +26,7 @@ public sealed class AIMemorySearchIndexProfileHandler : EmbeddingSearchIndexProf
/// <param name="aiClientFactory">The ai client factory.</param>
/// <param name="logger">The logger.</param>
public AIMemorySearchIndexProfileHandler(
ICatalog<AIDeployment> deploymentCatalog,
IAIDeploymentStore deploymentCatalog,
IAIClientFactory aiClientFactory,
ILogger<AIMemorySearchIndexProfileHandler> logger)
: base(IndexProfileTypes.AIMemory, deploymentCatalog, aiClientFactory, logger)
Expand All @@ -40,43 +39,52 @@ public AIMemorySearchIndexProfileHandler(
/// <param name="vectorDimensions">The vector dimensions.</param>
protected override IReadOnlyCollection<SearchIndexField> BuildFields(int vectorDimensions)
{
return [new SearchIndexField
{
Name = _memoryIdFieldName,
FieldType = SearchFieldType.Keyword,
IsKey = true,
IsFilterable = true,
}, new SearchIndexField
{
Name = _userIdFieldName,
FieldType = SearchFieldType.Keyword,
IsFilterable = true,
}, new SearchIndexField
{
Name = _nameFieldName,
FieldType = SearchFieldType.Text,
IsSearchable = true,
IsFilterable = true,
}, new SearchIndexField
{
Name = _descriptionFieldName,
FieldType = SearchFieldType.Text,
IsSearchable = true,
}, new SearchIndexField
{
Name = _contentFieldName,
FieldType = SearchFieldType.Text,
IsSearchable = true,
}, new SearchIndexField
{
Name = _updatedUtcFieldName,
FieldType = SearchFieldType.DateTime,
IsFilterable = true,
}, new SearchIndexField
{
Name = _embeddingFieldName,
FieldType = SearchFieldType.Vector,
VectorDimensions = vectorDimensions,
}, ];
return
[
new SearchIndexField
{
Name = _memoryIdFieldName,
FieldType = SearchFieldType.Keyword,
IsKey = true,
IsFilterable = true,
},
new SearchIndexField
{
Name = _userIdFieldName,
FieldType = SearchFieldType.Keyword,
IsFilterable = true,
},
new SearchIndexField
{
Name = _nameFieldName,
FieldType = SearchFieldType.Text,
IsSearchable = true,
IsFilterable = true,
},
new SearchIndexField
{
Name = _descriptionFieldName,
FieldType = SearchFieldType.Text,
IsSearchable = true,
},
new SearchIndexField
{
Name = _contentFieldName,
FieldType = SearchFieldType.Text,
IsSearchable = true,
},
new SearchIndexField
{
Name = _updatedUtcFieldName,
FieldType = SearchFieldType.DateTime,
IsFilterable = true,
},
new SearchIndexField
{
Name = _embeddingFieldName,
FieldType = SearchFieldType.Vector,
VectorDimensions = vectorDimensions,
},
];
}
}
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
using CrestApps.Core.AI.Clients;
using CrestApps.Core.AI.Models;
using CrestApps.Core.AI.Deployments;
using CrestApps.Core.Infrastructure;
using CrestApps.Core.Infrastructure.Indexing;
using CrestApps.Core.Infrastructure.Indexing.Models;
using CrestApps.Core.Services;
using Microsoft.Extensions.Logging;

namespace CrestApps.Core.AI.Indexing;
Expand All @@ -20,7 +19,7 @@ public sealed class DataSourceSearchIndexProfileHandler : EmbeddingSearchIndexPr
/// <param name="aiClientFactory">The ai client factory.</param>
/// <param name="logger">The logger.</param>
public DataSourceSearchIndexProfileHandler(
ICatalog<AIDeployment> deploymentCatalog,
IAIDeploymentStore deploymentCatalog,
IAIClientFactory aiClientFactory,
ILogger<DataSourceSearchIndexProfileHandler> logger)
: base(IndexProfileTypes.DataSource, deploymentCatalog, aiClientFactory, logger)
Expand All @@ -33,51 +32,62 @@ public DataSourceSearchIndexProfileHandler(
/// <param name="vectorDimensions">The vector dimensions.</param>
protected override IReadOnlyCollection<SearchIndexField> BuildFields(int vectorDimensions)
{
return [new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.ChunkId,
FieldType = SearchFieldType.Keyword,
IsKey = true,
IsFilterable = true,
}, new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.ReferenceId,
FieldType = SearchFieldType.Keyword,
IsFilterable = true,
}, new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.DataSourceId,
FieldType = SearchFieldType.Keyword,
IsFilterable = true,
}, new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.ReferenceType,
FieldType = SearchFieldType.Keyword,
IsFilterable = true,
}, new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.ChunkIndex,
FieldType = SearchFieldType.Integer,
}, new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.Title,
FieldType = SearchFieldType.Text,
IsSearchable = true,
}, new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.Content,
FieldType = SearchFieldType.Text,
IsSearchable = true,
}, new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.Timestamp,
FieldType = SearchFieldType.DateTime,
IsFilterable = true,
}, new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.Embedding,
FieldType = SearchFieldType.Vector,
VectorDimensions = vectorDimensions,
}, ];
return
[
new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.ChunkId,
FieldType = SearchFieldType.Keyword,
IsKey = true,
IsFilterable = true,
},
new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.ReferenceId,
FieldType = SearchFieldType.Keyword,
IsFilterable = true,
},
new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.DataSourceId,
FieldType = SearchFieldType.Keyword,
IsFilterable = true,
},
new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.ReferenceType,
FieldType = SearchFieldType.Keyword,
IsFilterable = true,
},
new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.ChunkIndex,
FieldType = SearchFieldType.Integer,
},
new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.Title,
FieldType = SearchFieldType.Text,
IsSearchable = true,
},
new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.Content,
FieldType = SearchFieldType.Text,
IsSearchable = true,
},
new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.Timestamp,
FieldType = SearchFieldType.DateTime,
IsFilterable = true,
},
new SearchIndexField
{
Name = DataSourceConstants.ColumnNames.Embedding,
FieldType = SearchFieldType.Vector,
VectorDimensions = vectorDimensions,
},
];
}
}
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
using System.ComponentModel.DataAnnotations;
using CrestApps.Core.AI.Clients;
using CrestApps.Core.AI.Deployments;
using CrestApps.Core.AI.Models;
using CrestApps.Core.Infrastructure.Indexing.Models;
using CrestApps.Core.Models;
using CrestApps.Core.Services;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Logging;

Expand All @@ -15,7 +15,7 @@ namespace CrestApps.Core.AI.Indexing;
public abstract class EmbeddingSearchIndexProfileHandlerBase : IndexProfileHandlerBase
{
private readonly string _type;
private readonly ICatalog<AIDeployment> _deploymentCatalog;
private readonly IAIDeploymentStore _deploymentCatalog;
private readonly IAIClientFactory _aiClientFactory;
private readonly ILogger<EmbeddingSearchIndexProfileHandlerBase> _logger;

Expand All @@ -28,7 +28,7 @@ public abstract class EmbeddingSearchIndexProfileHandlerBase : IndexProfileHandl
/// <param name="logger">The logger.</param>
protected EmbeddingSearchIndexProfileHandlerBase(
string type,
ICatalog<AIDeployment> deploymentCatalog,
IAIDeploymentStore deploymentCatalog,
IAIClientFactory aiClientFactory,
ILogger<EmbeddingSearchIndexProfileHandlerBase> logger)
{
Expand Down
10 changes: 1 addition & 9 deletions src/Primitives/CrestApps.Core.AI/ServiceCollectionExtensions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -145,18 +145,10 @@ public static IServiceCollection AddCoreAIServices(this IServiceCollection servi
.AddScoped<IAIClientFactory, DefaultAIClientFactory>()
.AddScoped<ISpeechVoiceResolver, DefaultSpeechVoiceResolver>();

// Register the multi-source stores and forward all catalog interfaces.
// Register the multi-source stores used for merged runtime lookups.
services.TryAddScoped<IAIDeploymentStore, DefaultAIDeploymentStore>();
services.TryAddScoped<INamedSourceCatalog<AIDeployment>>(sp => sp.GetRequiredService<IAIDeploymentStore>());
services.TryAddScoped<INamedCatalog<AIDeployment>>(sp => sp.GetRequiredService<IAIDeploymentStore>());
services.TryAddScoped<ISourceCatalog<AIDeployment>>(sp => sp.GetRequiredService<IAIDeploymentStore>());
services.TryAddScoped<ICatalog<AIDeployment>>(sp => sp.GetRequiredService<IAIDeploymentStore>());

services.TryAddScoped<IAIProviderConnectionStore, DefaultAIProviderConnectionStore>();
services.TryAddScoped<INamedSourceCatalog<AIProviderConnection>>(sp => sp.GetRequiredService<IAIProviderConnectionStore>());
services.TryAddScoped<INamedCatalog<AIProviderConnection>>(sp => sp.GetRequiredService<IAIProviderConnectionStore>());
services.TryAddScoped<ISourceCatalog<AIProviderConnection>>(sp => sp.GetRequiredService<IAIProviderConnectionStore>());
services.TryAddScoped<ICatalog<AIProviderConnection>>(sp => sp.GetRequiredService<IAIProviderConnectionStore>());

// Register the configuration-backed sources (Order=100, lower priority than DB).
services.TryAddEnumerable(ServiceDescriptor.Scoped<INamedSourceCatalogSource<AIDeployment>, ConfigurationAIDeploymentSource>());
Expand Down
Loading
Loading