[WIP -DO NOT REVIEW] AI: Adds AzureOpenAIEmbeddingGenerator provider for issue #5842#5849
[WIP -DO NOT REVIEW] AI: Adds AzureOpenAIEmbeddingGenerator provider for issue #5842#5849ananth7592 wants to merge 10 commits into
Conversation
…on (preview) Adds the public surface for the V0 embedding-generation feature (parent #5830, this PR addresses #5831): * New `Microsoft.Azure.Cosmos.IEmbeddingGenerator` interface with a single batched `GenerateEmbeddingsAsync(IEnumerable<string>, CancellationToken)` method. The cancellation token is included now to avoid a breaking signature change later. * New `QueryRequestOptions.EmbeddingGenerator` property (defaults to null, so existing behavior is unchanged). Both are gated on `#if PREVIEW` (public in preview, internal in GA), mirroring the `ReadConsistencyStrategy` pattern. Tests: * QueryRequestOptionsUniTests: added EmbeddingGenerator_DefaultsToNull and EmbeddingGenerator_RoundTripsAssignedInstance. * Regenerated DotNetPreviewSDKAPI.net6.json baseline. The actual surface additions are limited to IEmbeddingGenerator + the new property; the rest of the diff is JSON key reordering by the regen tool. PreviewContractChanges and the standard ContractChanges suite pass. This PR is intentionally surface-only - no behavior is wired up yet. The DTO/feature-flag work, resolver, plumbing into the pipeline, diagnostics datum, and broader tests are tracked in #5832-#5837. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Rename IEmbeddingGenerator → ICosmosEmbeddingGenerator to avoid collision with Microsoft.Extensions.AI.IEmbeddingGenerator - Change GenerateEmbeddingsAsync return type double → float to align with SDK's own VectorEmbedding<float> vector data model - Add [JsonIgnore] on CosmosClientOptions.EmbeddingGenerator to prevent diagnostic serialization crash - Add null guard in CosmosClientBuilder.WithEmbeddingGenerator: embeddingGenerator ?? throw new ArgumentNullException(...) - Strengthen thread-safety documentation: implementations MUST be safe to invoke concurrently (parallel queries/partition reads on one instance) - Document per-request opt-out behavior as intentional: null falls through to client-level; callers needing opt-out must provide a no-op implementation - Regenerate DotNetPreviewSDKAPI.net6.json contract Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- New Microsoft.Azure.Cosmos.Embedding package analogous to Microsoft.Azure.Cosmos.Encryption - Adds src project (netstandard2.0, Azure.AI.OpenAI 2.1.0, SdkProjectRef pattern, SNK signing) - Adds empty tests project (net6.0, MSTest, testkey.snk) - Adds version properties to root Directory.Build.props (EmbeddingOfficialVersion 1.0.0) - Adds Embedding solution folder + both projects to Microsoft.Azure.Cosmos.sln Part of: #5841 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…icrosoft.Azure.Cosmos.AI Design pivot: The optional provider package is renamed to Microsoft.Azure.Cosmos.AI to reflect its broader scope beyond embeddings (semantic re-ranking, OCR, LLM/SLM integrations). Uppercase AI matches Microsoft.Extensions.AI and Azure.AI.* conventions. - Rename folder Microsoft.Azure.Cosmos.Embedding -> Microsoft.Azure.Cosmos.AI - Rename AssemblyName, RootNamespace, PackageId to Microsoft.Azure.Cosmos.AI - Rename MSBuild version properties from Embedding* to CosmosAI* - Update Description and PackageTags to reflect broader AI scope - Update solution file (solution folder, project entries, config platforms) - Update InternalsVisibleTo to reference Microsoft.Azure.Cosmos.AI.Tests Part of: #5841 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…osoft.Azure.Cosmos.AI Adds the official NuGet publishing pipeline for the Microsoft.Azure.Cosmos.AI package, mirroring the pattern used by Microsoft.Azure.Cosmos.Encryption: - azure-pipelines-cosmosai-official.yml: Top-level pipeline with Release Gates stage (static analysis) and Build/Pack/Publish stage. Triggered manually (trigger: none). - templates/static-tools-cosmosai.yml: Static analysis job with three build variants: NuGet SDK reference, SdkProjectRef=true, and parity check (SdkProjectRef + no symbol). Includes BinSkim, CredScan, PoliCheck, AntiMalware, Component Governance Detection. - templates/cosmosai-nuget-pack.yml: Build, pack, symbols pack, SBOM manifest, Azure blob copy (cosmosdb/csharp/cosmosai/\), and artifact publish. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ed by Cosmos SDK Microsoft.Azure.Cosmos enforces that consumers explicitly reference Newtonsoft.Json >= 10.0.2. Added it with PrivateAssets=All so the check is satisfied without making it a transitive dependency for Microsoft.Azure.Cosmos.AI consumers (same pattern as Encryption). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implements ICosmosEmbeddingGenerator backed by Azure OpenAI's EmbeddingClient: - 4 public constructors: (EmbeddingSource+apiKey), (EmbeddingSource+TokenCredential), (endpoint+deploymentName+dimensions+apiKey), (endpoint+deploymentName+dimensions+TokenCredential) - Batching: splits large inputs at the Azure OpenAI hard limit of 2048 per call; results are concatenated in original order - Error wrapping: RequestFailedException → CosmosException(503) with endpoint/deployment context - Input validation: null, empty, and whitespace-only elements rejected up-front - Thread-safe: fresh EmbeddingGenerationOptions per batch call (avoids SDK mutex contention) - Testability: internal test constructor accepts a Func delegate to avoid mocking sealed SDK types - 25 unit tests covering construction, validation, batching, order preservation, cancellation, error propagation, and dispose semantics Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ypes without requiring IsPreview globally - Add AdditionalProperties="IsPreview=true" to the ProjectReference so the core SDK always exposes ICosmosEmbeddingGenerator and EmbeddingSource as public when building from source (SdkProjectRef=true), without the caller needing to pass /p:IsPreview=true on the dotnet CLI. - Remove the broken non-preview NuGet condition ([3.59.0,)) — the AI package requires ICosmosEmbeddingGenerator which is #if PREVIEW-gated and therefore always internal in a non-preview SDK build. The package always targets the preview NuGet path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…uild work without flags The Cosmos.AI package depends on ICosmosEmbeddingGenerator and EmbeddingSource, which are #if PREVIEW-gated in the core SDK. Previously, building from Visual Studio or running 'dotnet build /p:SdkProjectRef=true' without also passing /p:IsPreview=true would fail to resolve those types because PREVIEW was not defined. Setting IsPreview=true in the project's own PropertyGroup causes Directory.Build.props to add PREVIEW to DefineConstants automatically, and the existing AdditionalProperties="IsPreview=true" on the ProjectReference propagates it to the core SDK build. No manual flag needed from the developer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
|
||
| using System.Runtime.CompilerServices; | ||
|
|
||
| [assembly: InternalsVisibleTo("Microsoft.Azure.Cosmos.AI.Tests, PublicKey=0024000004800000940000000602000000240000525341310004000001000100197c25d0a04f73cb271e8181dba1c0c713df8deebb25864541a66670500f34896d280484b45fe1ff6c29f2ee7aa175d8bcbd0c83cc23901a894a86996030f6292ce6eda6e6f3e6c74b3c5a3ded4903c951e6747e6102969503360f7781bf8bf015058eb89b7621798ccc85aaca036ff1bc1556bb7f62de15908484886aa8bbae")] |
There was a problem hiding this comment.
In the .NET org, they have a task to generate this at build time so that you don't have to look at the PublicKey in source, not important, as this works too, but might be interesting to look at.
| <IsTestProject>true</IsTestProject> | ||
| <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||
| <Platform>AnyCPU</Platform> | ||
| <TargetFramework>net6.0</TargetFramework> |
There was a problem hiding this comment.
.NET 6 is out of support. We should target at least .NET 8 here.
| @@ -0,0 +1,29 @@ | |||
| trigger: none | |||
There was a problem hiding this comment.
Should these just be built as part of the regular SDK pipelines? What the scenario for having separate pipelines?
| /// Dispose the instance when done to release the underlying HTTP connection pool. | ||
| /// </para> | ||
| /// </remarks> | ||
| public sealed class AzureOpenAIEmbeddingGenerator : ICosmosEmbeddingGenerator, IDisposable |
There was a problem hiding this comment.
We don't appear to actually anything useful in Dispose, and IDisposable types are a pain. Let's not make it IDisposable unless we know of a reason to.
|
|
||
| /// <inheritdoc/> | ||
| public async Task<IEnumerable<ReadOnlyMemory<float>>> GenerateEmbeddingsAsync( | ||
| IEnumerable<string> text, |
There was a problem hiding this comment.
| IEnumerable<string> text, | |
| IEnumerable<string> texts, |
nit: It's common for enumerables to be pluralized.
| { | ||
| int count = Math.Min(MaxBatchSize, inputs.Count - offset); | ||
| List<string> chunk = inputs.GetRange(offset, count); | ||
| IReadOnlyList<ReadOnlyMemory<float>> batchResult = await this.InvokeBatchAsync(chunk, cancellationToken); |
There was a problem hiding this comment.
In theory we could parrallelize all of these by not awaiting here, and instead collect the tasks into a list/array and do a await Task.WhenAll(tasks) on the results.
Can be done later if we want.
| /// <summary> | ||
| /// Releases resources held by this instance. | ||
| /// </summary> | ||
| public void Dispose() | ||
| { | ||
| this.disposed = true; | ||
| } | ||
|
|
There was a problem hiding this comment.
| /// <summary> | |
| /// Releases resources held by this instance. | |
| /// </summary> | |
| public void Dispose() | |
| { | |
| this.disposed = true; | |
| } | |
|
|
||
| List<ReadOnlyMemory<float>> results = new List<ReadOnlyMemory<float>>(inputs.Count); | ||
|
|
||
| if (inputs.Count <= MaxBatchSize) |
There was a problem hiding this comment.
In theory you can get rid of this if and just do the loop below, which will only run once.
|
|
||
| private void ValidateInputs(List<string> inputs) | ||
| { | ||
| for (int i = 0; i < inputs.Count; i++) |
There was a problem hiding this comment.
Is doing this loop useful over just getting the wrapped exception from the service?
|
|
||
| private void ThrowIfDisposed() | ||
| { | ||
| if (this.disposed) | ||
| { | ||
| throw new ObjectDisposedException(nameof(AzureOpenAIEmbeddingGenerator)); | ||
| } | ||
| } |
There was a problem hiding this comment.
| private void ThrowIfDisposed() | |
| { | |
| if (this.disposed) | |
| { | |
| throw new ObjectDisposedException(nameof(AzureOpenAIEmbeddingGenerator)); | |
| } | |
| } |
|
Kevin, Thanks for your review. While all the files you reviewed in this PR are still relevant, this PR is still in a WIP state, I should have marked it in WIP. Will keep this posted and reply back once its ready for review . |
Introduces Microsoft.Azure.Cosmos.AI - an optional extension library that will host AI-powered capabilities (embeddings, semantic re-ranking, OCR, model integrations) layered on the v3 SDK. Infrastructure only; no public APIs are exposed yet. The first capability - the Azure OpenAI embedding provider - lands in #5849. Package layout - Microsoft.Azure.Cosmos.AI/src - netstandard2.0 library, delay-signed with the MSSharedLib key, three-build aware via SdkProjectRef: ProjectReference in-source, NuGet pin [3.59.0,) for stable, 3.60.0-preview.0 for preview. - Microsoft.Azure.Cosmos.AI/src/AssemblyInfo.cs - assembly-level attributes (CLSCompliant, ComVisible). - Microsoft.Azure.Cosmos.AI/src/Mirrored/AssemblyKeys.cs - local copy of the public-key constant so InternalsVisibleTo resolves without a cross-project Compile link. - Microsoft.Azure.Cosmos.AI/Directory.Build.props - package-scoped overrides layered above the root props. - Microsoft.Azure.Cosmos.AI/tests/Microsoft.Azure.Cosmos.AI.Tests - MSTest test project skeleton targeting net8.0. Versioning (root Directory.Build.props) - Add CosmosAIOfficialVersion, CosmosAIPreviewVersion, and CosmosAIPreviewSuffixVersion properties so the AI package versions independently from the core SDK. - Add COSMOSAIPREVIEW to DefineConstants alongside PREVIEW and ENCRYPTIONPREVIEW so future preview-gated AI code can #if-guard consistently with the established pattern. Pipelines - azure-pipelines-cosmosai.yml - PR validation pipeline. Triggers on changes under Microsoft.Azure.Cosmos.AI/**, the cosmosai templates, the AI pipelines themselves, and Directory.Build.props (so version bumps to the CosmosAI* properties run AI parity validation). - azure-pipelines-cosmosai-official.yml - release pipeline with its own approval gate and partner-drop blob path, mirroring the existing azure-pipelines-encryption-official.yml layout. - templates/cosmosai-nuget-pack.yml - shared build + pack template. PublishBuildArtifacts is at step-root so CI builds (not just release builds) publish the .nupkg as a downloadable artifact. - templates/static-tools-cosmosai.yml - static-analysis template. Dependencies - Pin Newtonsoft.Json 10.0.2 with PrivateAssets=All to satisfy the core SDK's transitive contract without leaking onto the public dependency graph. No business logic, no public types, no contract surface change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Summary
Implements AzureOpenAIEmbeddingGenerator — the concrete Azure OpenAI-backed implementation of ICosmosEmbeddingGenerator that lives in the Microsoft.Azure.Cosmos.AI package.
Closes #5842.
Dependencies
This PR cherry-picks from three unmerged dependency PRs. It will be rebased / squashed onto main once those land:
Changes
Microsoft.Azure.Cosmos.AI/src/AzureOpenAIEmbeddingGenerator.cs
New sealed class implementing ICosmosEmbeddingGenerator and IDisposable.
Constructors (4 public)
Behavior
Testability
An internal test constructor accepts a Func<IReadOnlyList, int, CancellationToken, Task<IReadOnlyList<ReadOnlyMemory>>> delegate, bypassing the sealed EmbeddingClient entirely. This is consistent with the approach used elsewhere in the SDK for injecting fake network calls.
Microsoft.Azure.Cosmos.AI/tests/.../AzureOpenAIEmbeddingGeneratorTests.cs
25 unit tests covering: