Skip to content

[WIP -DO NOT REVIEW] AI: Adds AzureOpenAIEmbeddingGenerator provider for issue #5842#5849

Open
ananth7592 wants to merge 10 commits into
mainfrom
users/amudumba/5842-azureopenai-embedding-generator
Open

[WIP -DO NOT REVIEW] AI: Adds AzureOpenAIEmbeddingGenerator provider for issue #5842#5849
ananth7592 wants to merge 10 commits into
mainfrom
users/amudumba/5842-azureopenai-embedding-generator

Conversation

@ananth7592
Copy link
Copy Markdown
Member

Summary

Implements AzureOpenAIEmbeddingGenerator — the concrete Azure OpenAI-backed implementation of ICosmosEmbeddingGenerator that lives in the Microsoft.Azure.Cosmos.AI package.

Closes #5842.

Dependencies

This PR cherry-picks from three unmerged dependency PRs. It will be rebased / squashed onto main once those land:

Changes

Microsoft.Azure.Cosmos.AI/src/AzureOpenAIEmbeddingGenerator.cs

New sealed class implementing ICosmosEmbeddingGenerator and IDisposable.

Constructors (4 public)

  • (EmbeddingSource source, int dimensions, string apiKey) — builds from the VectorEmbeddingPolicy's source block + API key
  • (EmbeddingSource source, int dimensions, TokenCredential credential) — same but Entra auth
  • (string endpoint, string deploymentName, int dimensions, string apiKey) — explicit params + API key
  • (string endpoint, string deploymentName, int dimensions, TokenCredential credential) — explicit params + Entra auth

Behavior

  • Batches inputs at the Azure OpenAI hard limit of 2048 per call; results concatenated in original order
  • Creates a fresh EmbeddingGenerationOptions { Dimensions = ... } per batch call (the Azure SDK may mutate options internally during concurrent serialization)
  • Wraps RequestFailedException → CosmosException(503) with endpoint/deployment in the message
  • Validates inputs up-front: null collection, null elements, empty or whitespace-only elements each throw ArgumentException
  • Thread-safe: the underlying EmbeddingClient is stateless for concurrent use

Testability

An internal test constructor accepts a Func<IReadOnlyList, int, CancellationToken, Task<IReadOnlyList<ReadOnlyMemory>>> delegate, bypassing the sealed EmbeddingClient entirely. This is consistent with the approach used elsewhere in the SDK for injecting fake network calls.

Microsoft.Azure.Cosmos.AI/tests/.../AzureOpenAIEmbeddingGeneratorTests.cs

25 unit tests covering:

  • Constructor validation (null/empty/whitespace endpoint, deployment, key; zero/negative dimensions; null/invalid EmbeddingSource)
  • GenerateEmbeddingsAsync input validation (null, empty collection, null elements, whitespace/empty elements)
  • Happy path (single batch, correct values, correct order)
  • Dimensions forwarded correctly to the batch func
  • CancellationToken forwarded correctly
  • Batching: 5000 inputs → 3 calls (2048 + 2048 + 904); exact 2048 → 1 call
  • Multi-batch order preservation (2050 inputs, numeric marker in each vector)
  • Cancellation propagation (TaskCanceledException)
  • Exception propagation from the batch func
  • ObjectDisposedException after Dispose()
  • Double-dispose is safe

aavasthy and others added 8 commits May 8, 2026 12:49
…on (preview)

Adds the public surface for the V0 embedding-generation feature
(parent #5830, this PR addresses #5831):

* New `Microsoft.Azure.Cosmos.IEmbeddingGenerator` interface with a
  single batched `GenerateEmbeddingsAsync(IEnumerable<string>,
  CancellationToken)` method. The cancellation token is included now
  to avoid a breaking signature change later.
* New `QueryRequestOptions.EmbeddingGenerator` property (defaults
  to null, so existing behavior is unchanged).

Both are gated on `#if PREVIEW` (public in preview, internal in GA),
mirroring the `ReadConsistencyStrategy` pattern.

Tests:
* QueryRequestOptionsUniTests: added EmbeddingGenerator_DefaultsToNull
  and EmbeddingGenerator_RoundTripsAssignedInstance.
* Regenerated DotNetPreviewSDKAPI.net6.json baseline. The actual
  surface additions are limited to IEmbeddingGenerator + the new
  property; the rest of the diff is JSON key reordering by the regen
  tool. PreviewContractChanges and the standard ContractChanges suite
  pass.

This PR is intentionally surface-only - no behavior is wired up yet.
The DTO/feature-flag work, resolver, plumbing into the pipeline,
diagnostics datum, and broader tests are tracked in #5832-#5837.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Rename IEmbeddingGenerator → ICosmosEmbeddingGenerator to avoid
  collision with Microsoft.Extensions.AI.IEmbeddingGenerator
- Change GenerateEmbeddingsAsync return type double → float to align
  with SDK's own VectorEmbedding<float> vector data model
- Add [JsonIgnore] on CosmosClientOptions.EmbeddingGenerator to prevent
  diagnostic serialization crash
- Add null guard in CosmosClientBuilder.WithEmbeddingGenerator:
  embeddingGenerator ?? throw new ArgumentNullException(...)
- Strengthen thread-safety documentation: implementations MUST be safe
  to invoke concurrently (parallel queries/partition reads on one instance)
- Document per-request opt-out behavior as intentional: null falls through
  to client-level; callers needing opt-out must provide a no-op implementation
- Regenerate DotNetPreviewSDKAPI.net6.json contract

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- New Microsoft.Azure.Cosmos.Embedding package analogous to Microsoft.Azure.Cosmos.Encryption
- Adds src project (netstandard2.0, Azure.AI.OpenAI 2.1.0, SdkProjectRef pattern, SNK signing)
- Adds empty tests project (net6.0, MSTest, testkey.snk)
- Adds version properties to root Directory.Build.props (EmbeddingOfficialVersion 1.0.0)
- Adds Embedding solution folder + both projects to Microsoft.Azure.Cosmos.sln

Part of: #5841

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…icrosoft.Azure.Cosmos.AI

Design pivot: The optional provider package is renamed to Microsoft.Azure.Cosmos.AI
to reflect its broader scope beyond embeddings (semantic re-ranking, OCR, LLM/SLM
integrations). Uppercase AI matches Microsoft.Extensions.AI and Azure.AI.* conventions.

- Rename folder Microsoft.Azure.Cosmos.Embedding -> Microsoft.Azure.Cosmos.AI
- Rename AssemblyName, RootNamespace, PackageId to Microsoft.Azure.Cosmos.AI
- Rename MSBuild version properties from Embedding* to CosmosAI*
- Update Description and PackageTags to reflect broader AI scope
- Update solution file (solution folder, project entries, config platforms)
- Update InternalsVisibleTo to reference Microsoft.Azure.Cosmos.AI.Tests

Part of: #5841

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…osoft.Azure.Cosmos.AI

Adds the official NuGet publishing pipeline for the Microsoft.Azure.Cosmos.AI package,
mirroring the pattern used by Microsoft.Azure.Cosmos.Encryption:

- azure-pipelines-cosmosai-official.yml: Top-level pipeline with Release Gates stage
  (static analysis) and Build/Pack/Publish stage. Triggered manually (trigger: none).
- templates/static-tools-cosmosai.yml: Static analysis job with three build variants:
  NuGet SDK reference, SdkProjectRef=true, and parity check (SdkProjectRef + no symbol).
  Includes BinSkim, CredScan, PoliCheck, AntiMalware, Component Governance Detection.
- templates/cosmosai-nuget-pack.yml: Build, pack, symbols pack, SBOM manifest,
  Azure blob copy (cosmosdb/csharp/cosmosai/\), and artifact publish.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ed by Cosmos SDK

Microsoft.Azure.Cosmos enforces that consumers explicitly reference
Newtonsoft.Json >= 10.0.2. Added it with PrivateAssets=All so the
check is satisfied without making it a transitive dependency for
Microsoft.Azure.Cosmos.AI consumers (same pattern as Encryption).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implements ICosmosEmbeddingGenerator backed by Azure OpenAI's EmbeddingClient:

- 4 public constructors: (EmbeddingSource+apiKey), (EmbeddingSource+TokenCredential),
  (endpoint+deploymentName+dimensions+apiKey), (endpoint+deploymentName+dimensions+TokenCredential)
- Batching: splits large inputs at the Azure OpenAI hard limit of 2048 per call;
  results are concatenated in original order
- Error wrapping: RequestFailedException → CosmosException(503) with endpoint/deployment context
- Input validation: null, empty, and whitespace-only elements rejected up-front
- Thread-safe: fresh EmbeddingGenerationOptions per batch call (avoids SDK mutex contention)
- Testability: internal test constructor accepts a Func delegate to avoid mocking sealed SDK types
- 25 unit tests covering construction, validation, batching, order preservation,
  cancellation, error propagation, and dispose semantics

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good!

ananth7592 and others added 2 commits May 8, 2026 13:21
…ypes without requiring IsPreview globally

- Add AdditionalProperties="IsPreview=true" to the ProjectReference so the core
  SDK always exposes ICosmosEmbeddingGenerator and EmbeddingSource as public when
  building from source (SdkProjectRef=true), without the caller needing to pass
  /p:IsPreview=true on the dotnet CLI.
- Remove the broken non-preview NuGet condition ([3.59.0,)) — the AI package
  requires ICosmosEmbeddingGenerator which is #if PREVIEW-gated and therefore
  always internal in a non-preview SDK build. The package always targets the
  preview NuGet path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…uild work without flags

The Cosmos.AI package depends on ICosmosEmbeddingGenerator and EmbeddingSource,
which are #if PREVIEW-gated in the core SDK. Previously, building from Visual Studio
or running 'dotnet build /p:SdkProjectRef=true' without also passing /p:IsPreview=true
would fail to resolve those types because PREVIEW was not defined.

Setting IsPreview=true in the project's own PropertyGroup causes Directory.Build.props
to add PREVIEW to DefineConstants automatically, and the existing
AdditionalProperties="IsPreview=true" on the ProjectReference propagates it to
the core SDK build. No manual flag needed from the developer.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

using System.Runtime.CompilerServices;

[assembly: InternalsVisibleTo("Microsoft.Azure.Cosmos.AI.Tests, PublicKey=0024000004800000940000000602000000240000525341310004000001000100197c25d0a04f73cb271e8181dba1c0c713df8deebb25864541a66670500f34896d280484b45fe1ff6c29f2ee7aa175d8bcbd0c83cc23901a894a86996030f6292ce6eda6e6f3e6c74b3c5a3ded4903c951e6747e6102969503360f7781bf8bf015058eb89b7621798ccc85aaca036ff1bc1556bb7f62de15908484886aa8bbae")]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the .NET org, they have a task to generate this at build time so that you don't have to look at the PublicKey in source, not important, as this works too, but might be interesting to look at.

<IsTestProject>true</IsTestProject>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Platform>AnyCPU</Platform>
<TargetFramework>net6.0</TargetFramework>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.NET 6 is out of support. We should target at least .NET 8 here.

@@ -0,0 +1,29 @@
trigger: none
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these just be built as part of the regular SDK pipelines? What the scenario for having separate pipelines?

/// Dispose the instance when done to release the underlying HTTP connection pool.
/// </para>
/// </remarks>
public sealed class AzureOpenAIEmbeddingGenerator : ICosmosEmbeddingGenerator, IDisposable
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't appear to actually anything useful in Dispose, and IDisposable types are a pain. Let's not make it IDisposable unless we know of a reason to.


/// <inheritdoc/>
public async Task<IEnumerable<ReadOnlyMemory<float>>> GenerateEmbeddingsAsync(
IEnumerable<string> text,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
IEnumerable<string> text,
IEnumerable<string> texts,

nit: It's common for enumerables to be pluralized.

{
int count = Math.Min(MaxBatchSize, inputs.Count - offset);
List<string> chunk = inputs.GetRange(offset, count);
IReadOnlyList<ReadOnlyMemory<float>> batchResult = await this.InvokeBatchAsync(chunk, cancellationToken);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory we could parrallelize all of these by not awaiting here, and instead collect the tasks into a list/array and do a await Task.WhenAll(tasks) on the results.

Can be done later if we want.

Comment on lines +253 to +260
/// <summary>
/// Releases resources held by this instance.
/// </summary>
public void Dispose()
{
this.disposed = true;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// <summary>
/// Releases resources held by this instance.
/// </summary>
public void Dispose()
{
this.disposed = true;
}


List<ReadOnlyMemory<float>> results = new List<ReadOnlyMemory<float>>(inputs.Count);

if (inputs.Count <= MaxBatchSize)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory you can get rid of this if and just do the loop below, which will only run once.


private void ValidateInputs(List<string> inputs)
{
for (int i = 0; i < inputs.Count; i++)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is doing this loop useful over just getting the wrapped exception from the service?

Comment on lines +364 to +371

private void ThrowIfDisposed()
{
if (this.disposed)
{
throw new ObjectDisposedException(nameof(AzureOpenAIEmbeddingGenerator));
}
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private void ThrowIfDisposed()
{
if (this.disposed)
{
throw new ObjectDisposedException(nameof(AzureOpenAIEmbeddingGenerator));
}
}

@ananth7592
Copy link
Copy Markdown
Member Author

Kevin, Thanks for your review. While all the files you reviewed in this PR are still relevant, this PR is still in a WIP state, I should have marked it in WIP. Will keep this posted and reply back once its ready for review .

@ananth7592 ananth7592 changed the title AI: Adds AzureOpenAIEmbeddingGenerator provider for issue #5842 [WIP -DO NOT REVIEW] AI: Adds AzureOpenAIEmbeddingGenerator provider for issue #5842 May 12, 2026
ananth7592 added a commit that referenced this pull request May 19, 2026
Introduces Microsoft.Azure.Cosmos.AI - an optional extension library
that will host AI-powered capabilities (embeddings, semantic re-ranking,
OCR, model integrations) layered on the v3 SDK. Infrastructure only;
no public APIs are exposed yet. The first capability - the Azure OpenAI
embedding provider - lands in #5849.

Package layout
- Microsoft.Azure.Cosmos.AI/src - netstandard2.0 library, delay-signed
  with the MSSharedLib key, three-build aware via SdkProjectRef:
  ProjectReference in-source, NuGet pin [3.59.0,) for stable,
  3.60.0-preview.0 for preview.
- Microsoft.Azure.Cosmos.AI/src/AssemblyInfo.cs - assembly-level
  attributes (CLSCompliant, ComVisible).
- Microsoft.Azure.Cosmos.AI/src/Mirrored/AssemblyKeys.cs - local copy of
  the public-key constant so InternalsVisibleTo resolves without a
  cross-project Compile link.
- Microsoft.Azure.Cosmos.AI/Directory.Build.props - package-scoped
  overrides layered above the root props.
- Microsoft.Azure.Cosmos.AI/tests/Microsoft.Azure.Cosmos.AI.Tests -
  MSTest test project skeleton targeting net8.0.

Versioning (root Directory.Build.props)
- Add CosmosAIOfficialVersion, CosmosAIPreviewVersion, and
  CosmosAIPreviewSuffixVersion properties so the AI package versions
  independently from the core SDK.
- Add COSMOSAIPREVIEW to DefineConstants alongside PREVIEW and
  ENCRYPTIONPREVIEW so future preview-gated AI code can #if-guard
  consistently with the established pattern.

Pipelines
- azure-pipelines-cosmosai.yml - PR validation pipeline. Triggers on
  changes under Microsoft.Azure.Cosmos.AI/**, the cosmosai templates,
  the AI pipelines themselves, and Directory.Build.props (so version
  bumps to the CosmosAI* properties run AI parity validation).
- azure-pipelines-cosmosai-official.yml - release pipeline with its
  own approval gate and partner-drop blob path, mirroring the existing
  azure-pipelines-encryption-official.yml layout.
- templates/cosmosai-nuget-pack.yml - shared build + pack template.
  PublishBuildArtifacts is at step-root so CI builds (not just release
  builds) publish the .nupkg as a downloadable artifact.
- templates/static-tools-cosmosai.yml - static-analysis template.

Dependencies
- Pin Newtonsoft.Json 10.0.2 with PrivateAssets=All to satisfy the
  core SDK's transitive contract without leaking onto the public
  dependency graph.

No business logic, no public types, no contract surface change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Embedding V0] AzureOpenAIEmbeddingGenerator provider implementation

3 participants