Implement custom Foundry Local models list. #12031

sebastienros · 2025-10-15T01:00:34Z

Description

The model identifiers are different from hosted and local environments, so is the list of available models. This is adding a new sub-section in the current list named FoundryLocal such that user can specifically pick the ones that work locally (assuming their cpu/gpu/npu supports it).

Fixes #11592

Checklist

Is this feature complete?
- Yes. Ready to ship.
- No. Follow-up changes expected.
Are you including unit tests for the changes and scenario tests if relevant?
- Yes
- No
Did you add public API?
- Yes
  - If yes, did you have an API Review for it?
    - Yes
    - No
  - Did you add <remarks /> and <code /> elements on your triple slash comments?
    - Yes
    - No
- No
Does the change make any security assumptions or guarantees?
- Yes
  - If yes, have you done a threat model and had a security review?
    - Yes
    - No
- No
Does the change require an update in our Aspire docs?
- Yes - Update Azure Foundry doc to use the existing and new model constants docs-aspire#5200
- No

github-actions · 2025-10-15T01:00:45Z

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 12031

Or

Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 12031"

Copilot

Pull Request Overview

This PR implements custom Foundry Local model lists to differentiate between hosted and local AI model identifiers in Azure AI Foundry. The change allows users to specifically select models that work in local environments based on CPU/GPU/NPU support.

Key changes:

Adds new FoundryLocal subsection containing models optimized for local inference
Creates code generation tools to fetch and generate local model definitions
Updates playground example to conditionally use local vs hosted models

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
GenModelLocal.cs	New code generation tool that fetches local AI Foundry models from Azure API and generates strongly-typed model definitions
GenModel.cs	Updated hosted model generation with new filter for unified endpoint deployment options
AIFoundryModel.Local.Generated.cs	Generated file containing FoundryLocal class with 18 local model definitions optimized for CUDA GPUs
AIFoundryModel.Generated.cs	Updated hosted models with new additions like DeepSeek-V3.1, Phi variants, GPT models, and Stability AI models
Program.cs	Updated playground to conditionally select local vs hosted models based on emulator status
update-ai-foundry-models.yml	Added local model generation to the GitHub workflow

src/Aspire.Hosting.Azure.AIFoundry/tools/GenModelLocal.cs

src/Aspire.Hosting.Azure.AIFoundry/AIFoundryModel.Local.Generated.cs

playground/AzureAIFoundryEndToEnd/AzureAIFoundryEndToEnd.AppHost/Program.cs

src/Aspire.Hosting.Azure.AIFoundry/AIFoundryModel.Generated.cs

…st/Program.cs Co-authored-by: James Newton-King <[email protected]>

src/Aspire.Hosting.Azure.AIFoundry/tools/GenModel.cs

eerhardt · 2025-10-15T16:10:07Z

src/Aspire.Hosting.Azure.AIFoundry/tools/GenModel.cs

                    {"field": "properties/userProperties/is-promptflow", "operator": "notexists"},
                    {"field": "labels", "operator": "eq", "values": ["latest"]},
-                    {"field": "properties/name", "operator": "eq", "values": ["dall-e-3", "gpt-35-turbo", "gpt-35-turbo-16k", "gpt-4", "gpt-4-32k", "gpt-4o", "gpt-4o-mini", "gpt-4o-audio-preview", "gpt-4.1", "gpt-4.1-mini", "gpt-4.1-nano", "o1", "o3-mini", "o4-mini", "ada", "text-embedding-ada-002", "babbage", "curie", "davinci", "text-embedding-3-small", "text-embedding-3-large", "AI21-Jamba-1.5-Large", "AI21-Jamba-1.5-Mini", "AI21-Jamba-Instruct", "Codestral-2501", "cohere-command-a", "Cohere-command-r", "Cohere-command-r-08-2024", "Cohere-command-r-plus", "Cohere-command-r-plus-08-2024", "Cohere-embed-v3-english", "Cohere-embed-v3-multilingual", "DeepSeek-R1", "DeepSeek-R1-0528", "DeepSeek-V3", "DeepSeek-V3-0324", "embed-v-4-0", "jais-30b-chat", "Llama-3.2-11B-Vision-Instruct", "Llama-3.2-90B-Vision-Instruct", "Llama-3.3-70B-Instruct", "MAI-DS-R1", "Meta-Llama-3-70B-Instruct", "Meta-Llama-3-8B-Instruct", "Meta-Llama-3.1-405B-Instruct", "Meta-Llama-3.1-70B-Instruct", "Meta-Llama-3.1-8B-Instruct", "Ministral-3B", "Mistral-large-2407", "Mistral-Large-2411", "Mistral-Nemo", "Mistral-small", "mistral-small-2503", "Phi-4", "Phi-4-mini-instruct", "Phi-4-reasoning", "Phi-4-mini-reasoning", "Llama-4-Maverick-17B-128E-Instruct-FP8", "Llama-4-Scout-17B-16E-Instruct", "mistral-medium-2505", "mistral-document-ai-2505", "grok-3", "grok-3-mini", "FLUX-1.1-pro", "FLUX.1-Kontext-pro", "gpt-oss-120b", "code-cushman-001", "code-cushman-fine-tune-002", "whisper", "text-ada-001", "text-similarity-ada-001", "text-search-ada-doc-001", "text-search-ada-query-001", "code-search-ada-code-001", "code-search-ada-text-001", "text-babbage-001", "text-similarity-babbage-001", "text-search-babbage-doc-001", "text-search-babbage-query-001", "code-search-babbage-code-001", "code-search-babbage-text-001", "text-curie-001", "text-similarity-curie-001", "text-search-curie-doc-001", "text-search-curie-query-001", "text-davinci-001", "text-davinci-002", "text-davinci-003", "text-davinci-fine-tune-002", "code-davinci-002", "code-davinci-fine-tune-002", "text-similarity-davinci-001", "text-search-davinci-doc-001", "text-search-davinci-query-001", "dall-e-2", "gpt-35-turbo-instruct", "gpt-4o-mini-audio-preview", "o1-mini", "gpt-4o-mini-realtime-preview", "gpt-4o-realtime-preview", "gpt-4o-transcribe", "gpt-4o-mini-transcribe", "gpt-4o-mini-tts", "gpt-5-mini", "gpt-5-nano", "gpt-5-chat", "model-router", "codex-mini", "sora", "tts", "tts-hd", "babbage-002", "davinci-002", "Azure-AI-Speech", "Azure-AI-Language", "Azure-AI-Vision", "Azure-AI-Translator", "Azure-AI-Content-Understanding", "Azure-AI-Document-Intelligence", "Azure-AI-Content-Safety"]}
+                    {"field": "annotations/tags/deploymentOptions", "operator": "contains", "values": ["UnifiedEndpointMaaS"]}


What does this condition mean? What is UnifiedEndpointMaaS?

Instead of filtering by name (because they will change) we use this field which is what I found was the common denominator for all the models we are taking it. "model as a service?"

src/Aspire.Hosting.Azure.AIFoundry/tools/GenModelLocal.cs

src/Aspire.Hosting.Azure.AIFoundry/AIFoundryModel.Local.Generated.cs

…into sebros/localmodel

eerhardt

Thanks!

sebastienros added 2 commits October 14, 2025 09:54

Generate Foundry Local specific models

2cbb1df

Improve logic

6294b35

Copilot AI review requested due to automatic review settings October 15, 2025 01:00

github-actions bot added the area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication label Oct 15, 2025

Copilot AI reviewed Oct 15, 2025

View reviewed changes

sebastienros mentioned this pull request Oct 15, 2025

Update Azure Foundry doc to use the existing and new model constants dotnet/docs-aspire#5200

Open

sebastienros added 2 commits October 14, 2025 18:15

Fix docs

61a1c37

Comment potential problem

d99be1f

sebastienros requested review from davidfowl and eerhardt and removed request for eerhardt October 15, 2025 01:21

JamesNK reviewed Oct 15, 2025

View reviewed changes

radical and others added 2 commits October 15, 2025 00:33

Merge remote-tracking branch 'origin/main' into sebros/localmodel

65c4ffa

Update playground/AzureAIFoundryEndToEnd/AzureAIFoundryEndToEnd.AppHo…

7080cef

…st/Program.cs Co-authored-by: James Newton-King <[email protected]>

davidfowl approved these changes Oct 15, 2025

View reviewed changes

eerhardt reviewed Oct 15, 2025

View reviewed changes

src/Aspire.Hosting.Azure.AIFoundry/tools/GenModel.cs Outdated Show resolved Hide resolved

eerhardt reviewed Oct 15, 2025

View reviewed changes

src/Aspire.Hosting.Azure.AIFoundry/tools/GenModelLocal.cs Outdated Show resolved Hide resolved

eerhardt reviewed Oct 15, 2025

View reviewed changes

src/Aspire.Hosting.Azure.AIFoundry/AIFoundryModel.Local.Generated.cs Outdated Show resolved Hide resolved

sebastienros added 8 commits October 15, 2025 09:17

Improve naming

cb4e9b9

Merge branch 'sebros/localmodel' of https://github.com/dotnet/aspire …

7c58457

…into sebros/localmodel

Feedback

e3a798a

Fix model assignment for emulator condition

7e17d01

Fix class name

0627847

Fix new lines

6943a64

Add obsoleted models

0800fc7

Removed more new lines

b0fef2c

eerhardt approved these changes Oct 15, 2025

View reviewed changes

sebastienros merged commit 36c145d into main Oct 16, 2025
330 checks passed

dotnet-policy-service bot added this to the 13.0 milestone Oct 16, 2025

sebastienros deleted the sebros/localmodel branch October 16, 2025 00:10

Copilot AI mentioned this pull request Oct 17, 2025

Update Azure AI Foundry and GitHub Models documentation to use strongly-typed model constants dotnet/docs-aspire#5302

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Implement custom Foundry Local models list. #12031

Implement custom Foundry Local models list. #12031

Uh oh!

sebastienros commented Oct 15, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Oct 15, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eerhardt Oct 15, 2025

Uh oh!

sebastienros Oct 15, 2025

Uh oh!

Uh oh!

Uh oh!

eerhardt left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Implement custom Foundry Local models list. #12031

Implement custom Foundry Local models list. #12031

Uh oh!

Conversation

sebastienros commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

github-actions bot commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eerhardt Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

sebastienros Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

eerhardt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sebastienros commented Oct 15, 2025 •

edited

Loading

github-actions bot commented Oct 15, 2025 •

edited

Loading