[dotnet-ai] ONNX Runtime + local LLM inference

[luisquintanilla]

Parent Issue

Part of the dotnet-ai plugin proposal: #225

Summary

Add the onnx-runtime-inference and local-llm-inference skills to the dotnet-ai plugin, including one reference file.

These two skills cover model execution — running pre-trained models and LLMs locally. They are submitted together because they share the deployment-at-the-edge theme and reference each other in their routing guidance.

Skills in This Issue

onnx-runtime-inference

Guides developers through running pre-trained ONNX models for inference in .NET, both standalone (via Microsoft.ML.OnnxRuntime) and through ML.NET (via Microsoft.ML.OnnxTransformer).

Covers:

• Standalone approach — InferenceSession, input/output tensor construction, execution
• ML.NET approach — ApplyOnnxModel() within an ML.NET pipeline
• Execution providers — CPU (default), CUDA GPU, DirectML, CoreML, selecting and configuring providers
• Model sources — ONNX Model Zoo, Hugging Face ONNX exports, custom Python→ONNX exports
• Tensor I/O — references/tensors.md for shape management, named inputs/outputs, batch inference

Does NOT cover:

• Training models from scratch (use mlnet for classical ML and supported deep learning; TorchSharp for custom architectures)
• LLM text generation (use meai-chat-integration or local-llm-inference)

local-llm-inference

Guides developers through running LLMs locally via Ollama and Foundry Local, both surfacing models through MEAI's IChatClient abstraction.

Covers:

• Ollama — installation, pulling models, connecting via OllamaSharp as IChatClient, model selection
• Foundry Local — installation, Microsoft.AI.Foundry.Local SDK, launching models, connecting as IChatClient
• Provider-agnostic patterns — both produce IChatClient, so all MEAI middleware (streaming, function calling, caching) works identically
• Model selection — guidance on choosing models by task (chat, code, small/large)

Does NOT cover:

• Cloud-hosted LLMs (use meai-chat-integration)
• Non-LLM models (use onnx-runtime-inference)
• Classical ML tasks (use mlnet)

Reference Files

File	Purpose
references/tensors.md	Tensor I/O patterns for ONNX Runtime — shape management, named inputs/outputs, batch processing, memory management with OrtValue

Files

plugins/dotnet-ai/skills/onnx-runtime-inference/
  SKILL.md
  references/
    tensors.md
plugins/dotnet-ai/skills/local-llm-inference/SKILL.md
tests/dotnet-ai/onnx-runtime-inference/eval.yaml
tests/dotnet-ai/local-llm-inference/eval.yaml

Eval Scenarios

onnx-runtime-inference:

1. Run ONNX model for inference — prompt asks to run a pre-trained image classification model; expects InferenceSession or ApplyOnnxModel with execution provider selection

local-llm-inference:

1. Set up Ollama for local development — prompt asks to run an LLM locally without API keys; expects Ollama setup with IChatClient
2. Reject local inference for production cloud — prompt asks to use local inference for a cloud-deployed production app; expects redirect to cloud providers

Key Packages

• Microsoft.ML.OnnxRuntime (standalone ONNX)
• Microsoft.ML.OnnxRuntime.Gpu (CUDA)
• Microsoft.ML.OnnxRuntime.DirectML (DirectML)
• Microsoft.ML.OnnxTransformer (ML.NET integration)
• OllamaSharp (Ollama IChatClient)
• Microsoft.AI.Foundry.Local (Foundry Local)

Dependencies

Requires the plugin scaffold (sub-issue #1) to be merged first.

Acceptance Criteria

• onnx-runtime-inference covers both standalone and ML.NET approaches
• Execution providers (CPU, CUDA, DirectML) are documented
• tensors.md reference provides tensor shape and memory management guidance
• local-llm-inference presents Ollama and Foundry Local neutrally (no prioritization)
• Both skills surface models through IChatClient abstraction
• eval.yaml files have scenarios for each skill