Windows & .NET

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page was written on 2026-09-30 against Windows ML as documented on Microsoft Learn that day (the 2.x and 1.8.x lines of the Microsoft.WindowsAppSDK.ML package), Microsoft.ML.OnnxRuntime 1.30.0, Microsoft.ML.OnnxRuntimeGenAI 0.17.1 (a 0.x preview line), Microsoft.Extensions.AI.Abstractions 10.10.1 and Whisper.net 1.9.1 (NuGet, as checked that day), with .NET 8 or later. Windows is the one platform in this sub-section where the operating system itself ships an inference runtime, a set of built-in models and a model server, so the choice is less "which library" and more "which layer". This page walks through the layers from the lowest (an ONNX file and InferenceSession) to the highest (an IChatClient that does not care where the model runs), and continues the DocsAssistant goes offline scenario of this sub-section: transcribe a spoken question, answer it with a small local model, speak the answer, and classify or describe a screenshot. A dedicated Windows Apps section of this site is planned but does not exist yet, so packaging, WinUI and MSIX topics are not covered here. Model formats are in Model Formats; producing ONNX files is in Conversion & Quantization; the models themselves are in Small LLMs & Embeddings, Speech Models: STT & TTS and Vision Models. The same ONNX Runtime ideas on the server side are in Python Backends and Java & Spring Boot. No benchmark numbers or prices are quoted.

The Windows on-device stack

Five Microsoft components overlap, and most confusion comes from mixing them up. Windows ML is the runtime. The Windows AI APIs are finished features built on Microsoft’s own models. Foundry Local is a model catalogue plus a local server. ONNX Runtime GenAI is the generation loop for LLM-style ONNX models. Microsoft.Extensions.AI is only an interface layer, it runs nothing itself.

flowchart TB APP["C# app
.NET 8 or later"] --> MEAI["Microsoft.Extensions.AI
IChatClient, IEmbeddingGenerator"] APP --> WAI["Windows AI APIs
Phi Silica, TextRecognizer"] APP --> ORT["Microsoft.ML.OnnxRuntime
InferenceSession"] MEAI -->|"OpenAI-compatible HTTP"| FL["Foundry Local
local server and catalogue"] MEAI -->|"Ollama API"| OL["Ollama"] FL --> GENAI["ONNX Runtime GenAI
generation loop"] GENAI --> WML["Windows ML
system ONNX Runtime"] ORT --> WML WAI --> WML WML --> EP{"Execution providers
downloaded on demand"} EP --> NPU["NPU
Copilot+ PC"] EP --> GPU["GPU"] EP --> CPU["CPU fallback"] HUB[("Hugging Face Hub
ONNX weights")] -.->|"download once"| FL HUB -.->|"download once"| ORT
Figure 1. DocsAssistant on Windows with no cloud dependency
Table 1. Which Windows option fits which DocsAssistant task
Task Preferred Windows option Alternative Main cost

Transcribe a spoken question

Whisper through ONNX Runtime GenAI or Foundry Local, or Whisper.net (ggml weights)

An ONNX Whisper run by hand with InferenceSession

Audio decoding to 16 kHz mono and model-specific input handling

Answer it

Foundry Local behind IChatClient

Phi Silica (Windows AI APIs), or ORT GenAI in process

Model download size; Phi Silica needs a Limited Access Feature unlock and hardware support

Speak the answer

Windows.Media.SpeechSynthesis.SpeechSynthesizer with an installed system voice

A Hub TTS such as Kokoro exported to ONNX

System voices are not Hub models; Hub TTS needs phonemization written or bundled

Classify or describe a screenshot

A Hub classifier exported to ONNX and run with Windows ML, or a vision chat model on Foundry Local

TextRecognizer (OCR) when the text in the screenshot is what matters

Preprocessing must match the model card; OCR needs an NPU

Retrieve passages (embeddings)

IEmbeddingGenerator over a local server

An embedding model in ONNX run in process

A vector size that must match the index

Windows ML: the system ONNX Runtime

Windows ML is Microsoft’s name for a Windows-supported and maintained copy of ONNX Runtime that an app can share system-wide or carry itself. Its code is the normal ONNX Runtime C# API, so existing Microsoft.ML.OnnxRuntime code keeps working. What Windows ML adds is distribution: the optional execution providers (EPs) for NPUs and specific GPUs are installed and kept up to date by Windows, not bundled in your app, and the app asks the ExecutionProviderCatalog for the ones that fit the machine.

The documented requirements on 2026-09-30 are a Windows version that the Windows App SDK supports, x64 or ARM64, .NET 8 or later for the full API, and a Windows 10 specific target framework moniker such as net8.0-windows10.0.17763.0 or later. CPU (and GPU through the in-box DirectML provider) works on all supported versions, while the downloadable hardware-tuned providers need Windows 11 24H2 (build 26100) or later. There are two deployment modes: self-contained (the runtime DLLs are bundled with the app, which the docs size at roughly 41 MB and which does not auto-update) and framework-dependent (the runtime is shared and serviced with the Windows App SDK).

Table 2. Choosing a Windows ML deployment mode
Aspect Self-contained Framework-dependent

NuGet packages

Microsoft.Windows.AI.MachineLearning or Microsoft.WindowsAppSDK.ML, without the Windows App SDK runtime package

Microsoft.WindowsAppSDK.ML plus Microsoft.WindowsAppSDK.Runtime

App size

Larger, runtime bundled

Smaller, runtime shared

Updates

You ship a new build

Delivered through Windows App SDK servicing

Best for

Strict version control

Small installers, Store distribution

The vendor providers are listed per silicon class: NPUs use OpenVINO (Intel), QNN (Qualcomm) or VitisAI (AMD), while GPUs use MIGraphX (AMD), NvTensorRtRtx (NVIDIA), OpenVINO or QNN, with the CPU provider and the legacy DirectML provider in the box. A Copilot+ PC is the Windows PC class with a capable NPU, and it is where NPU providers deliver their battery benefit; the Windows 11 24H2 floor above still applies. Each provider lists minimum driver versions, so a "works on my machine" result with a new driver does not transfer to an older fleet.

Program.cs: install and register every compatible execution provider, then open a session
using Microsoft.ML.OnnxRuntime;
using Microsoft.Windows.AI.MachineLearning;

// Downloads missing Windows-certified EPs (first run can take a while), then
// registers every present EP with the ONNX Runtime that ships with Windows ML.
var catalog = ExecutionProviderCatalog.GetDefault();
await catalog.EnsureAndRegisterCertifiedAsync();

// See which devices ONNX Runtime now knows about (CPU, GPU, NPU ...).
foreach (OrtEpDevice d in OrtEnv.Instance().GetEpDevices())
{
    Console.WriteLine($"{d.EpName} ({d.HardwareDevice.Type})");
}

// Let ONNX Runtime pick: MAX_EFFICIENCY typically prefers an NPU with a CPU fallback.
var options = new SessionOptions();
options.SetEpSelectionPolicy(ExecutionProviderDevicePolicy.MAX_EFFICIENCY);

using var session = new InferenceSession("screenshot-classifier.onnx", options);

Official pages: Install Windows ML execution providers and Select execution providers.

The list of EP devices can change at run time when a provider or a driver updates, so the docs recommend starting with explicit selection through GetEpDevices and AppendExecutionProvider, and only then trying device policies. They also recommend pre-compiling a model for the chosen provider once and caching the result, because a cold compile can take minutes; the C# API for that is OrtModelCompilationOptions with SetInputModelPath, SetOutputModelPath and CompileModel.

The winml CLI

The Windows ML CLI (winml) is a command-line tool that turns a source model, for example one from the Hugging Face Hub, into a hardware-optimized artifact for Windows ML. Microsoft Learn names the primitive commands export, analyze, optimize, quantize and compile, and a winml build command that generates a whole workflow. It targets AMD, Intel, NVIDIA and Qualcomm hardware, offers a curated catalogue of supported models, and can also analyze and optimize an ONNX file you already have, which makes it usable in CI. Install instructions, flags and the supported-model list live on the CLI’s own documentation site and are not repeated here, because the tool is young and its options are the part most likely to change. The first step of any Hub-to-Windows path is still the one described in Conversion & Quantization: get a correct, quantized ONNX file.

A Hub-to-Windows-ML path, using only commands named on the official page
export     a Hub model to ONNX          (winml export ...)
analyze    operator and shape problems  (winml analyze ...)
optimize   graph optimizations          (winml optimize ...)
quantize   reduce size                  (winml quantize ...)
compile    per execution provider       (winml compile ...)
or, in one go:                             (winml build ...)

Official pages: Windows ML CLI and the Windows ML overview.

Windows AI APIs: Phi Silica, text recognition and image APIs

The Windows AI APIs ship with the Windows App SDK and expose models that Microsoft built and optimized, so an app does not find, convert or package a model. They live in the Microsoft.Windows.AI namespaces (Microsoft.Windows.AI.Text for language models, Microsoft.Windows.AI.Imaging for vision features). Every feature follows the same readiness pattern: check GetReadyState(), call EnsureReadyAsync() if the model is not present, then create the object.

Phi Silica and the Limited Access Feature

Phi Silica is a local language model for chat-style generation, summarization and rewriting. It runs on the NPU of Copilot+ PCs, and Microsoft Learn also documents a GPU path on supported NVIDIA and AMD cards that needs Developer Mode, a recent driver installed from the GPU vendor and a Windows Insider build, and downloads the model in the background the first time EnsureReadyAsync is called. The Phi Silica APIs are a Limited Access Feature: the app must request an unlock token through Microsoft’s request form and unlock the feature with the LimitedAccessFeatures class before the calls succeed. The docs also say the features are not available in China.

Phi Silica: ensure the model, then generate the answer to a question
using Microsoft.Windows.AI;
using Microsoft.Windows.AI.Text;

if (LanguageModel.GetReadyState() == AIFeatureReadyState.NotReady)
{
    await LanguageModel.EnsureReadyAsync();
}

using LanguageModel languageModel = await LanguageModel.CreateAsync();

var result = await languageModel.GenerateResponseAsync(
    "Answer briefly: how do I load a model once per process in FastAPI?");

Console.WriteLine(result.Text);

Official page: Get started with Phi Silica in the Windows App SDK.

The Phi Silica page currently carries a banner saying that Phi Silica is being replaced by a new on-device model named Aion Instruct, that Aion Instruct does not need a Limited Access Feature token, and giving a schedule: a sideloadable test package in early October 2026, a Windows Insider rollout in October 2026 and retail devices in November 2026, when Phi Silica is removed. That is a vendor announcement on the official page, not something this page could test; because dates in such rollouts move, treat them as planning information and re-read the banner and the Windows AI APIs overview before building on Phi Silica. Code written against LanguageModel today may need changes, so isolate it behind your own interface (for example an IChatClient wrapper).

Text recognition (OCR)

TextRecognizer detects characters, words, lines and word bounding polygons with a confidence per match. The docs state these APIs run only on devices with an NPU, and describe them as faster and more accurate than the legacy Windows.Media.Ocr.OcrEngine. In the DocsAssistant scenario it answers a narrower question than a vision model: not "what does this screenshot show" but "what does it say".

TextRecognizer: read the text in a screenshot
using Microsoft.Graphics.Imaging;
using Microsoft.Windows.AI;
using Microsoft.Windows.AI.Imaging;
using Windows.Graphics.Imaging;

static async Task<string> ReadScreenshotAsync(SoftwareBitmap bitmap)
{
    if (TextRecognizer.GetReadyState() == AIFeatureReadyState.NotReady)
    {
        var load = await TextRecognizer.EnsureReadyAsync();
        if (load.Status != AIFeatureReadyResultState.Success)
            throw new InvalidOperationException(load.ExtendedError.Message);
    }

    TextRecognizer recognizer = await TextRecognizer.CreateAsync();
    ImageBuffer buffer = ImageBuffer.CreateForSoftwareBitmap(bitmap);
    RecognizedText text = recognizer.RecognizeTextFromImage(buffer);

    return string.Join(Environment.NewLine, text.Lines.Select(l => l.Text));
}

Official page: Get Started with Text Recognition (OCR) in the Windows App SDK.

The same namespace also holds image features (for example image description and scaling) that follow the same ready-state pattern; their availability and hardware requirements differ per feature, so check the Windows AI APIs overview for the current list instead of assuming that one feature’s requirements apply to another.

Foundry Local

Foundry Local is Microsoft’s on-device runtime and model catalogue. It has three faces: a CLI named foundry, a native SDK (NuGet packages Microsoft.AI.Foundry.Local for cross-platform use and Microsoft.AI.Foundry.Local.WinML for Windows, where the package integrates with Windows ML and exposes the same API with broader hardware acceleration), and an OpenAI-compatible local server. On first use it downloads the execution providers for the machine, then a model from its catalogue; the catalogue entries are addressed by an alias (which picks the variant that best fits the hardware, such as a CPU, GPU or NPU build) or by an exact model ID. The CLI is documented as a preview, so command names can change.

Installing and trying Foundry Local from a terminal
winget install Microsoft.FoundryLocal
foundry --version

# Catalogue, optionally filtered by device or task.
foundry model list
foundry model list --type chat --device npu

# Interactive chat, or run a model (routes to chat or transcription).
foundry chat qwen2.5-0.5b
foundry run qwen2.5-0.5b

# The OpenAI-compatible service: start it and read the URL it reports.
foundry server start
foundry server status

Official page: Foundry Local CLI reference.

One correction to a common shorthand: the model subcommands documented there are list, info, download, load and unload, while running a model is the top-level foundry run <model> (or foundry chat <model>), not foundry model run. Foundry Local assigns a dynamic port each time the server starts unless --port is given, so read the endpoint from foundry server status rather than hard-coding it. The local service has no authentication of its own by design, which is fine on loopback and not something to expose on a network interface.

The SDK route needs no separate server process. The quickstart shape is: create the manager, download and register execution providers, find a model in the catalogue by alias, download and load it, then ask it for a chat client.

Program.cs: Foundry Local SDK quickstart (native chat client, streaming)
using Microsoft.AI.Foundry.Local;
using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;

var config = new Configuration { AppName = "docs_assistant" };
await FoundryLocalManager.CreateAsync(config, Utils.GetAppLogger());
var mgr = FoundryLocalManager.Instance;

await mgr.DownloadAndRegisterEpsAsync((ep, percent) =>
    Console.Write($"\r{ep}: {percent:F1}%"));

var catalog = await mgr.GetCatalogAsync();
var model = await catalog.GetModelAsync("qwen2.5-0.5b")
    ?? throw new InvalidOperationException("Model not found");

await model.DownloadAsync(p => Console.Write($"\rmodel: {p:F1}%"));
await model.LoadAsync();

var chat = await model.GetChatClientAsync();
List<ChatMessage> messages =
[
    new ChatMessage { Role = "user", Content = "How do I stream tokens from an IChatClient?" }
];

await foreach (var chunk in chat.CompleteChatStreamingAsync(messages, CancellationToken.None))
{
    Console.Write(chunk.Choices[0].Message.Content);
}

await model.UnloadAsync();

Official page: Get started with Foundry Local. The snippet follows its C# tab (which uses Utils.GetAppLogger() from the sample repository’s helper project); a model alias that is in the catalogue today may be renamed, so list the catalogue and pick an alias that exists. The documented model aliases cover chat, speech and embedding tasks (foundry model list --type accepts chat, speech and embedding), which is what makes one local runtime usable for several steps of the DocsAssistant pipeline.

ONNX Runtime GenAI

ONNX Runtime GenAI is the generation loop around ONNX Runtime: tokenization and other pre- and post-processing, logits processing, search and sampling, KV-cache management and grammar support for tool calling. Plain InferenceSession runs one forward pass; an LLM needs hundreds of them with state carried between calls, which is the part this library implements. It is what Foundry Local and Windows ML use for generative models, and it is distributed on NuGet as Microsoft.ML.OnnxRuntimeGenAI (and a .WinML variant for use with Windows ML). The libraries are a 0.x preview line and their API is documented as subject to change. Supported models include several LLM families and generative speech-to-text models; the project’s C# examples folder contains chat, multimodal and speech-recognition samples.

Generation loop with the C# API (chat prompt already templated)
using Microsoft.ML.OnnxRuntimeGenAI;

using var model = new Model("C:\\models\\phi-4-mini-instruct-int4-cpu");
using var tokenizer = new Tokenizer(model);
using var stream = tokenizer.CreateStream();

var sequences = tokenizer.Encode("<|user|>What is ONNX?<|end|><|assistant|>");

using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 512);

using var generator = new Generator(model, generatorParams);
generator.AppendTokenSequences(sequences);

while (!generator.IsDone())
{
    generator.GenerateNextToken();
    Console.Write(stream.Decode(generator.GetSequence(0)[^1]));
}

Official pages: ONNX Runtime GenAI C# API and the C# examples. The chat template shown above is model-specific; take it from the model card (see Small LLMs & Embeddings) and do not reuse one model’s template for another. Method names in a 0.x library move, so compare the snippet with the example project for the exact package version you install.

The model builder

Many Hub models are not published as ONNX GenAI folders. The project’s model builder converts a Hub or local model into the folder layout the runtime expects (ONNX graph, generation config, tokenizer files) and quantizes it in the same step.

Building an int4 CPU model from a Hub model with the model builder
pip install onnxruntime-genai

python -m onnxruntime_genai.models.builder `
    -m microsoft/Phi-4-mini-instruct `
    -o .\phi-4-mini-instruct-int4-cpu `
    -p int4 `
    -e cpu `
    -c .\hf-cache

Official page: Model builder guide. The flags are -m (Hub model name), -o (output folder), -p (precision: fp16, bf16, fp32, int4 or int8), -e (execution provider target: cpu, cuda, dml, webgpu and NvTensorRtRtx are accepted by the current builder, and the list grows, so run the builder with -h for the provider you need) and -c (cache directory); -i can point at a local PyTorch folder or a GGUF file instead of -m, and --extra_options key=value passes builder-specific settings. Run the conversion in a virtual environment, prefer models from publishers you trust, and read the licence on the card before shipping converted weights, as covered in Choosing Models & Licences. Where Microsoft already publishes an ONNX variant of a model (for example microsoft/Phi-4-mini-instruct-onnx, MIT at the time of checking), downloading it skips the build.

DirectML status

DirectML is the DirectX 12 based, low-level machine-learning API that older ONNX Runtime-on-Windows setups used through the Microsoft.ML.OnnxRuntime.DirectML package. Microsoft Learn now states that DirectML is in sustained engineering: it stays supported, but new feature development has moved to Windows ML for Windows-based ONNX Runtime deployments. A new project should start from Windows ML, which offers the same ONNX Runtime APIs and selects providers by hardware; an existing DirectML app keeps working and can migrate by replacing the provider setup with the catalog and SetEpSelectionPolicy calls shown above. The in-box DirectML provider that Windows ML still ships is labelled legacy in its documentation.

Official page: Introduction to DirectML.

Microsoft.ML.OnnxRuntime: the plain ONNX path

When the model is a classifier, embedding model or another single-pass network, the lowest layer is often enough. The NuGet package Microsoft.ML.OnnxRuntime (1.30.0 on 2026-09-30, targeting .NET Standard 2.0 and .NET 9) gives InferenceSession, SessionOptions, NamedOnnxValue for the classic API and OrtValue for the lower-allocation API. The code is the same whether the process loads this package or the runtime that ships with Windows ML; with Windows ML, reference its package instead, as the deployment table above says, and do not mix both in one app.

Classify a screenshot with an ONNX image classifier (input tensor prepared elsewhere)
using Microsoft.ML.OnnxRuntime;
using Microsoft.ML.OnnxRuntime.Tensors;

using var session = new InferenceSession("screenshot-classifier.onnx");

// Shape and normalization come from the model card's preprocessing config.
var pixels = new DenseTensor<float>(new[] { 1, 3, 224, 224 });
// ... fill pixels: resize, scale to 0..1, normalize, channels first ...

string inputName = session.InputMetadata.Keys.First();
using var results = session.Run(
    [NamedOnnxValue.CreateFromTensor(inputName, pixels)]);

float[] logits = results.First().AsEnumerable<float>().ToArray();
int best = Array.IndexOf(logits, logits.Max());
Console.WriteLine($"class index: {best}");

Official page: ONNX Runtime with C#. Image preprocessing is where most wrong results come from: the resize method, mean and standard deviation, and channel order must match the model’s card or its preprocessor config, which is the same lesson as in Vision Models. A SigLIP 2 style zero-shot classifier also needs a text tower and a tokenizer, so a fixed-label classifier is the simpler first model for a screenshot type detector.

A memory estimate that applies on Windows too

Windows ML apps share RAM and VRAM with everything else on the PC, so estimate before downloading. For a model with parameters stored at bits per weight, the weights alone need roughly

and the KV cache and activations come on top of that. The derivation and the sizing rules are in Hardware Sizing and Quantization for Inference; the NPU and GPU requirements quoted by each provider are a separate constraint and are not derived from this formula.

Microsoft.Extensions.AI: one interface over any local runtime

Microsoft.Extensions.AI.Abstractions (10.10.1 on 2026-09-30) defines IChatClient and IEmbeddingGenerator<TInput, TEmbedding>; the Microsoft.Extensions.AI package adds middleware (function invocation, caching, OpenTelemetry, logging) on top. Application code written against the interfaces does not change when the model moves from Foundry Local to Ollama to a cloud service, which is the point for an app that wants a local default and a fallback. The core calls are GetResponseAsync (returns a ChatResponse), GetStreamingResponseAsync (an IAsyncEnumerable of ChatResponseUpdate) and, for embeddings, GenerateAsync (plus the GenerateVectorAsync helper for a single value).

Answer a transcribed question through IChatClient against a local Ollama model
using Microsoft.Extensions.AI;
using OllamaSharp;

IChatClient client = new OllamaApiClient(new Uri("http://localhost:11434/"), "phi3:mini");

List<ChatMessage> history =
[
    new(ChatRole.System, "You are DocsAssistant. Answer from the provided context only."),
    new(ChatRole.User, transcribedQuestion),
];

await foreach (ChatResponseUpdate update in client.GetStreamingResponseAsync(history))
{
    Console.Write(update);
}

Official page: Use the IChatClient interface. The model tag is the docs' own example; choose a model that fits the machine using Ollama and Desktop Runners, and see Open-Weight Models for what is current.

Foundry Local exposes an OpenAI-compatible endpoint, so any OpenAI-compatible adapter for IChatClient can point at it. With the official OpenAI SDK plus the Microsoft.Extensions.AI.OpenAI package that means creating an OpenAIClient with a non-default endpoint and adapting its chat client with AsIChatClient(); the local service does not validate the API key, so a placeholder string is enough. The URL and the model ID are the values reported by foundry server status and foundry model list. Adapter method names have changed between releases of the package, so check the current package’s API before copying this wiring; the OpenAI endpoint contract itself is what stays stable.

Embeddings behind the same abstraction
using Microsoft.Extensions.AI;
using OllamaSharp;

IEmbeddingGenerator<string, Embedding<float>> generator =
    new OllamaApiClient(new Uri("http://localhost:11434/"), "all-minilm");

GeneratedEmbeddings<Embedding<float>> embeddings =
    await generator.GenerateAsync(["How do I load a model once?", "What is lifespan?"]);

foreach (Embedding<float> e in embeddings)
{
    Console.WriteLine($"vector of {e.Vector.Length} floats");
}

Official page: Use the IEmbeddingGenerator interface. Embedding model choice, dimensions and metrics are in Small LLMs & Embeddings and Generating Embeddings in Practice; changing the embedding model means re-embedding the corpus.

The DocsAssistant scenario in C#

The four steps map onto the layers above. Only the last step of each snippet changes if the runtime changes.

Transcribe

Three routes exist. ONNX Runtime GenAI lists generative speech-to-text models among its supported models and ships a speech recognition C# sample; Foundry Local’s CLI has a transcribe command and its catalogue lists models for the speech task; and Whisper.net wraps whisper.cpp with ggml weights for C#. This page shows the third because its API is small and stable.

Whisper.net: transcribe a 16 kHz WAV file with a ggml Whisper model
using Whisper.net;

using var factory = WhisperFactory.FromPath("ggml-base.bin");
using var processor = factory.CreateBuilder()
    .WithLanguage("auto")
    .Build();

using var wav = File.OpenRead("question.wav");

var text = new System.Text.StringBuilder();
await foreach (var segment in processor.ProcessAsync(wav))
{
    text.Append(segment.Text);
}

string transcribedQuestion = text.ToString().Trim();

Official page: Whisper.net (MIT). The Whisper.net package is the API and the Whisper.net.Runtime packages (CPU by default, with CUDA, Vulkan, OpenVINO and other variants) supply the native code. The ggml weights are a different format from ONNX; see Model Formats and Speech Models: STT & TTS for how Whisper variants differ, and confirm the licence of the weights you download (for example openai/whisper-small is Apache-2.0 on its card at the time of checking).

Answer

Use the IChatClient shown above with Foundry Local or Ollama, or Phi Silica where the Limited Access Feature is unlocked and the hardware qualifies. Retrieval for the answer is the embedding generator from the previous section. For multi-turn state, keep the List<ChatMessage> and add the response messages back with AddMessages, as the IChatClient page shows.

Speak

The lowest-effort route is the system voice API, which uses Microsoft-signed voices that are installed on the PC; these are not Hub models, and a voice exists per language and region.

SpeechSynthesizer: synthesize the answer to an audio stream
using Windows.Media.SpeechSynthesis;

using var synth = new SpeechSynthesizer();
SpeechSynthesisStream stream = await synth.SynthesizeTextToStreamAsync(answerText);

// stream.ContentType says what the audio is; hand the stream to your media player.

Official page: SpeechSynthesizer class.

For a Hub voice, Kokoro is published in ONNX form (for example onnx-community/Kokoro-82M-v1.0-ONNX, Apache-2.0 on its card at the time of checking) and can be run with InferenceSession. The ONNX file takes token IDs and a voice style vector, not text, so a C# app must also supply a phonemizer and the voice files; that preprocessing is the real work and is not shown here. Treat a C# Kokoro integration as an experiment to verify against the model’s card and its example code, not as a documented Microsoft path.

Classify or describe a screenshot

Pick by what the answer must contain. For the text in the screenshot, use TextRecognizer on a Copilot+ PC. For "which kind of screen is this", use a Hub classifier exported to ONNX and run it with the InferenceSession snippet above, with Windows ML choosing the provider. For a free-text description, use a vision-capable chat model on Foundry Local or Ollama through IChatClient, which accepts image content in messages. Model choices are in Vision Models.

Operating notes

  • Download execution providers and models on first run with a visible progress indicator; the docs warn that the first call to EnsureAndRegisterCertifiedAsync can take minutes, and Phi Silica’s GPU model is several gigabytes and downloads in the background.

  • Pre-compile models for the chosen provider once and cache them, then recompile after EP or runtime updates.

  • Handle NotSupportedOnCurrentSystem and missing NPUs with a CPU or Foundry Local fallback instead of failing; the EP device list can change at run time.

  • Keep model weights out of the installer when size matters, and download them from the Hub or the Foundry catalogue on demand; authenticate Hub downloads of gated models with an environment variable such as $HF_TOKEN, never with a literal in source.

  • Prefer safetensors or ONNX from publishers you trust, and never load pickle-based checkpoints or run remote-code model repositories inside a desktop app without reviewing them; a model file is code-adjacent input.

  • Bind any local server to loopback only. Foundry Local’s and Ollama’s endpoints are unauthenticated by default.

References