Choosing a Foundry Tool (or a Foundry Model)

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

See the section bibliography for the reference material consulted while preparing these pages.

This page is current as of 2026-10-06 and assumes Azure CLI 2.90.0. Versions for each tool are on its own page and in the sub-section index. Microsoft is moving several older features to general-purpose models, so check each tool’s What’s new page before you commit to one.

Almost every job in this sub-section can be done three ways:

  • with a prebuilt Foundry Tool: a task-specific API such as Speech to text, Document Intelligence or Content Safety

  • with a Foundry Model: a general-purpose LLM or multimodal model from the Foundry model catalog (Azure OpenAI GPT-5.x, GPT-4.1 and so on), driven by a prompt

  • with a custom model: a Foundry Tool you train on your own data (custom speech, custom neural NER, a custom Document Intelligence model), or a model you fine-tune yourself

This page helps you choose. It then settles the four places where tools overlap: OCR, transcription, translation and documents.

Prebuilt tool vs LLM vs custom model

Concern Prebuilt Foundry Tool Foundry Model (LLM) Custom model

Cost unit

Per transaction, character, page, audio hour or image. Easy to forecast from volume.

Per input and output token. It depends on prompt length, output length and reasoning effort.

Training and hosting cost on top of the per-call price (for example, custom speech endpoint hosting per hour).

Latency

Low and predictable. Most sync calls return in tens to hundreds of milliseconds.

Higher and more variable. It grows with output length and reasoning.

Similar to the prebuilt tool it extends.

Determinism

The same input gives the same output for a given model version. Fields come with confidence scores.

Non-deterministic by default. You need structured outputs and evaluation to keep it stable.

Deterministic for a given trained version.

Compliance

Each tool has its own transparency note and certifications (HIPAA, SOC, ISO). Text Analytics for health and PII detection were built for regulated data.

Covered by the Foundry Models terms and the Azure OpenAI data-handling commitments. You still own prompt-level redaction.

Same as the base tool, plus governance of your training data.

Data residency

Regional resources. Some tools keep results for a short time (for example, 24 hours for async results).

Depends on the deployment type: Standard (regional), Data Zone or Global.

Training data stays in your resource’s region.

Offline / edge

Many tools ship containers, including disconnected ones (Pricing, Quotas, Containers and Monitoring).

Not available as a Foundry container.

Some custom models can run in containers (custom speech, custom NER).

Structured documents

Best accuracy on invoices, receipts, IDs and tables, with bounding boxes and confidence per field.

Good at free-form understanding. Weaker on exact table geometry and page coordinates, unless you put a layout model in front of it.

Best when your forms are company-specific and stable.

Effort to start

Lowest: call the API.

Low: write a prompt. Evaluation takes more work.

Highest: label data, train, evaluate and redeploy.

As a rule of thumb, pick the prebuilt tool when the job is narrow, high volume, latency-sensitive or regulated. Pick an LLM when the job needs judgment, open-ended output or a combination of steps. Pick a custom model only when a prebuilt tool is close but not accurate enough on your own data, and you can keep a labeled dataset up to date.

Microsoft’s own direction

Microsoft is moving several older features to general-purpose models. This is the clearest signal of where each choice is heading:

  • Language legacy features (sentiment and opinion mining, key phrases, summarization, CLU, custom question answering, custom text classification and orchestration) reach end of support on 2029-03-31. Microsoft recommends Foundry Models for them. Language core features (PII, language detection, NER, Text Analytics for health) stay. See Language: Sentiment, Summarization, CLU and Question Answering.

  • Image Analysis 3.2 and 4.0 retire on 2028-09-25. The migration options are Document Intelligence Read for OCR, Face for faces, Content Understanding for structured extraction, and Foundry Models for captions and tags. See Vision: Image Analysis and OCR.

  • Content Understanding is itself built on LLMs. It is the managed bridge between "prebuilt tool" and "prompt an LLM", with schemas, confidence and grounding.

Decision flowchart

Start from the input modality, then the goal:

flowchart TD start([What is the input?]) --> audio[Audio] start --> text[Text] start --> image[Image] start --> doc[Document: PDF, scan, Office file] start --> video[Video] audio --> aGoal{Goal} aGoal -->|transcribe| stt[Speech to text] aGoal -->|translate speech| st[Speech translation] aGoal -->|live voice agent| vl[Voice Live] aGoal -->|summary plus fields| cuA[Content Understanding audio] text --> tGoal{Goal} tGoal -->|redact PII or extract entities| lang[Language core] tGoal -->|translate| tr[Translator] tGoal -->|block harmful or jailbreak input| cs[Content Safety] tGoal -->|sentiment, summary, intent| llm[Foundry Model with a prompt] tGoal -->|search or retrieve| srch[Azure AI Search] image --> iGoal{Goal} iGoal -->|read text| diRead[Document Intelligence Read] iGoal -->|caption or tag| llmV[Foundry Model or Image Analysis 4.0 until 2028] iGoal -->|faces| face[Face, Limited Access] iGoal -->|moderate| csI[Content Safety image] doc --> dGoal{Goal} dGoal -->|known form type| diPre[Document Intelligence prebuilt] dGoal -->|own fixed forms| diCustom[Document Intelligence custom] dGoal -->|fields that need reasoning| cuD[Content Understanding] dGoal -->|RAG ingestion| layout[Layout to Markdown] video --> vGoal{Goal} vGoal -->|chapters, summary, fields| cuV[Content Understanding video]

The arrows point to pages in this sub-section: Speech to Text, Speech Translation, Pronunciation Assessment and Other Speech Features, Voice Live API for Voice Agents, Language: PII, Entities, Language Detection and Health, Text Translation with Translator, Content Safety, Prompt Shields and Groundedness, Azure AI Search with Foundry Tools, Document Intelligence: Read, Layout and Prebuilt Models, Document Intelligence: Custom Models, Classification and RAG Ingestion, Face Detection, Verification and Liveness and Content Understanding: Multimodal Analyzers.

Overlaps resolved

OCR: Vision Read vs Document Intelligence Read vs Content Understanding

Option Choose it when Avoid it when

Vision Image Analysis 4.0 read feature

You already call Image Analysis for captions or tags on photos and want the short text in the same call (signs, labels, book spines).

You are starting new work. It retires on 2028-09-25.

Document Intelligence prebuilt-read

You need OCR on scans, PDFs or photos of documents, with pages, lines, words, languages and handwriting. It is Microsoft’s recommended OCR replacement for Image Analysis.

You need named fields. Use a prebuilt or custom model, or Content Understanding.

Content Understanding prebuilt-read / prebuilt-layout

You already use Content Understanding analyzers and want one API for documents, images, audio and video.

You only need plain OCR at the lowest cost per page.

Transcription: Speech vs Whisper/gpt-transcribe vs LLM speech

  • Speech to text (real-time, fast and batch) has the widest language coverage, diarization, word timestamps, custom speech and containers. It is the default.

  • Whisper / gpt-transcribe models in Foundry Models are good for noisy, accented or mixed-language audio when you don’t need diarization or custom vocabulary training.

  • LLM speech (preview) in Speech uses a large speech model with prompt tuning, and can transcribe and translate in one pass.

The details, with a comparison figure, are in Speech to Text.

Translation: Translator NMT vs LLM translation vs Speech translation

  • Translator NMT is the cheapest and fastest option, billed per character.

  • Translator LLM translation uses the same 2026-06-06 API with a GPT model deployment, adding tone and gender control. It is billed per token and needs a Foundry resource.

  • Speech translation handles audio in and text or audio out, in one stream.

See Text Translation with Translator and Speech Translation, Pronunciation Assessment and Other Speech Features.

Documents: Document Intelligence vs Content Understanding

  • Document Intelligence gives fixed-schema prebuilt models (invoice, receipt, ID and so on) and trainable custom models. Its output is deterministic, with page geometry.

  • Content Understanding lets you describe the fields you want in a schema, and an LLM extracts, classifies or generates them. It can reason across a document, and it also works on audio and video.

The figure on Content Understanding: Multimodal Analyzers compares both with a plain LLM call.

The same job, two ways

Bookshelf needs the title and author from a scanned back cover. A prebuilt OCR call returns lines of text, and the app picks the fields. An LLM call returns the fields directly but costs tokens and needs a prompt.

from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.identity import DefaultAzureCredential

client = DocumentIntelligenceClient(
    "https://foundry-bookshelf.cognitiveservices.azure.com/", DefaultAzureCredential()
)
with open("back-cover.jpg", "rb") as f:
    result = client.begin_analyze_document("prebuilt-read", body=f).result()
lines = [line.content for page in result.pages for line in page.lines]
print(lines[:5])  # the app applies its own rules to find title and author

Source: Document Intelligence quickstart

using Azure;
using Azure.AI.OpenAI;
using Azure.Identity;
using OpenAI.Chat;

var client = new AzureOpenAIClient(
    new Uri("https://foundry-bookshelf.openai.azure.com/"), new DefaultAzureCredential());
ChatClient chat = client.GetChatClient("gpt-4.1-mini");

var image = ChatMessageContentPart.CreateImagePart(
    BinaryData.FromBytes(File.ReadAllBytes("back-cover.jpg")), "image/jpeg");
ChatCompletion reply = chat.CompleteChat(
    new UserChatMessage(
        ChatMessageContentPart.CreateTextPart("Return JSON with title and author from this book cover."),
        image));
Console.WriteLine(reply.Content[0].Text);

Source: Use vision-enabled chat models

The LLM call is simpler to write. The prebuilt call is cheaper, faster and repeatable at volume. Bookshelf uses the prebuilt call in its ingestion job and keeps the LLM for the long tail of covers where the rules fail.

AWS and Google Cloud equivalents

Job Foundry Tool AWS Google Cloud

Speech to text

Speech to text

Amazon Transcribe

Speech-to-Text (Chirp)

Text to speech

Text to speech

Amazon Polly

Text-to-Speech

Text translation

Translator

Amazon Translate

Cloud Translation

Entities, PII, sentiment

Language

Amazon Comprehend (and Comprehend Medical)

Natural Language API (and Healthcare Natural Language)

Image analysis and faces

Vision, Face

Amazon Rekognition

Cloud Vision API

Documents and forms

Document Intelligence, Content Understanding

Amazon Textract, Amazon Bedrock Data Automation

Document AI

Content moderation and prompt attacks

Content Safety

Amazon Bedrock Guardrails, Rekognition moderation

Model Armor, Vision SafeSearch

Enterprise search and RAG

Azure AI Search

Amazon Kendra, Bedrock Knowledge Bases

Vertex AI Search

The AWS and Google Cloud sides are covered on AI, Data and Analytics Services Overview and AI, Data and Analytics Overview.

Related pages

References