Vision: Image Analysis and OCR

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

See the section bibliography for the reference material consulted while preparing these pages.

This page targets Azure CLI 2.90.0, the Image Analysis 4.0 REST API 2024-02-01 (and 2023-10-01), and the client libraries azure-ai-vision-imageanalysis 1.0.0 (Python), Azure.AI.Vision.ImageAnalysis 1.0.0 (.NET) and com.azure:azure-ai-vision-imageanalysis 1.0.10 (Java), checked on 2026-10-06. Some Learn quickstarts still install beta packages with --prerelease; the 1.0.x packages above are GA.

Image Analysis 3.2 and 4.0, including multimodal embeddings, are deprecated and retire on 2028-09-25. Microsoft’s migration options are: Document Intelligence Read for OCR, Face for face detection, Content Understanding for structured extraction and classification, and Foundry Models (multimodal LLMs) for captions, tags and visual question answering. Don’t start new long-lived features on Image Analysis. See Image Analysis migration options.

Azure Vision in Foundry Tools describes what is in an image. Image Analysis 4.0 answers several questions about one image in a single call; you choose them with the features parameter.

Image Analysis 4.0 features

Feature Returns

Feature	Returns
`caption`	One sentence describing the image, with a confidence score. `gender-neutral-caption=true` avoids gendered words.
`denseCaptions`	Up to 10 captions for regions of the image, with bounding boxes.
`tags`	Content tags (objects, scenery, actions) with confidence.
`objects`	Detected objects with bounding boxes.
`people`	Bounding boxes of people (no identification).
`smartCrops`	Suggested crop regions for given aspect ratios that keep the area of interest.
`read`	Synchronous OCR of printed and handwritten text in the image, by line and word.

caption

One sentence describing the image, with a confidence score. gender-neutral-caption=true avoids gendered words.

denseCaptions

Up to 10 captions for regions of the image, with bounding boxes.

tags

Content tags (objects, scenery, actions) with confidence.

objects

Detected objects with bounding boxes.

people

Bounding boxes of people (no identification).

smartCrops

Suggested crop regions for given aspect ratios that keep the area of interest.

read

Synchronous OCR of printed and handwritten text in the image, by line and word.

Multimodal embeddings (retrieval:vectorizeImage and retrieval:vectorizeText) map images and text into one vector space for image search. The multilingual model accepts text queries in 100+ languages.

Input limits: JPEG, PNG, GIF, BMP, WEBP, ICO, TIFF or MPO; under 20 MB; between 50 x 50 and 16,000 x 16,000 pixels. Caption and dense captions are available only in some regions (for example East US, West Europe, West US and others); check region availability before you create the resource.

Features only in 3.2: celebrity and landmark recognition, brand detection, adult-content flags, image type and color scheme. For moderation, use Content Safety instead (Content Safety, Prompt Shields and Groundedness).

Source: Image Analysis overview

Provisioning

In the Azure portal

Use the Foundry resource foundry-bookshelf in a region that supports captions (East US 2 doesn’t support every 4.0 feature; check the region table and use a separate resource if needed).
In the Foundry portal (https://ai.azure.com), open Playgrounds → Vision playground to try captions, tags and OCR on sample images.
Assign Cognitive Services User to the calling identity.

With the Azure CLI

az cognitiveservices account create \
  --name vision-bookshelf --resource-group "$RG" --location eastus \
  --kind ComputerVision --sku S1 --custom-domain vision-bookshelf --yes

Source: Azure CLI reference — az cognitiveservices account create

A single-service ComputerVision resource in East US is used here because caption features aren’t available in every region. Run az cognitiveservices account list-kinds first to confirm the kind name.

Calling Image Analysis

Generate alt text and a square thumbnail crop for a book cover:

from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures
from azure.identity import DefaultAzureCredential

client = ImageAnalysisClient(
    endpoint="https://vision-bookshelf.cognitiveservices.azure.com/",
    credential=DefaultAzureCredential(),
)
result = client.analyze_from_url(
    image_url="https://stbookshelf.blob.core.windows.net/covers/9780143111597.jpg",
    visual_features=[VisualFeatures.CAPTION, VisualFeatures.SMART_CROPS, VisualFeatures.READ],
    smart_crops_aspect_ratios=[1.0],
    gender_neutral_caption=True,
)
print(result.caption.text, result.caption.confidence)
print(result.smart_crops.list[0].bounding_box)
print([line.text for block in result.read.blocks for line in block.lines])

Source: Image Analysis 4.0 quickstart (Python)

using Azure.AI.Vision.ImageAnalysis;
using Azure.Identity;

var client = new ImageAnalysisClient(
    new Uri("https://vision-bookshelf.cognitiveservices.azure.com/"), new DefaultAzureCredential());

ImageAnalysisResult result = client.Analyze(
    new Uri("https://stbookshelf.blob.core.windows.net/covers/9780143111597.jpg"),
    VisualFeatures.Caption | VisualFeatures.SmartCrops,
    new ImageAnalysisOptions { GenderNeutralCaption = true, SmartCropsAspectRatios = new[] { 1.0f } });

Console.WriteLine($"{result.Caption.Text} ({result.Caption.Confidence:F2})");
Console.WriteLine(result.SmartCrops.Values[0].BoundingBox);

Source: Image Analysis 4.0 quickstart (C#)

import com.azure.ai.vision.imageanalysis.ImageAnalysisClient;
import com.azure.ai.vision.imageanalysis.ImageAnalysisClientBuilder;
import com.azure.ai.vision.imageanalysis.models.*;
import com.azure.identity.DefaultAzureCredentialBuilder;
import java.util.List;

ImageAnalysisClient client = new ImageAnalysisClientBuilder()
    .endpoint("https://vision-bookshelf.cognitiveservices.azure.com/")
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildClient();

ImageAnalysisResult result = client.analyzeFromUrl(
    "https://stbookshelf.blob.core.windows.net/covers/9780143111597.jpg",
    List.of(VisualFeatures.CAPTION, VisualFeatures.TAGS),
    new ImageAnalysisOptions().setGenderNeutralCaption(true));

System.out.println(result.getCaption().getText());

Source: Image Analysis 4.0 quickstart (Java)

curl -s -X POST \
  "https://vision-bookshelf.cognitiveservices.azure.com/computervision/imageanalysis:analyze?api-version=2024-02-01&features=caption,tags,smartCrops&smartcrops-aspect-ratios=1.0&gender-neutral-caption=true" \
  -H "Authorization: Bearer $(az account get-access-token --resource https://cognitiveservices.azure.com --query accessToken -o tsv)" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://stbookshelf.blob.core.windows.net/covers/9780143111597.jpg"}'

Source: Image Analysis REST API — Analyze image

The Read container

The Read OCR model is also available as a container (mcr.microsoft.com/azure-cognitive-services/vision/read), for on-premises OCR. It reports usage to a Vision resource for billing. New OCR work should consider the Document Intelligence Read container instead, since Image Analysis is deprecated.

OCR choices

Option Strengths Use it for

Option	Strengths	Use it for
Image Analysis 4.0 `read`	Synchronous, one call with captions and tags.	Short text in photos, until 2028-09-25.
Document Intelligence `prebuilt-read`	Pages, paragraphs, handwriting, 300+ languages, PDFs and Office files, searchable PDF output. Microsoft’s recommended replacement.	Scans, receipts, book pages, anything document-like.
Content Understanding `prebuilt-read`	Same family as Document Intelligence, inside a multimodal analyzer API.	When you already use Content Understanding for other fields.

Image Analysis 4.0 read

Synchronous, one call with captions and tags.

Short text in photos, until 2028-09-25.

Document Intelligence prebuilt-read

Pages, paragraphs, handwriting, 300+ languages, PDFs and Office files, searchable PDF output. Microsoft’s recommended replacement.

Scans, receipts, book pages, anything document-like.

Content Understanding prebuilt-read

Same family as Document Intelligence, inside a multimodal analyzer API.

When you already use Content Understanding for other fields.

More detail is on Document Intelligence: Read, Layout and Prebuilt Models and Choosing a Foundry Tool (or a Foundry Model).

Custom Vision

Custom Vision retires on 2028-09-25. Migrate image classification and object detection models to Azure Machine Learning AutoML for images, to a Foundry Model (few-shot prompting of a multimodal model), or to a Content Understanding classifier. See Custom Vision migration options.

Background removal, Product Recognition and Image Analysis model customization were retired on 2025-03-31, and Spatial Analysis and Video Retrieval were retired in 2025. For video, use Content Understanding with Azure AI Search.

Use cases

Scenario Feature Why this feature Example input → output

Scenario	Feature	Why this feature	Example input → output
Bookshelf: alt text and smart thumbnails for covers uploaded to `stbookshelf`	`caption` (gender-neutral) + `smartCrops` at 1:1	One call per upload, from an Event Grid-triggered job; accessible catalog pages.	`covers/<isbn>.jpg` → "a book cover with a red lighthouse" + crop box for a square thumbnail
Alt text for accessibility	`caption`, `denseCaptions`	Automatic descriptions for user-uploaded images.	Photo → "a person reading on a train"
Product-photo tagging	`tags`, `objects`	Filters and search facets.	Product photo → `book`, `table`, `coffee cup`
Reading signs and labels	`read`	Short text in a scene.	Shelf photo → spine titles
Image similarity search	Multimodal embeddings	Search images by text or by example.	"lighthouse cover" → similar cover images

Bookshelf: alt text and smart thumbnails for covers uploaded to stbookshelf

caption (gender-neutral) + smartCrops at 1:1

One call per upload, from an Event Grid-triggered job; accessible catalog pages.

covers/<isbn>.jpg → "a book cover with a red lighthouse" + crop box for a square thumbnail

Alt text for accessibility

caption, denseCaptions

Automatic descriptions for user-uploaded images.

Photo → "a person reading on a train"

Product-photo tagging

tags, objects

Filters and search facets.

Product photo → book, table, coffee cup

Reading signs and labels

read

Short text in a scene.

Shelf photo → spine titles

Image similarity search

Multimodal embeddings

Search images by text or by example.

"lighthouse cover" → similar cover images

When not to use it

For new long-lived features, prefer a Foundry Model for captions and tags, and Document Intelligence for OCR. Image Analysis retires on 2028-09-25.
For moderation of user images, use Content Safety.
For faces, use Face under Limited Access (Face Detection, Verification and Liveness).
See Choosing a Foundry Tool (or a Foundry Model) for the general trade-off.

Clean up

az cognitiveservices account delete --name vision-bookshelf --resource-group "$RG"
az cognitiveservices account purge --name vision-bookshelf --resource-group "$RG" --location eastus

Source: Recover or purge deleted resources

Document Intelligence: Read, Layout and Prebuilt Models covers the recommended OCR replacement.
Content Understanding: Multimodal Analyzers covers image analyzers with your own fields.
Blob Storage covers stbookshelf and Event Grid blob events.
Multimodal RAG covers image embeddings in retrieval.