Azure AI Search with Foundry Tools

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

See the section bibliography for the reference material consulted while preparing these pages.

This page targets Azure CLI 2.90.0, the Azure AI Search data-plane REST API 2026-04-01 (stable) and 2026-08-01-preview, and the client libraries azure-search-documents 12.0.0 (Python), Azure.Search.Documents 12.0.0 (.NET), com.azure:azure-search-documents 12.0.2 (Java) and @azure/search-documents 13.0.0 (JavaScript), checked on 2026-10-06.

Azure AI Search is a separate resource, not part of the Foundry resource, but it is listed in the Foundry Tools catalog because it calls Foundry Tools during indexing and powers retrieval for Foundry agents. The short introduction is on AI, Data and IoT Services Overview; this page covers how Search and the other tools work together.

AI enrichment skillsets

An indexer pulls content from a data source (Blob storage, Cosmos DB, SQL), cracks documents, runs a skillset over them, and writes the enriched results to an index.

flowchart LR blob[(stbookshelf back-covers)] --> idx[Indexer] idx --> crack[Document cracking] crack --> ss[Skillset] ss --> ocr[OCR skill] ocr --> merge[Merge text] merge --> ner[Entity recognition skill] merge --> split[Text split skill] split --> emb[Azure OpenAI embedding skill] ner --> index[(catalog index on search-bookshelf)] emb --> index ss -.->|billed through| foundry[Attached foundry-bookshelf]

Three kinds of skill call Foundry Tools:

  • Built-in skills billed through an attached Foundry resource: OCR, image analysis, entity recognition, PII detection, key phrases, language detection, sentiment, text translation and multimodal embeddings. Attach foundry-bookshelf to the skillset (keyless, with the search service’s managed identity) to go beyond the free 20 documents per indexer per day.

  • Skills that call models or tools on your own resource: the Content Understanding skill (layout and chunking of complex documents), the Azure OpenAI embedding skill, and the GenAI Prompt skill (an LLM prompt per document, for summaries or classification).

  • Custom Web API skills: your own HTTP endpoint, for example an Azure Function that calls any Foundry Tool not offered as a built-in skill.

Image analysis and OCR skills depend on Image Analysis, which retires on 2028-09-25; for new pipelines, prefer the Content Understanding or Document Layout skill for OCR.

Source: Built-in skills for AI enrichment

Integrated vectorization

Integrated vectorization chunks and embeds content during indexing (Text Split + embedding skill) and embeds the query at search time with a vectorizer defined on the index, so clients send plain text and Search handles vectors. Combined with hybrid search (BM25 keyword plus vector, merged with Reciprocal Rank Fusion) and the semantic ranker (a cross-encoder that reranks the top results and returns captions and answers), this is the retrieval stack most RAG apps on Azure use.

Source: Integrated vectorization

Agentic retrieval

Agentic retrieval uses an LLM to plan the search: it breaks a complex or conversational question into sub-queries, runs them in parallel against one or more knowledge sources, reranks with the semantic ranker, and returns grounded results with citations (or a synthesized answer).

  • A knowledge source points at an index, a Blob container, SharePoint, OneLake or the web.

  • A knowledge base groups knowledge sources with a model deployment for query planning and a reasoning effort (minimal, low, medium).

  • GA 2026-04-01 covers knowledge bases over search indexes and Blob; more sources, answer synthesis options and streaming are in 2026-08-01-preview.

  • Billing: query-planning tokens on your model deployment, plus agentic retrieval tokens on the search service (with a free monthly allowance), plus semantic ranker.

sequenceDiagram participant Agent as Support assistant participant KB as Knowledge base support-kb participant LLM as Planning model gpt-4.1-mini participant KS as Knowledge sources Agent->>KB: retrieve with conversation history KB->>LLM: Plan sub-queries LLM-->>KB: Three focused queries par Parallel search KB->>KS: Hybrid query on catalog index KB->>KS: Hybrid query on policies container end KS-->>KB: Ranked chunks KB->>KB: Semantic rerank and merge KB-->>Agent: Grounded results with references

Foundry IQ is the Foundry-side name for this capability: a knowledge base in Azure AI Search that Foundry agents connect to as a tool, so several agents share one governed knowledge layer.

Source: Agentic retrieval overview

Tiers

Free, Basic, Standard (S1 to S3), Storage Optimized (L1, L2) and serverless (preview), which bills per use instead of per search unit. Semantic ranker and agentic retrieval are available from Basic up. Choose the tier by index size, query rate and whether you need private endpoints.

Source: Choose a service tier

Provisioning

In the Azure portal

  1. Create the search service search-bookshelf (Basic or higher) in rg-bookshelf, with a system-assigned managed identity.

  2. On foundry-bookshelf, assign Cognitive Services User to the managed identity of search-bookshelf; on stbookshelf, assign Storage Blob Data Reader.

  3. In search-bookshelf, use Import data (new) with the RAG or Multimodal RAG option: pick the Blob source, attach foundry-bookshelf for skills, choose the embedding deployment, and enable the semantic ranker.

  4. For agentic retrieval, open Agentic retrieval → Knowledge bases (or connect from the Foundry portal’s Foundry IQ page).

With the Azure CLI

az search service create \
  --name search-bookshelf --resource-group "$RG" --location "$LOCATION" \
  --sku basic --identity-type SystemAssigned --semantic-search standard \
  --auth-options aadOrApiKey --aad-auth-failure-mode http401WithBearerChallenge

SEARCH_PRINCIPAL=$(az search service show --name search-bookshelf --resource-group "$RG" --query identity.principalId -o tsv)
az role assignment create --assignee-object-id "$SEARCH_PRINCIPAL" --assignee-principal-type ServicePrincipal \
  --role "Cognitive Services User" \
  --scope "$(az cognitiveservices account show --name "$FOUNDRY" --resource-group "$RG" --query id -o tsv)"

Source: Azure CLI reference — az search service create

Bookshelf: enriched catalog index and support knowledge base

The catalog index on search-bookshelf is fed by an indexer over scanned back covers in stbookshelf. A skillset runs OCR, entity recognition (people and organizations become author and publisher facets), text split and embedding. The skillset, attached to foundry-bookshelf keylessly:

TOKEN=$(az account get-access-token --resource https://search.azure.com --query accessToken -o tsv)

curl -s -X PUT "https://search-bookshelf.search.windows.net/skillsets/catalog-enrichment?api-version=2026-04-01" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{
    "name": "catalog-enrichment",
    "cognitiveServices": {
      "@odata.type": "#Microsoft.Azure.Search.AIServicesByIdentity",
      "subdomainUrl": "https://foundry-bookshelf.cognitiveservices.azure.com/"
    },
    "skills": [
      {"@odata.type": "#Microsoft.Skills.Vision.OcrSkill", "context": "/document/normalized_images/*",
       "inputs": [{"name": "image", "source": "/document/normalized_images/*"}],
       "outputs": [{"name": "text", "targetName": "ocrText"}]},
      {"@odata.type": "#Microsoft.Skills.Text.V3.EntityRecognitionSkill", "context": "/document",
       "categories": ["Person", "Organization"],
       "inputs": [{"name": "text", "source": "/document/content"}],
       "outputs": [{"name": "persons", "targetName": "authors"}, {"name": "organizations", "targetName": "publishers"}]}
    ]
  }'

Source: Attach a Foundry resource to a skillset

Query it with hybrid search and the semantic ranker:

from azure.identity import DefaultAzureCredential
from azure.search.documents import SearchClient
from azure.search.documents.models import VectorizableTextQuery

search = SearchClient("https://search-bookshelf.search.windows.net", "catalog", DefaultAzureCredential())
results = search.search(
    search_text="lighthouse novel by a Welsh author",
    vector_queries=[VectorizableTextQuery(text="lighthouse novel by a Welsh author", k_nearest_neighbors=50, fields="contentVector")],
    query_type="semantic", semantic_configuration_name="default", top=5,
)
for r in results:
    print(r["title"], r["@search.reranker_score"])

Source: Create a hybrid query

using Azure.Identity;
using Azure.Search.Documents;
using Azure.Search.Documents.Models;

var client = new SearchClient(
    new Uri("https://search-bookshelf.search.windows.net"), "catalog", new DefaultAzureCredential());

var options = new SearchOptions
{
    QueryType = SearchQueryType.Semantic,
    SemanticSearch = new SemanticSearchOptions { SemanticConfigurationName = "default" },
    VectorSearch = new VectorSearchOptions
    {
        Queries = { new VectorizableTextQuery("lighthouse novel by a Welsh author") { Fields = { "contentVector" } } },
    },
    Size = 5,
};
await foreach (SearchResult<SearchDocument> r in client.Search<SearchDocument>("lighthouse novel", options).Value.GetResultsAsync())
    Console.WriteLine(r.Document["title"]);

Source: Azure AI Search client library for .NET

import com.azure.identity.DefaultAzureCredentialBuilder;
import com.azure.search.documents.SearchClient;
import com.azure.search.documents.SearchClientBuilder;
import com.azure.search.documents.models.*;

SearchClient client = new SearchClientBuilder()
    .endpoint("https://search-bookshelf.search.windows.net")
    .indexName("catalog")
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildClient();

SearchOptions options = new SearchOptions()
    .setQueryType(QueryType.SEMANTIC)
    .setSemanticSearch(new SemanticSearchOptions().setSemanticConfigurationName("default"))
    .setTop(5);
client.search("lighthouse novel by a Welsh author", options, null)
    .forEach(r -> System.out.println(r.getDocument(SearchDocument.class).get("title")));

Source: Azure AI Search client library for Java

For the support assistant, a knowledge base support-kb combines the catalog index and a policies Blob knowledge source (shipping and returns PDFs), and the assistant calls its retrieve action:

curl -s -X POST "https://search-bookshelf.search.windows.net/knowledgebases/support-kb/retrieve?api-version=2026-04-01" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":[{"type":"text","text":"Can I return a signed first edition I bought last month?"}]}]}'

Source: Quickstart: agentic retrieval

Use cases

Scenario Feature Why this feature Example input → output

Bookshelf: catalog index enriched from scanned back covers

Skillset with OCR and entity recognition, integrated vectorization

Text on scanned covers becomes searchable; authors and publishers become facets.

Back-cover scan → indexed blurb, authors, publishers, vector

Bookshelf: support assistant knowledge base

Agentic retrieval over catalog and policies

Multi-part customer questions; citations required.

"Return a signed edition?" → grounded passages from the returns policy

Enterprise search over scanned PDFs

OCR or Content Understanding skill, semantic ranker

Scans have no text layer.

Archive of scans → searchable index

Multilingual catalog search

Text translation skill, multilingual embeddings

Search in one language, find content in another.

Spanish query → English documents

RAG for support agents

Hybrid search + semantic ranker, knowledge base

Best relevance for LLM context.

Ticket → top 5 chunks with captions

When not to use it

Clean up

az search service delete --name search-bookshelf --resource-group "$RG" --yes

Source: Azure CLI reference — az search service delete

Related pages

References