Using Foundry Tools from Agents (MCP and Toolbox)
|
This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. See the section bibliography for the reference material consulted while preparing these pages. |
This page targets Azure CLI 2.90.0, the Azure Developer CLI (azd) with the ai agent extension, and the Foundry
SDK azure-ai-projects 2.8.0 (Python), Azure.AI.Projects 2.0.1 (.NET), com.azure:azure-ai-projects 2.6.1 (Java)
and @azure/ai-projects 2.8.0 (JavaScript), checked on 2026-10-06. The Foundry Tools Catalog and some MCP servers on
this page are preview.
Two meanings of "tools"
"Tools" means two different things in Microsoft Foundry, and this page is where they meet:
-
Foundry Tools is the product catalog this sub-section documents: Speech, Translator, Language, Vision, Document Intelligence, Content Understanding, Content Safety and Azure AI Search.
-
Agent tools are what a Foundry agent can call while it reasons: function calling, MCP servers, code interpreter, file search, Azure AI Search, web search, browser automation and so on.
A Foundry Tool becomes an agent tool in one of three ways:
-
through a managed MCP server from the Foundry Tools Catalog (preview), such as the Azure Speech MCP server and the Azure Language MCP server
-
through a built-in agent tool that wraps it, such as the Azure AI Search tool or a knowledge base (Azure AI Search with Foundry Tools)
-
through a function tool in your own code that calls the tool’s SDK
The Toolbox
The Toolbox gives an agent a single MCP-compatible endpoint that bundles the tools you enable for it: built-in tools, catalog MCP servers, OpenAPI tools and connectors. Clients outside Foundry (an IDE, another agent framework) can also connect to that endpoint as an MCP server, so the same governed tool set is reused everywhere.
The Foundry Tools Catalog
The Foundry Tools Catalog (portal: Build → Tools → Catalog) lists managed MCP servers and connectors from Microsoft and partners. Adding one creates a connection on the project, with authentication handled by Foundry (the project’s managed identity or OAuth on behalf of the user).
Source: Agent Toolbox overview
Azure Speech MCP server
The Azure Speech MCP server exposes speech to text and text to speech as MCP tools, so an agent can transcribe an audio file from a URL or produce narration. Setting it up in the catalog asks for:
-
the Speech resource (or Foundry resource) endpoint
-
the authentication method: the project’s managed identity with
Cognitive Services Speech User, or a key stored in the connection -
optionally a storage location for generated audio
Limitations at the time of writing: it isn’t available for Foundry resources that use network isolation (private endpoints and disabled public access), and some features need the Enterprise tier of the agent service. Check the how-to page before relying on it.
Source: Azure Speech tool for agents
Azure Language MCP server
The Azure Language MCP server exposes Language features (PII redaction, NER, language detection, CLU and question answering) as MCP tools. It comes as a remote endpoint added from the catalog, or as a local server you run next to an agent framework (for example in VS Code or in a container).
Source: Azure Language tools for agents
Agent templates: Intent Routing and Exact Question Answering
Two azd templates combine Language with an LLM agent:
-
Intent Routing classifies each user message with CLU (deterministic for known intents) and falls back to the LLM for everything else.
-
Exact Question Answering answers from a custom question answering project verbatim when a high-confidence match exists (approved policy text), and lets the LLM answer otherwise.
azd ai agent init --template intent-routing # or exact-question-answering
azd up
Source: Azure Language tools for agents
Calling a tool SDK from a function tool
When no catalog server fits, write a function and register it as a tool. The agent decides when to call it; your
code runs it with the tool’s SDK and returns the result. Below, a Bookshelf support agent gets a redact_pii tool
backed by Language.
import json
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import FunctionTool, PromptAgentDefinition
from azure.ai.textanalytics import TextAnalyticsClient
from azure.identity import DefaultAzureCredential
cred = DefaultAzureCredential()
project = AIProjectClient(
endpoint="https://foundry-bookshelf.services.ai.azure.com/api/projects/proj-bookshelf", credential=cred
)
language = TextAnalyticsClient("https://foundry-bookshelf.cognitiveservices.azure.com/", cred)
def redact_pii(text: str) -> str:
return language.recognize_pii_entities([text])[0].redacted_text
redact_tool = FunctionTool(
name="redact_pii",
description="Remove personal data from customer text before storing or quoting it",
parameters={"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]},
)
agent = project.agents.create_version(
agent_name="bookshelf-support",
definition=PromptAgentDefinition(
model="gpt-4.1-mini",
instructions="You help Bookshelf customers. Always redact customer text before quoting it.",
tools=[redact_tool],
),
)
openai = project.get_openai_client()
response = openai.responses.create(
input="Summarize: 'Jane Doe, jane@example.com, says order 4512 is late.'",
extra_body={"agent": {"name": agent.name, "type": "agent_reference"}},
)
# Answer every tool call of a round in one request, and keep going until no calls remain
while calls := [item for item in response.output if item.type == "function_call"]:
outputs = [
{
"type": "function_call_output",
"call_id": call.call_id,
"output": redact_pii(**json.loads(call.arguments)) if call.name == "redact_pii" else "unknown tool",
}
for call in calls
]
response = openai.responses.create(
previous_response_id=response.id,
input=outputs,
extra_body={"agent": {"name": agent.name, "type": "agent_reference"}},
)
print(response.output_text)
using Azure.AI.Projects;
using Azure.Identity;
var project = new AIProjectClient(
new Uri("https://foundry-bookshelf.services.ai.azure.com/api/projects/proj-bookshelf"),
new DefaultAzureCredential());
// List the connections (catalog MCP servers, AI Search, storage) available to agents in this project
foreach (var connection in project.Connections.GetConnections())
Console.WriteLine($"{connection.Name}: {connection.Type}");
import com.azure.ai.projects.AIProjectClientBuilder;
import com.azure.ai.projects.ConnectionsClient;
import com.azure.identity.DefaultAzureCredentialBuilder;
ConnectionsClient connections = new AIProjectClientBuilder()
.endpoint("https://foundry-bookshelf.services.ai.azure.com/api/projects/proj-bookshelf")
.credential(new DefaultAzureCredentialBuilder().build())
.buildConnectionsClient();
connections.listConnections().forEach(c -> System.out.println(c.getName() + ": " + c.getType()));
The Foundry SDK and the project endpoint
Agents, connections, evaluations and the OpenAI-compatible responses API all go through the project endpoint,
https://<resource>.services.ai.azure.com/api/projects/<project>. The Foundry Tools themselves keep using the
cognitiveservices.azure.com endpoint and their own SDKs, as in the function above. Check connectivity with curl:
TOKEN=$(az account get-access-token --scope https://ai.azure.com/.default --query accessToken -o tsv)
curl -s "https://$FOUNDRY.services.ai.azure.com/api/projects/proj-bookshelf/connections?api-version=v1" \
-H "Authorization: Bearer $TOKEN" | jq -r '.value[].name'
Classic agents
|
Foundry classic agents (the Assistants-style threads and runs API) retire on 2027-03-31. Build new agents with the current Foundry agents (agent versions and the responses API), and migrate classic agents before that date. See Foundry Agent Service overview. |
Provisioning
In the Azure portal
-
In the Foundry portal (
https://ai.azure.com), openproj-bookshelfand go to Build → Tools. -
From Catalog, add Azure Speech and Azure Language; select
foundry-bookshelfand managed identity authentication. -
Grant the project’s managed identity
Cognitive Services Useronfoundry-bookshelfandSearch Index Data Readeronsearch-bookshelf. -
Create the agent under Agents, add the tools, and test in the playground.
With the Azure CLI
PROJECT_PRINCIPAL=$(az cognitiveservices account project show --name "$FOUNDRY" --resource-group "$RG" \
--project-name proj-bookshelf --query identity.principalId -o tsv)
az role assignment create --assignee-object-id "$PROJECT_PRINCIPAL" --assignee-principal-type ServicePrincipal \
--role "Cognitive Services User" \
--scope "$(az cognitiveservices account show --name "$FOUNDRY" --resource-group "$RG" --query id -o tsv)"
Use case: a support agent that combines four tools
Bookshelf’s after-hours support line takes voicemail. A support agent handles each message:
-
Speech (MCP server): transcribe the voicemail from the
mediacontainer. -
Language (function tool or MCP server): redact PII from the transcript.
-
Azure AI Search (knowledge base tool): retrieve the relevant shipping and returns policy passages.
-
LLM: draft a reply grounded in those passages.
-
Content Safety (guardrail on the deployment): check the draft for groundedness and harmful content before it is queued for a human to send.
| Scenario | Feature | Why this feature | Example input → output |
|---|---|---|---|
Bookshelf: voicemail support agent |
Speech MCP, Language PII, knowledge base, guardrails |
Each step uses the specialist tool; the LLM only plans and writes. |
Voicemail → redacted transcript → grounded reply draft |
IT help desk |
Intent Routing template |
Known intents handled deterministically. |
"Reset my password" → |
Regulated FAQ |
Exact Question Answering template |
Approved answers returned word for word. |
"What is the APR?" → approved text |
When not to use it
-
When the job is a fixed pipeline with no decisions (always transcribe, then redact, then index), a plain job or workflow is cheaper and more predictable than an agent.
-
When a single tool call answers the request, call the tool directly from your app.
-
See Choosing a Foundry Tool (or a Foundry Model) and Tool Design.
Clean up
Remove what this page created: the resources provisioned by the azd agent template, and the project identity’s
role assignment on foundry-bookshelf. Delete the bookshelf-support agent from Agents in the Foundry portal if
you created it with the Python sample.
# In the folder where you ran azd ai agent init
azd down --purge --force
az role assignment delete --assignee "$PROJECT_PRINCIPAL" --role "Cognitive Services User" \
--scope "$(az cognitiveservices account show --name "$FOUNDRY" --resource-group "$RG" --query id -o tsv)"
Related pages
-
Model Context Protocol (MCP) explains MCP servers, clients and transports.
-
Tool Design covers how to design tools an agent can use well.
-
Voice Live API for Voice Agents covers real-time voice agents.
-
Language: Sentiment, Summarization, CLU and Question Answering covers CLU and question answering behind the templates.