Speech Translation, Pronunciation Assessment and Other Speech Features

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

See the section bibliography for the reference material consulted while preparing these pages.

This page targets Azure CLI 2.90.0, Speech SDK 1.52.0 and the speech to text REST API 2025-10-15, checked on 2026-10-06.

Beyond plain transcription and synthesis, Azure Speech in Foundry Tools translates speech, scores how well a learner pronounces a phrase, identifies the spoken language, and listens for a wake word. This page covers those features, and the ones that were retired.

Speech translation

The TranslationRecognizer in the Speech SDK takes audio in one language and returns:

  • speech to text translation: the recognized text plus its translation into one or more target languages

  • speech to speech translation: the same, plus synthesized audio of the translation in a chosen voice

Up to 2 target languages are included in the speech translation price. Each additional target language is billed as Translator text characters.

flowchart LR mic[Microphone or stream] --> sr[Speech recognition] sr --> lid{Language identification} lid --> tr[Translation into target languages] tr --> txt[Translated text events] tr --> tts[Text to speech in target voice] tts --> audio[Synthesized audio events] sr --> src[Source transcript events]
import azure.cognitiveservices.speech as speechsdk
from azure.identity import DefaultAzureCredential

config = speechsdk.translation.SpeechTranslationConfig(
    endpoint="https://foundry-bookshelf.cognitiveservices.azure.com/",
    token_credential=DefaultAzureCredential(),
)
config.speech_recognition_language = "en-US"
config.add_target_language("es")
config.add_target_language("fr")
config.voice_name = "es-ES-ElviraNeural"  # speech to speech output for Spanish

recognizer = speechsdk.translation.TranslationRecognizer(
    translation_config=config,
    audio_config=speechsdk.audio.AudioConfig(use_default_microphone=True),
)
result = recognizer.recognize_once_async().get()
print(result.text, result.translations["es"], result.translations["fr"])

Source: How to translate speech (Python)

using Azure.Identity;
using Microsoft.CognitiveServices.Speech.Audio;
using Microsoft.CognitiveServices.Speech.Translation;

var config = SpeechTranslationConfig.FromEndpoint(
    new Uri("https://foundry-bookshelf.cognitiveservices.azure.com/"), new DefaultAzureCredential());
config.SpeechRecognitionLanguage = "en-US";
config.AddTargetLanguage("es");
config.AddTargetLanguage("de");

using var recognizer = new TranslationRecognizer(config, AudioConfig.FromDefaultMicrophoneInput());
recognizer.Recognized += (_, e) =>
{
    foreach (var (language, text) in e.Result.Translations)
        Console.WriteLine($"{language}: {text}");
};
await recognizer.StartContinuousRecognitionAsync();
Console.ReadLine();
await recognizer.StopContinuousRecognitionAsync();

Source: How to translate speech (C#)

import com.azure.identity.DefaultAzureCredentialBuilder;
import com.microsoft.cognitiveservices.speech.audio.AudioConfig;
import com.microsoft.cognitiveservices.speech.translation.*;
import java.net.URI;

SpeechTranslationConfig config = SpeechTranslationConfig.fromEndpoint(
    new URI("https://foundry-bookshelf.cognitiveservices.azure.com/"),
    new DefaultAzureCredentialBuilder().build());
config.setSpeechRecognitionLanguage("en-US");
config.addTargetLanguage("es");

try (TranslationRecognizer recognizer =
         new TranslationRecognizer(config, AudioConfig.fromDefaultMicrophoneInput())) {
    TranslationRecognitionResult result = recognizer.recognizeOnceAsync().get();
    System.out.println(result.getText() + " -> " + result.getTranslations().get("es"));
}

Source: How to translate speech (Java)

Live Interpreter

Live Interpreter is a speech translation mode built for live conversations:

  • it auto-detects the input language, so you don’t declare it up front

  • it handles mid-session language switching, for example a speaker who moves between English and Spanish

  • it can speak the translation in the speaker’s own personal voice (Limited Access), so listeners hear a translated version of the same voice

  • latency is tuned for simultaneous interpretation rather than sentence-by-sentence translation

Source: Speech translation overview

Pronunciation assessment

Pronunciation assessment compares what a learner says against a reference text (scripted) or against what the service recognized (unscripted). It returns scores from 0 to 100:

  • accuracy: how closely phonemes match a native speaker’s

  • fluency: pauses and rhythm

  • completeness: how much of the reference text was spoken

  • prosody: stress, intonation and rate

It works per phoneme, word and sentence, and supports a continuous mode for long reading passages.

pa_config = speechsdk.PronunciationAssessmentConfig(
    reference_text="The quick brown fox jumps over the lazy dog.",
    grading_system=speechsdk.PronunciationAssessmentGradingSystem.HundredMark,
    granularity=speechsdk.PronunciationAssessmentGranularity.Phoneme,
)
pa_config.enable_prosody_assessment()

recognizer = speechsdk.SpeechRecognizer(
    speech_config=speech_config,
    audio_config=speechsdk.audio.AudioConfig(filename="learner.wav"),
)
pa_config.apply_to(recognizer)
result = speechsdk.PronunciationAssessmentResult(recognizer.recognize_once())
print(result.accuracy_score, result.fluency_score, result.completeness_score, result.prosody_score)

Source: Use pronunciation assessment

Pronunciation content assessment (scoring vocabulary, grammar and topic) was retired in July 2025. Microsoft’s migration guidance is to send the transcript to an Azure OpenAI model with a grading prompt.

Language identification

Language identification (LID) picks the spoken language from a list of candidates:

  • at-start LID detects once, from up to 4 candidates

  • continuous LID re-detects throughout the audio, from up to 10 candidates

In the SDK, pass an AutoDetectSourceLanguageConfig. With fast transcription, pass several locales:

curl -s -X POST \
  "https://$FOUNDRY.cognitiveservices.azure.com/speechtotext/transcriptions:transcribe?api-version=2025-10-15" \
  -H "Authorization: Bearer $(az account get-access-token --resource https://cognitiveservices.azure.com --query accessToken -o tsv)" \
  -F "audio=@author-event-qa.wav" \
  -F 'definition={"locales":["en-US","es-ES","fr-FR"]}'

Source: Use the fast transcription API

Each phrase in the response carries its detected locale.

Custom keyword

Custom keyword trains a small on-device model for a wake word ("Hey Bookshelf") in Speech Studio. The model runs in the Speech SDK on the device, so audio is only streamed to the service after the keyword is heard. You download the .table file and load it with KeywordRecognitionModel.

Source: Create a custom keyword

Retired features: migration notes

Speaker recognition (verifying or identifying a person by voice) was retired on 2025-09-30, in Speech SDK 1.47. There is no direct replacement. Diarization in speech to text, which labels speakers without identifying them, is not affected.

Speech SDK intent recognition (IntentRecognizer with LUIS or pattern matching) was retired in October 2025, also in SDK 1.47. Transcribe with speech to text, then classify the text with CLU or an LLM, as shown on Language: Sentiment, Summarization, CLU and Question Answering.

ConversationTranslator (multi-device conversation translation with a shared room code) was removed in Speech SDK 1.51. Use speech translation per device, or Live Interpreter for a shared session.

The full list with dates is on Immersive Reader, Video Indexer and Retired Services.

Provisioning

In the Azure portal

  1. Use the Foundry resource foundry-bookshelf.

  2. In the Foundry portal, open Playgrounds → Speech playground and choose Speech translation, Live Interpreter or Pronunciation assessment to try each feature.

  3. Custom keyword is created in Speech Studio (https://speech.microsoft.com) under Custom keyword.

With the Azure CLI

az role assignment create \
  --assignee "$(az ad signed-in-user show --query id -o tsv)" \
  --role "Cognitive Services Speech User" \
  --scope "$(az cognitiveservices account show --name "$FOUNDRY" --resource-group "$RG" --query id -o tsv)"

Source: Role-based access control for Speech resources

Use cases

Scenario Feature Why this feature Example input → output

Bookshelf: live-translated online author events

Live Interpreter, speech to text translation for captions

The author speaks English; attendees read Spanish and French captions or listen to translated audio.

Author Q&A audio → captions in es and fr plus Spanish audio

Multilingual customer service

Speech translation, continuous LID

The agent and the caller don’t share a language.

Caller speaks Portuguese → agent sees English text

Travel and interpretation apps

Speech to speech translation

Hands-free, conversational.

"Where is the station?" → spoken Japanese

Language-learning apps

Pronunciation assessment with prosody

Phoneme-level feedback for the learner.

Learner reads a sentence → accuracy 82, fluency 74, errors on two words

Voice-assistant wake word

Custom keyword

Runs on the device; no audio leaves until the keyword is heard.

"Hey Bookshelf" → session starts

When not to use it

  • For text you already have, use Translator directly. It is cheaper than speech translation.

  • For grading what a learner said (content, grammar), use a Foundry Model with a rubric prompt. Pronunciation assessment scores how it was said.

  • For a voice agent that translates and answers questions, Voice Live with an LLM is simpler. See Choosing a Foundry Tool (or a Foundry Model).

Related pages

References