Azure.AI.VoiceLive Namespace

Configuration for animation outputs including blendshapes and visemes metadata.

The AssistantMessageItem.

Echo cancellation configuration for server-side audio processing.

Configuration for input audio transcription.

Configuration for input audio noise reduction.

Configuration for avatar streaming and behavior during the session.

Context class which will be filled in by the System.ClientModel.SourceGeneration. For more information https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/core/System.ClientModel/src/docs/ModelReaderWriterContext.md

Azure custom voice configuration.

Azure personal voice configuration.

Azure semantic end-of-utterance detection (default).

Azure semantic end-of-utterance detection (default).

Azure semantic end-of-utterance detection (default).

Base model for VAD-based turn detection.

Base model for VAD-based turn detection.

Base model for VAD-based turn detection.

Azure standard voice configuration.

Base for Azure voice configurations. Please note this is the abstract base class. The derived classes available for instantiation are: AzureCustomVoice, AzureStandardVoice, and AzurePersonalVoice.

Details of output token usage.

Base for any response item; discriminated by type. Please note this is the abstract base class. The derived classes available for instantiation are: MessageItem, FunctionCallItem, and FunctionCallOutputItem.

Top-level union for end-of-utterance (EOU) semantic detection configuration. Please note this is the abstract base class. The derived classes available for instantiation are: AzureSemanticEouDetection, AzureSemanticEouDetectionEn, and AzureSemanticEouDetectionMultilingual.

A function call item within a conversation.

A function call output item within a conversation.

ICE server configuration for WebRTC connection negotiation.

Input audio content part.

Input text content part.

Details of input token usage.

A single log probability entry for a token.

Base for any message content part; discriminated by type. Please note this is the abstract base class. The derived classes available for instantiation are: InputTextContentPart, InputAudioContentPart, and OutputTextContentPart.

A message item within a conversation.

Disables turn detection.

OpenAI voice configuration with explicit type field.

This provides a unified interface for OpenAI voices, complementing the existing string-based OAIVoice for backward compatibility.

Output text content part.

Details of output token usage.

An audio content part for a request.

A text content part for a request.

An audio content part for a response.

Details for a cancelled response.

Details for a failed response.

A function call item within a conversation.

A function call output item within a conversation.

Details for an incomplete response.

Base for all non-success response details. Please note this is the abstract base class. The derived classes available for instantiation are: ResponseCancelledDetails, ResponseIncompleteDetails, and ResponseFailedDetails.

A text content part for a response.

Overall usage statistics for a response.

Base model for VAD-based turn detection.

The response resource.

Base for any response item; discriminated by type. Please note this is the abstract base class. The derived classes available for instantiation are: SessionResponseMessageItem, ResponseFunctionCallItem, and ResponseFunctionCallOutputItem.

Base type for message item within a conversation.

A voicelive server event. Please note this is the abstract base class. The derived classes available for instantiation are: SessionUpdateError, SessionUpdateSessionCreated, SessionUpdateSessionUpdated, SessionUpdateAvatarConnecting, SessionUpdateInputAudioBufferCommitted, SessionUpdateInputAudioBufferCleared, SessionUpdateInputAudioBufferSpeechStarted, SessionUpdateInputAudioBufferSpeechStopped, SessionUpdateConversationItemCreated, SessionUpdateConversationItemInputAudioTranscriptionCompleted, SessionUpdateConversationItemInputAudioTranscriptionFailed, SessionUpdateConversationItemTruncated, SessionUpdateConversationItemDeleted, SessionUpdateResponseCreated, SessionUpdateResponseDone, SessionUpdateResponseOutputItemAdded, SessionUpdateResponseOutputItemDone, SessionUpdateResponseContentPartAdded, SessionUpdateResponseContentPartDone, SessionUpdateResponseTextDelta, SessionUpdateResponseTextDone, SessionUpdateResponseAudioTranscriptDelta, SessionUpdateResponseAudioTranscriptDone, SessionUpdateResponseAudioDelta, SessionUpdateResponseAudioDone, SessionUpdateResponseAnimationBlendshapeDelta, SessionUpdateResponseAnimationBlendshapeDone, SessionUpdateResponseAudioTimestampDelta, SessionUpdateResponseAudioTimestampDone, SessionUpdateResponseAnimationVisemeDelta, SessionUpdateResponseAnimationVisemeDone, SessionUpdateConversationItemInputAudioTranscriptionDelta, SessionUpdateConversationItemRetrieved, SessionUpdateResponseFunctionCallArgumentsDelta, and SessionUpdateResponseFunctionCallArgumentsDone.

Sent when the server is in the process of establishing an avatar media connection and provides its SDP answer.

Returned when a conversation item is created. There are several scenarios that produce this event:

• The server is generating a Response, which if successful will produce either one or two Items, which will be of type message (role assistant) or type function_call.
• The input audio buffer has been committed, either by the client or the server (in server_vad mode). The server will take the content of the input audio buffer and add it to a new user message Item.
• The client has sent a conversation.item.create event to add a new Item to the Conversation.

Returned when an item in the conversation is deleted by the client with a conversation.item.delete event. This event is used to synchronize the server's understanding of the conversation history with the client's view.

This event is the output of audio transcription for user audio written to the user audio buffer. Transcription begins when the input audio buffer is committed by the client or server (in server_vad mode). Transcription runs asynchronously with Response creation, so this event may come before or after the Response events. VoiceLive API models accept audio natively, and thus input transcription is a separate process run on a separate ASR (Automatic Speech Recognition) model. The transcript may diverge somewhat from the model's interpretation, and should be treated as a rough guide.

Returned when the text value of an input audio transcription content part is updated.

Returned when input audio transcription is configured, and a transcription request for a user message failed. These events are separate from other error events so that the client can identify the related Item.

Returned when a conversation item is retrieved with conversation.item.retrieve.

Returned when an earlier assistant audio message item is truncated by the client with a conversation.item.truncate event.

Returned when an error occurs, which could be a client problem or a server problem. Most errors are recoverable and the session will stay open, we recommend to implementors to monitor and log error messages by default.

Details of the error.

Returned when the input audio buffer is cleared by the client with a input_audio_buffer.clear event.

Returned when an input audio buffer is committed, either by the client or automatically in server VAD mode. The item_id property is the ID of the user message item that will be created, thus a conversation.item.created event will also be sent to the client.

The SessionUpdateInputAudioBufferSpeechStarted.

The SessionUpdateInputAudioBufferSpeechStopped.

Represents a delta update of blendshape animation frames for a specific output of a response.

Indicates the completion of blendshape animation processing for a specific output of a response.

Represents a viseme ID delta update for animation based on audio.

Indicates completion of viseme animation delivery for a response.

Returned when the model-generated audio is updated.

Returned when the model-generated audio is done. Also emitted when a Response is interrupted, incomplete, or cancelled.

Represents a word-level audio timestamp delta for a response.

Indicates completion of audio timestamp delivery for a response.

Returned when the model-generated transcription of audio output is updated.

Returned when the model-generated transcription of audio output is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

Returned when a new content part is added to an assistant message item during response generation.

Returned when a content part is done streaming in an assistant message item. Also emitted when a Response is interrupted, incomplete, or cancelled.

Returned when a new Response is created. The first event of response creation, where the response is in an initial state of in_progress.

Returned when a Response is done streaming. Always emitted, no matter the final state. The Response object included in the response.done event will include all output Items in the Response but will omit the raw audio data.

Returned when the model-generated function call arguments are updated.

Returned when the model-generated function call arguments are done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

Returned when a new Item is created during Response generation.

Returned when an Item is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

Returned when the text value of a "text" content part is updated.

Returned when the text value of a "text" content part is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

Returned when a Session is created. Emitted automatically when a new connection is established as the first server event. This event will contain the default Session configuration.

Returned when a session is updated with a session.update event, unless there is an error.

The SystemMessageItem.

Represents constraints placed on tool calls made by the model.

Top-level union for turn detection configuration. Please note this is the abstract base class. The derived classes available for instantiation are: ServerVadTurnDetection, AzureSemanticVadTurnDetection, AzureSemanticVadTurnDetectionEn, and AzureSemanticVadTurnDetectionMultilingual.

The UserMessageItem.

Defines a video background, either a solid color or an image URL (mutually exclusive).

Defines a video crop rectangle.

Video streaming parameters for avatar.

Resolution of the video feed in pixels.

The VoiceLiveClient.

Client options for VoiceLiveClient.

Base for any content part; discriminated by type. Please note this is the abstract base class. The derived classes available for instantiation are: RequestTextContentPart, RequestAudioContentPart, ResponseTextContentPart, and ResponseAudioContentPart.

Error object returned in case of API failure.

The definition of a function tool as used by the voicelive endpoint.

A factory class for creating instances of the models for mocking.

The response resource.

Represents a WebSocket-based session for real-time voice communication with the Azure VoiceLive service.

The VoiceLiveRequestSession.

Base for session configuration in the response.

The base representation of a voicelive tool definition. Please note this is the abstract base class. The derived classes available for instantiation are: VoiceLiveFunctionDefinition.

Base interface for the different voice types supported by the VoiceLive service

Specifies the types of animation data to output.

Output timestamp types supported in audio response content.

Threshold level settings for Azure semantic end-of-utterance detection.

Input audio format types supported.

Supported modalities for the session.

Indicates the processing status of an item or parameter.

Supported OpenAI voice names (string enum).

Output audio format types supported.

PersonalVoice models.

Indicates the processing status of a response item.

Terminal status of a response.

The available set of mode-level, string literal tool_choice options for the voicelive endpoint.

The version of the service to use.