> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/kyutai-labs/unmute/llms.txt
> Use this file to discover all available pages before exploring further.

# UnmuteHandler

> Main async stream handler that orchestrates speech-to-text, text-to-speech, and LLM components for real-time voice conversations

## Overview

The `UnmuteHandler` class is the core component of the Unmute system that manages the complete voice conversation pipeline. It inherits from `AsyncStreamHandler` and coordinates audio streaming, speech recognition, language model generation, and text-to-speech synthesis.

## Class Definition

```python theme={null}
class UnmuteHandler(AsyncStreamHandler)
```

### Constructor

```python theme={null}
def __init__(self) -> None
```

Initializes the UnmuteHandler with default configuration:

* **input\_sample\_rate**: 24000 Hz
* **output\_frame\_size**: 480 samples
* **output\_sample\_rate**: 24000 Hz

## Properties

### stt

```python theme={null}
@property
def stt(self) -> SpeechToText | None
```

Returns the current SpeechToText instance if available, otherwise None.

**Returns:** `SpeechToText | None`

### tts

```python theme={null}
@property
def tts(self) -> TextToSpeech | None
```

Returns the current TextToSpeech instance if available, otherwise None.

**Returns:** `TextToSpeech | None`

## Core Methods

### receive

```python theme={null}
async def receive(self, frame: tuple[int, np.ndarray]) -> None
```

Processes incoming audio frames from the user's microphone.

<ParamField path="frame" type="tuple[int, np.ndarray]" required>
  Audio frame tuple containing sample rate and mono audio array
</ParamField>

**Behavior:**

* Sends audio to STT for transcription
* Detects pauses in user speech
* Handles bot interruptions
* Manages conversation state transitions

### emit

```python theme={null}
async def emit(self) -> HandlerOutput | None
```

Returns output to be sent to the client (audio, events, or additional outputs).

**Returns:** `tuple[int, np.ndarray] | AdditionalOutputs | ora.ServerEvent | CloseStream | None`

### start\_up

```python theme={null}
async def start_up(self)
```

Initializes the handler by starting up the STT service and preparing for audio processing.

### cleanup

```python theme={null}
async def cleanup()
```

Cleans up resources, including shutting down the recorder if enabled.

## Conversation Management

### add\_chat\_message\_delta

```python theme={null}
async def add_chat_message_delta(
    self,
    delta: str,
    role: Literal["user", "assistant"],
    generating_message_i: int | None = None,
) -> bool
```

Adds a message delta to the chat history.

<ParamField path="delta" type="str" required>
  Text to add to the current message
</ParamField>

<ParamField path="role" type="Literal['user', 'assistant']" required>
  Role of the message sender
</ParamField>

<ParamField path="generating_message_i" type="int | None">
  Index of the message being generated (to avoid race conditions)
</ParamField>

**Returns:** `bool` - True if this created a new message, False if appended to existing

### interrupt\_bot

```python theme={null}
async def interrupt_bot()
```

Interrupts the bot while it's speaking, clearing queues and stopping TTS/LLM tasks.

**Raises:** `RuntimeError` if called when conversation state is not "bot\_speaking"

### update\_session

```python theme={null}
async def update_session(self, session: ora.SessionConfig)
```

Updates session configuration including instructions, voice, and recording preferences.

<ParamField path="session" type="ora.SessionConfig" required>
  Session configuration object
</ParamField>

## Utility Methods

### audio\_received\_sec

```python theme={null}
def audio_received_sec(self) -> float
```

Returns how much audio has been received in seconds. Used for timing instead of wall-clock time.

**Returns:** `float` - Seconds of audio received

### get\_gradio\_update

```python theme={null}
def get_gradio_update(self) -> AdditionalOutputs
```

Returns debug information and chat history for UI updates.

**Returns:** `AdditionalOutputs` containing `GradioUpdate` with:

* `chat_history`: List of message dictionaries
* `debug_dict`: Debug information dictionary
* `debug_plot_data`: Plot data for visualizations

### copy

```python theme={null}
def copy(self)
```

Creates a new instance of UnmuteHandler.

**Returns:** `UnmuteHandler`

## Internal Methods

### determine\_pause

```python theme={null}
def determine_pause(self) -> bool
```

Determines if the user has paused speaking based on STT pause prediction.

**Returns:** `bool` - True if pause detected

### detect\_long\_silence

```python theme={null}
async def detect_long_silence()
```

Handles situations where the user doesn't respond for more than 7 seconds by inserting a silence marker.

### check\_for\_bot\_goodbye

```python theme={null}
async def check_for_bot_goodbye()
```

Checks if the assistant's last message ends with "bye!" and closes the stream if so.

## Configuration Constants

```python theme={null}
USER_SILENCE_TIMEOUT = 7.0  # seconds
FIRST_MESSAGE_TEMPERATURE = 0.7
FURTHER_MESSAGES_TEMPERATURE = 0.3
UNINTERRUPTIBLE_BY_VAD_TIME_SEC = 3  # seconds
```

## Example Usage

```python theme={null}
import asyncio
from unmute.unmute_handler import UnmuteHandler

async def main():
    handler = UnmuteHandler()
    
    async with handler:
        await handler.start_up()
        
        # Handler is now ready to process audio frames
        # Audio frames would be received via the receive() method
        # and outputs retrieved via emit()
        
        await handler.cleanup()

asyncio.run(main())
```

## Notes

* The handler uses a quest manager system to coordinate async tasks for STT, TTS, and LLM
* Conversation state is managed through the internal `Chatbot` instance
* Audio is processed at 24kHz sample rate with mono channel
* The handler supports echo cancellation and VAD-based interruption detection


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.