> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/kyutai-labs/unmute/llms.txt
> Use this file to discover all available pages before exploring further.

# WebSocket Overview

> Real-time WebSocket API for voice conversations in Unmute

## Introduction

Unmute uses a WebSocket-based protocol inspired by the [OpenAI Realtime API](https://platform.openai.com/docs/api-reference/realtime) for real-time voice conversations. The protocol enables bidirectional streaming of audio, transcriptions, and conversation state.

## Connection Details

### Endpoint

```
ws://localhost:8000/v1/realtime
```

**WebSocket Subprotocol**: `realtime`

The `realtime` subprotocol is required. Clients must specify this when establishing the connection, otherwise the server will reject the connection.

### Example Connection (JavaScript)

```javascript theme={null}
const ws = new WebSocket(
  'ws://localhost:8000/v1/realtime',
  'realtime'
);

ws.onopen = () => {
  console.log('Connected to Unmute');
};

ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
  console.log('Received:', message.type);
};
```

## Message Format

All messages are JSON-encoded with a common structure:

```json theme={null}
{
  "type": "event.name",
  "event_id": "event_ABC123xyz",
  // ... additional fields specific to event type
}
```

<ParamField path="type" type="string" required>
  The event type identifier (e.g., `session.update`, `response.audio.delta`)
</ParamField>

<ParamField path="event_id" type="string" required>
  Unique identifier for the event, automatically generated with format `event_` followed by 21 random alphanumeric characters
</ParamField>

## Connection Lifecycle

### 1. Health Check (Optional)

Before connecting, check server health:

```bash theme={null}
curl http://localhost:8000/v1/health
```

**Response**:

```json theme={null}
{
  "tts_up": true,
  "stt_up": true,
  "llm_up": true,
  "voice_cloning_up": true,
  "ok": true
}
```

### 2. Establish WebSocket Connection

Connect to `/v1/realtime` with the `realtime` subprotocol.

### 3. Configure Session

Send a `session.update` event to configure the voice and instructions. **The backend will not start processing until it receives this message**.

```json theme={null}
{
  "type": "session.update",
  "session": {
    "instructions": {
      "character": "helpful assistant",
      "scenario": "general conversation"
    },
    "voice": "default",
    "allow_recording": false
  }
}
```

### 4. Stream Audio

Begin sending `input_audio_buffer.append` events with microphone audio and receive `response.audio.delta` events with generated speech.

### 5. Graceful Shutdown

Close the WebSocket connection when done. The server handles cleanup automatically.

## Audio Format

All audio is encoded using the **Opus codec** with the following specifications:

* **Sample Rate**: 24 kHz
* **Channels**: Mono
* **Encoding**: Base64-encoded Opus bytes

Both client audio (sent to server) and server audio (received from server) use this format.

## Rate Limiting

The server limits concurrent connections to **4 clients** by default. If the limit is reached, the connection will be rejected with an error message.

## Error Handling

The server sends `error` events when issues occur. See [Server Events](/api/server-events#error) for details.

Common error scenarios:

* Invalid JSON format
* Unrecognized event types
* Service unavailability
* Internal server errors

## Next Steps

<CardGroup cols={2}>
  <Card title="Client Events" icon="arrow-up" href="/api/client-events">
    Events sent from client to server
  </Card>

  <Card title="Server Events" icon="arrow-down" href="/api/server-events">
    Events sent from server to client
  </Card>

  <Card title="Session Management" icon="gear" href="/api/session-management">
    Configure voice and conversation settings
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.