> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/kyutai-labs/unmute/llms.txt
> Use this file to discover all available pages before exploring further.

# Client Events

> Events sent from client to server via WebSocket

## Overview

Client events are messages sent from the frontend to the Unmute backend server. These events control session configuration and stream audio data.

## session.update

Configure the conversation session, including voice selection and conversation instructions.

**The backend requires this event before it begins processing**. Send this immediately after connecting.

### Parameters

<ParamField path="type" type="string" required>
  Must be `"session.update"`
</ParamField>

<ParamField path="event_id" type="string" required>
  Unique event identifier (auto-generated)
</ParamField>

<ParamField path="session" type="object" required>
  Session configuration object

  <ParamField path="session.instructions" type="object">
    Conversation instructions (Unmute extension)

    <ParamField path="session.instructions.character" type="string">
      Character personality and behavior
    </ParamField>

    <ParamField path="session.instructions.scenario" type="string">
      Conversation scenario or context
    </ParamField>
  </ParamField>

  <ParamField path="session.voice" type="string">
    Voice identifier for text-to-speech. Get available voices from `/v1/voices` endpoint.
  </ParamField>

  <ParamField path="session.allow_recording" type="boolean" required>
    Whether to allow recording of the conversation. Set to `false` to disable recording.
  </ParamField>
</ParamField>

### Example

```json theme={null}
{
  "type": "session.update",
  "event_id": "event_ABC123xyz",
  "session": {
    "instructions": {
      "character": "You are a helpful AI assistant.",
      "scenario": "Casual conversation"
    },
    "voice": "default",
    "allow_recording": false
  }
}
```

### Response

The server responds with a `session.updated` event confirming the configuration.

***

## input\_audio\_buffer.append

Stream audio data from the user's microphone to the server.

### Parameters

<ParamField path="type" type="string" required>
  Must be `"input_audio_buffer.append"`
</ParamField>

<ParamField path="event_id" type="string" required>
  Unique event identifier (auto-generated)
</ParamField>

<ParamField path="audio" type="string" required>
  Base64-encoded Opus audio data

  **Audio Specifications:**

  * Codec: Opus
  * Sample Rate: 24 kHz
  * Channels: Mono
  * Encoding: Base64 string
</ParamField>

### Example

```json theme={null}
{
  "type": "input_audio_buffer.append",
  "event_id": "event_XYZ789abc",
  "audio": "T2dnUwACAAAAAAAAAADqnjMlAAAAAP4lQ6gBE09w..." 
}
```

### Implementation Notes

* The server decodes the base64 audio and processes it through an Opus stream reader
* The first packet must have the "beginning of stream" bit set (bit 2 in byte 5)
* Audio is processed in real-time for speech detection and transcription
* The server may send `input_audio_buffer.speech_started` when speech is detected

### JavaScript Example

```javascript theme={null}
// Capture microphone audio
const stream = await navigator.mediaDevices.getUserMedia({ 
  audio: {
    sampleRate: 24000,
    channelCount: 1
  } 
});

const mediaRecorder = new MediaRecorder(stream, {
  mimeType: 'audio/webm;codecs=opus',
  audioBitsPerSecond: 16000
});

mediaRecorder.ondataavailable = (event) => {
  // Convert to base64 and send
  const reader = new FileReader();
  reader.onloadend = () => {
    const base64Audio = reader.result.split(',')[1];
    ws.send(JSON.stringify({
      type: 'input_audio_buffer.append',
      audio: base64Audio
    }));
  };
  reader.readAsDataURL(event.data);
};

mediaRecorder.start(100); // Send every 100ms
```

***

## Error Handling

If the client sends invalid messages, the server responds with an `error` event:

### Invalid JSON

```json theme={null}
{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "Invalid JSON: Expecting value: line 1 column 1 (char 0)"
  }
}
```

### Invalid Message Structure

```json theme={null}
{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "Invalid message",
    "details": [
      {
        "type": "missing",
        "loc": ["session"],
        "msg": "Field required"
      }
    ]
  }
}
```

## Next Steps

<CardGroup cols={2}>
  <Card title="Server Events" icon="arrow-down" href="/api/server-events">
    Learn about events sent from server to client
  </Card>

  <Card title="Session Management" icon="gear" href="/api/session-management">
    Configure voice and conversation settings
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.