# H3 Max

> H3 Max API endpoint


## Overview

- **Endpoint**: `https://fal.run/minimax/h3-max/reference-director/start-session`
- **Model ID**: `minimax/h3-max/reference-director/start-session`
- **Category**: unknown
- **Kind**: inference


## Pricing

- **Price**: $0 per compute seconds

For more details, see [fal.ai pricing](https://fal.ai/pricing).

## Realtime API (WMA/WebRTC)

This endpoint publishes a WMA WebRTC contract. Open it with `fal.realtime.open`; do **not** call the session endpoint with `fal.run`, `fal.subscribe`, or the Queue API.

The browser negotiates through fal and then exchanges media with the model over WebRTC.

Client contract for the WebRTC session created by the linked OpenAPI operation.


### Install the experimental client

```bash
npm install @fal-ai/client@alpha @fal-ai/server-proxy@alpha
```

### Authentication and server proxy

Keep `FAL_KEY` on your server. Expose an authenticated proxy route to the browser. In a Next.js App Router application, create `app/api/fal/proxy/route.ts`:

```typescript
import { route } from "@fal-ai/server-proxy/nextjs";

export const { GET, POST, PUT } = route;
```

Protect that route with your application's authentication before deploying it.

### Open a browser session

```typescript
import { createFalClient } from "@fal-ai/client";
import { wma } from "@fal-ai/client/realtime";

const fal = createFalClient({ proxyUrl: "/api/fal/proxy" });

const session = fal.realtime.open(wma("minimax/h3-max/reference-director"), {
  receive: [
    "video",
    "audio"
  ],
  onMedia: (stream) => {
      const element = document.querySelector("video");
      if (element) element.srcObject = stream;
    },
  onData: (raw) => {
      const message = JSON.parse(raw);
      console.log(message);
    },
  onState: (state) => console.log("realtime:", state),
  onError: (error) => console.error(error),
});


// Send any client message described below.
session.send({
  "protocol_version": 1,
  "reference_audio": [],
  "prompt": "A traveler explores",
  "dialogue_words_per_second": 2,
  "reference_images": [],
  "prompt_version": 1,
  "resolution": "480p",
  "num_inference_steps": 10,
  "aspect_ratio": "16:9",
  "reference_image_urls": [],
  "type": "configure",
  "mxfp8_ref2va_attention": false,
  "reference_audio_urls": [],
  "pin_audio": false,
  "memory": 12
});

// Later, release the peer connection and network resources:
// await session.close();
```


## Media contract

Tracks are described from the browser's perspective.

### Browser sends to the model

None.

### Browser receives from the model

- `video` — optional
  - Settings: `{"frameRate":24}`
- `audio` — optional

## Client-to-model control messages

### Client configure message

- **Wire type**: `configure`
- **Payload**: `object`

- **`prompt`** (`string`, _required_)
  - Constraints: minLength=1, maxLength=50000
- **`resolution`** (`string`, _optional_)
  - Default: `"480p"`
  - Values: `"480p"`, `"544p"`, `"640p"`, `"704p"`, `"768p"`, `"1080p"`, `"1440p"`
- **`aspect_ratio`** (`string`, _optional_)
  - Default: `"16:9"`
  - Values: `"16:9"`, `"9:16"`, `"1:1"`
- **`reference_images`** (`(object & object)[]`, _optional_)
  - Default: `[]`
  - Constraints: maxItems=9
  - Array items: `object & object`
    - allOf alternatives:
      1. `object`
        - **`name`** (`string | null`, _optional_)
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Constraints: maxLength=64
            2. `null`
        - **`url`** (`string`, _required_)
          - Constraints: minLength=1
        - **`fidelity`** (`string | null`, _optional_)
          Vision-conditioning detail for image references: high (768px), medium (512px) or low (256px). Omitted keeps the reference's previous fidelity, or high for a new reference. Ignored for audio references.
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Values: `"high"`, `"medium"`, `"low"`
            2. `null`
        - **`id`** (`string | null`, _optional_)
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Constraints: pattern="^ref-[1-9][0-9]*$"
            2. `null`
        - Additional properties: not allowed
      2. `object`
        - **`url`** (`unknown`, _optional_)
- **`reference_image_urls`** (`string[]`, _optional_)
  - Default: `[]`
  - Constraints: maxItems=9
  - Array items: `string`
- **`script`** (`object[] | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `object[]`
      - Constraints: minItems=1, maxItems=64
      - Array items: `object`
        - **`offset`** (`number`, _required_)
          Seconds from the start of the first video generated under this script; fractions are allowed and quantize to the 24 fps frame grid. Text and audio placed here start at this moment. A beat carrying reference_images makes those identities available from the whole second at or before this offset, so its own prompt can show them immediately.
          - Constraints: minimum=0
        - **`prompt`** (`string | null`, _optional_)
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Constraints: minLength=1, maxLength=50000
            2. `null`
        - **`audio_url`** (`string | null`, _optional_)
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Constraints: minLength=1
            2. `null`
        - **`reference_images`** (`(object & object)[]`, _optional_)
          - Default: `[]`
          - Constraints: maxItems=9
          - Array items: `object & object`
            - allOf alternatives:
              1. `object`
                - **`name`** (`string | null`, _optional_)
                  - Default: `null`
                  - anyOf alternatives:
                    1. `string`
                      - Constraints: maxLength=64
                    2. `null`
                - **`url`** (`string`, _required_)
                  - Constraints: minLength=1
                - **`fidelity`** (`string | null`, _optional_)
                  Vision-conditioning detail for image references: high (768px), medium (512px) or low (256px). Omitted keeps the reference's previous fidelity, or high for a new reference. Ignored for audio references.
                  - Default: `null`
                  - anyOf alternatives:
                    1. `string`
                      - Values: `"high"`, `"medium"`, `"low"`
                    2. `null`
                - **`id`** (`string | null`, _optional_)
                  - Default: `null`
                  - anyOf alternatives:
                    1. `string`
                      - Constraints: pattern="^ref-[1-9][0-9]*$"
                    2. `null`
                - Additional properties: not allowed
              2. `object`
                - **`url`** (`unknown`, _optional_)
        - Additional properties: not allowed
    2. `null`
- **`reference_audio`** (`(object & object)[]`, _optional_)
  - Default: `[]`
  - Constraints: maxItems=3
  - Array items: `object & object`
    - allOf alternatives:
      1. `object`
        - **`name`** (`string | null`, _optional_)
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Constraints: maxLength=64
            2. `null`
        - **`url`** (`string`, _required_)
          - Constraints: minLength=1
        - **`fidelity`** (`string | null`, _optional_)
          Vision-conditioning detail for image references: high (768px), medium (512px) or low (256px). Omitted keeps the reference's previous fidelity, or high for a new reference. Ignored for audio references.
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Values: `"high"`, `"medium"`, `"low"`
            2. `null`
        - **`id`** (`string | null`, _optional_)
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Constraints: pattern="^ref-[1-9][0-9]*$"
            2. `null`
        - Additional properties: not allowed
      2. `object`
        - **`url`** (`unknown`, _optional_)
- **`reference_audio_urls`** (`string[]`, _optional_)
  - Default: `[]`
  - Constraints: maxItems=3
  - Array items: `string`
- **`pin_audio`** (`boolean`, _optional_)
  - Default: `false`
- **`memory`** (`integer`, _optional_)
  Number of prior segment prompts retained as context for future prompt expansion.
  - Default: `12`
  - Constraints: minimum=1, maximum=50
- **`audio_bitrate`** (`integer | null`, _optional_)
  Session audio target in bits/s: 96000, 128000, or 192000. Explicit values use Opus audio mode on direct WebRTC; LiveKit uses its native mode with the same bitrate target. Null preserves transport defaults (WebRTC: 96000, voip). Immutable; reconnect to compare.
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Values: `96000`, `128000`, `192000`
    2. `null`
- **`seed`** (`integer | null`, _optional_)
  - Default: `null`
- **`num_inference_steps`** (`integer`, _optional_)
  - Default: `10`
  - Constraints: minimum=1, maximum=50
- **`mxfp8_ref2va_attention`** (`boolean`, _optional_)
  Use MXFP8 attention for dense generation steps.
  - Default: `false`
- **`forced_cut_max_segments`** (`integer | null`, _optional_)
  Most segments in one unbroken continuation run before a forced hard cut opens a fresh shot (Ref2VA burn-in mitigation). 0 disables forced cuts; omitted keeps the deployment default. Pinned for the whole session.
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=0, maximum=64
    2. `null`
- **`dialogue_words_per_second`** (`number`, _optional_)
  Spoken words per second of new screen time the planner budgets dialogue against. The default keeps delivery calm; raise it for deliberately fast talkers, lower it for sparser dialogue.
  - Default: `2`
  - Constraints: minimum=0.5, maximum=6
- **`type`** (`string`, _required_)
  - Constant: `"configure"`
- **`protocol_version`** (`integer`, _optional_)
  - Constant: `1`
- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1
- Additional properties: not allowed

**Example**

```json
{
  "protocol_version": 1,
  "reference_audio": [],
  "prompt": "A traveler explores",
  "dialogue_words_per_second": 2,
  "reference_images": [],
  "prompt_version": 1,
  "resolution": "480p",
  "num_inference_steps": 10,
  "aspect_ratio": "16:9",
  "reference_image_urls": [],
  "type": "configure",
  "mxfp8_ref2va_attention": false,
  "reference_audio_urls": [],
  "pin_audio": false,
  "memory": 12
}
```

### Client ping message

- **Wire type**: `ping`
- **Payload**: `object`

- **`type`** (`string`, _required_)
  - Constant: `"ping"`
- **`ts`** (`number`, _required_)
- Additional properties: not allowed

**Example**

```json
{
  "type": "ping",
  "ts": 0
}
```

### Client prompt message

- **Wire type**: `prompt`
- **Payload**: `object`

- **`prompt`** (`string | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `string`
      - Constraints: minLength=1, maxLength=50000
    2. `null`
- **`reference_images`** (`(object & object)[] | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `(object & object)[]`
      - Constraints: maxItems=9
      - Array items: `object & object`
        - allOf alternatives:
          1. `object`
            - **`name`** (`string | null`, _optional_)
              - Default: `null`
              - anyOf alternatives:
                1. `string`
                  - Constraints: maxLength=64
                2. `null`
            - **`url`** (`string`, _required_)
              - Constraints: minLength=1
            - **`fidelity`** (`string | null`, _optional_)
              Vision-conditioning detail for image references: high (768px), medium (512px) or low (256px). Omitted keeps the reference's previous fidelity, or high for a new reference. Ignored for audio references.
              - Default: `null`
              - anyOf alternatives:
                1. `string`
                  - Values: `"high"`, `"medium"`, `"low"`
                2. `null`
            - **`id`** (`string | null`, _optional_)
              - Default: `null`
              - anyOf alternatives:
                1. `string`
                  - Constraints: pattern="^ref-[1-9][0-9]*$"
                2. `null`
            - Additional properties: not allowed
          2. `object`
            - **`url`** (`unknown`, _optional_)
    2. `null`
- **`reference_image_urls`** (`string[] | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `string[]`
      - Constraints: maxItems=9
      - Array items: `string`
    2. `null`
- **`script`** (`object[] | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `object[]`
      - Constraints: minItems=1, maxItems=64
      - Array items: `object`
        - **`offset`** (`number`, _required_)
          Seconds from the start of the first video generated under this script; fractions are allowed and quantize to the 24 fps frame grid. Text and audio placed here start at this moment. A beat carrying reference_images makes those identities available from the whole second at or before this offset, so its own prompt can show them immediately.
          - Constraints: minimum=0
        - **`prompt`** (`string | null`, _optional_)
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Constraints: minLength=1, maxLength=50000
            2. `null`
        - **`audio_url`** (`string | null`, _optional_)
          - Default: `null`
          - anyOf alternatives:
            1. `string`
              - Constraints: minLength=1
            2. `null`
        - **`reference_images`** (`(object & object)[]`, _optional_)
          - Default: `[]`
          - Constraints: maxItems=9
          - Array items: `object & object`
            - allOf alternatives:
              1. `object`
                - **`name`** (`string | null`, _optional_)
                  - Default: `null`
                  - anyOf alternatives:
                    1. `string`
                      - Constraints: maxLength=64
                    2. `null`
                - **`url`** (`string`, _required_)
                  - Constraints: minLength=1
                - **`fidelity`** (`string | null`, _optional_)
                  Vision-conditioning detail for image references: high (768px), medium (512px) or low (256px). Omitted keeps the reference's previous fidelity, or high for a new reference. Ignored for audio references.
                  - Default: `null`
                  - anyOf alternatives:
                    1. `string`
                      - Values: `"high"`, `"medium"`, `"low"`
                    2. `null`
                - **`id`** (`string | null`, _optional_)
                  - Default: `null`
                  - anyOf alternatives:
                    1. `string`
                      - Constraints: pattern="^ref-[1-9][0-9]*$"
                    2. `null`
                - Additional properties: not allowed
              2. `object`
                - **`url`** (`unknown`, _optional_)
        - Additional properties: not allowed
    2. `null`
- **`audio_behavior`** (`string`, _optional_)
  'replace' cuts to this audio at the next chunk; 'queue' appends it.
  - Default: `"replace"`
  - Values: `"replace"`, `"queue"`
- **`replan`** (`boolean`, _optional_)
  When a prompt is included: true (default) busts the planned prompt queue so the new direction applies at the next undispatched chunk; false appends the direction after the already-planned chunks.
  - Default: `true`
- **`reference_audio`** (`(object & object)[] | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `(object & object)[]`
      - Constraints: maxItems=3
      - Array items: `object & object`
        - allOf alternatives:
          1. `object`
            - **`name`** (`string | null`, _optional_)
              - Default: `null`
              - anyOf alternatives:
                1. `string`
                  - Constraints: maxLength=64
                2. `null`
            - **`url`** (`string`, _required_)
              - Constraints: minLength=1
            - **`fidelity`** (`string | null`, _optional_)
              Vision-conditioning detail for image references: high (768px), medium (512px) or low (256px). Omitted keeps the reference's previous fidelity, or high for a new reference. Ignored for audio references.
              - Default: `null`
              - anyOf alternatives:
                1. `string`
                  - Values: `"high"`, `"medium"`, `"low"`
                2. `null`
            - **`id`** (`string | null`, _optional_)
              - Default: `null`
              - anyOf alternatives:
                1. `string`
                  - Constraints: pattern="^ref-[1-9][0-9]*$"
                2. `null`
            - Additional properties: not allowed
          2. `object`
            - **`url`** (`unknown`, _optional_)
    2. `null`
- **`reference_audio_urls`** (`string[] | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `string[]`
      - Constraints: maxItems=3
      - Array items: `string`
    2. `null`
- **`type`** (`string`, _required_)
  - Constant: `"prompt"`
- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1
- **`script_mode`** (`string`, _optional_)
  'replace' cuts to the script at the next chunk; 'append' queues it.
  - Default: `"replace"`
  - Values: `"replace"`, `"append"`
- Additional properties: not allowed

**Example**

```json
{
  "script_mode": "replace",
  "reference_images": [
    {
      "url": "https://example.com/new-coat.jpg"
    }
  ],
  "type": "prompt",
  "prompt_version": 2,
  "audio_behavior": "replace",
  "replan": true
}
```

### Client stop message

- **Wire type**: `stop`
- **Payload**: `object`

- **`type`** (`string`, _required_)
  - Constant: `"stop"`
- Additional properties: not allowed

**Example**

```json
{
  "type": "stop"
}
```

## Model-to-client control messages

### Server audio_applied message

- **Wire type**: `audio_applied`
- **Payload**: `object`

An audio update was accepted; it conditions the next eligible chunk.

``starts_at_chunk_index`` is the first chunk that can carry it (the next
undispatched chunk for ``replace``; for ``queue`` the chunk where the
preceding sources end, if already known).

- **`transcribed`** (`boolean`, _required_)
- **`remaining_seconds`** (`number`, _required_)
  - Constraints: minimum=0
- **`type`** (`string`, _required_)
  - Constant: `"audio_applied"`
- **`behavior`** (`string`, _required_)
  - Values: `"replace"`, `"queue"`
- **`source`** (`string`, _required_)
- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1
- **`queued_sources`** (`integer`, _required_)
  - Constraints: minimum=0
- **`duration_seconds`** (`number`, _required_)
  - Constraints: exclusiveMinimum=0

**Example**

```json
{
  "transcribed": true,
  "remaining_seconds": 0,
  "type": "audio_applied",
  "behavior": "replace",
  "source": "string",
  "prompt_version": 1,
  "queued_sources": 0,
  "duration_seconds": 5e-324
}
```

### Server audio_exhausted message

- **Wire type**: `audio_exhausted`
- **Payload**: `object`

Accepted audio ran out inside this chunk; the rest is silence.

Later chunks continue without audio conditioning (checkpoint continuation
on the same family) until new audio is accepted.

- **`source_version`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=0
    2. `null`
- **`chunk_index`** (`integer`, _required_)
  - Constraints: minimum=0
- **`silent_seconds`** (`number`, _required_)
  - Constraints: minimum=0
- **`type`** (`string`, _required_)
  - Constant: `"audio_exhausted"`

**Example**

```json
{
  "chunk_index": 0,
  "silent_seconds": 0,
  "type": "audio_exhausted"
}
```

### Server audio_pending message

- **Wire type**: `audio_pending`
- **Payload**: `object`

- **`type`** (`string`, _required_)
  - Constant: `"audio_pending"`
- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1
- **`behavior`** (`string`, _required_)
  - Values: `"replace"`, `"queue"`

**Example**

```json
{
  "type": "audio_pending",
  "prompt_version": 1,
  "behavior": "replace"
}
```

### Server audio_rejected message

- **Wire type**: `audio_rejected`
- **Payload**: `object`

- **`reason`** (`string`, _required_)
  - Values: `"invalid_audio"`, `"content_policy"`, `"preparation_failed"`, `"queue_full"`, `"stale_prompt_version"`
- **`type`** (`string`, _required_)
  - Constant: `"audio_rejected"`
- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1
- **`error`** (`string`, _required_)

**Example**

```json
{
  "reason": "invalid_audio",
  "type": "audio_rejected",
  "prompt_version": 1,
  "error": "string"
}
```

### Server chunk message

- **Wire type**: `chunk`
- **Payload**: `object`

- **`route`** (`string`, _required_)
  - Constant: `"k10-reference"`
- **`scheduling_slack_ms`** (`number`, _required_)
- **`next_generation_estimate_seconds`** (`number`, _required_)
  - Constraints: minimum=0
- **`requested_duration_seconds`** (`integer`, _required_)
  - Constraints: minimum=5, maximum=15
- **`trimmed_context_frames`** (`integer`, _required_)
  - Constraints: minimum=0
- **`script_end_keyframe`** (`boolean | null`, _optional_)
  - Default: `null`
- **`hard_cut`** (`boolean`, _optional_)
  - Default: `false`
- **`native_playable_frame_count`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: exclusiveMinimum=0
    2. `null`
- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1
- **`buffer_depth_chunks`** (`integer`, _required_)
  - Constraints: minimum=0
- **`script_offset_seconds`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=0
    2. `null`
- **`script_version`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=1
    2. `null`
- **`type`** (`string`, _required_)
  - Constant: `"chunk"`
- **`dispatch`** (`object`, _required_)
  - **`classified_ms`** (`number`, _required_)
    - Constraints: minimum=0
  - **`wall_ms`** (`number`, _required_)
    - Constraints: minimum=0
  - **`phases_ms`** (`object`, _required_)
    - Additional properties: number
  - **`overhead_ms`** (`number`, _required_)
    - Constraints: minimum=0
- **`scheduling_lead_ms`** (`number`, _required_)
- **`generation_seconds`** (`number`, _required_)
  - Constraints: minimum=0
- **`generated_frame_count`** (`integer`, _required_)
  - Constraints: exclusiveMinimum=0
- **`buffer_depth_seconds`** (`number`, _required_)
  - Constraints: minimum=0
- **`presented_frame_count`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: exclusiveMinimum=0
    2. `null`
- **`chunk_index`** (`integer`, _required_)
  - Constraints: minimum=0
- **`playback_seconds`** (`number`, _required_)
  - Constraints: exclusiveMinimum=0

**Example**

```json
{
  "route": "k10-reference",
  "scheduling_slack_ms": 0,
  "next_generation_estimate_seconds": 0,
  "requested_duration_seconds": 5,
  "trimmed_context_frames": 0,
  "prompt_version": 1,
  "buffer_depth_chunks": 0,
  "type": "chunk",
  "dispatch": {
    "classified_ms": 0,
    "wall_ms": 0,
    "phases_ms": {},
    "overhead_ms": 0
  },
  "scheduling_lead_ms": 0,
  "generation_seconds": 0,
  "generated_frame_count": 1,
  "buffer_depth_seconds": 0,
  "chunk_index": 0,
  "playback_seconds": 5e-324
}
```

### Server chunk_metrics message

- **Wire type**: `chunk_metrics`
- **Payload**: `object`

- **`route`** (`string`, _required_)
  - Constant: `"k10-reference"`
- **`chunk_consumable_interval_ms`** (`number | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `number`
      - Constraints: minimum=0
    2. `null`
- **`type`** (`string`, _required_)
  - Constant: `"chunk_metrics"`
- **`phases_ms`** (`object`, _required_)
  - Additional properties: number
- **`units`** (`string`, _required_)
  - Constant: `"ms"`
- **`gauges`** (`object`, _required_)
  - Additional properties: number
- **`chunk_index`** (`integer`, _required_)
  - Constraints: minimum=0
- **`chunk_consumable_ready_ms`** (`number | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `number`
      - Constraints: minimum=0
    2. `null`

**Example**

```json
{
  "route": "k10-reference",
  "type": "chunk_metrics",
  "phases_ms": {},
  "units": "ms",
  "gauges": {},
  "chunk_index": 0
}
```

### Server configured message

- **Wire type**: `configured`
- **Payload**: `object`

- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1
- **`has_initial_image`** (`boolean | null`, _optional_)
  - Default: `null`
- **`aspect_ratio`** (`string | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `string`
      - Values: `"16:9"`, `"9:16"`, `"1:1"`
    2. `null`
- **`type`** (`string`, _required_)
  - Constant: `"configured"`
- **`acceleration`** (`string | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `string`
      - Values: `"none"`, `"regular"`
    2. `null`
- **`resolution`** (`string | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `string`
      - Values: `"480p"`, `"544p"`, `"640p"`, `"704p"`, `"768p"`, `"1080p"`, `"1440p"`
    2. `null`
- **`enable_safety_checker`** (`boolean`, _required_)
- **`memory`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=1, maximum=50
    2. `null`
- **`audio_bitrate`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Values: `96000`, `128000`, `192000`
    2. `null`
- **`chunk_duration`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=5, maximum=15
    2. `null`
- **`has_initial_audio`** (`boolean | null`, _optional_)
  - Default: `null`

**Example**

```json
{
  "prompt_version": 1,
  "type": "configured",
  "enable_safety_checker": true
}
```

### Server deadline_missed message

- **Wire type**: `deadline_missed`
- **Payload**: `object`

- **`late_by_seconds`** (`number`, _required_)
  - Constraints: minimum=0
- **`type`** (`string`, _required_)
  - Constant: `"deadline_missed"`
- **`chunk_index`** (`integer`, _required_)
  - Constraints: minimum=0
- **`behavior`** (`string`, _required_)
  - Constant: `"freeze_video_and_silence_audio_until_ready"`

**Example**

```json
{
  "late_by_seconds": 0,
  "type": "deadline_missed",
  "chunk_index": 0,
  "behavior": "freeze_video_and_silence_audio_until_ready"
}
```

### Server error message

- **Wire type**: `error`
- **Payload**: `object`

- **`detail`** (`object[] | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `object[]`
      - Array items: `object`
    2. `null`
- **`type`** (`string`, _required_)
  - Constant: `"error"`
- **`prompt_version`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=1
    2. `null`
- **`code`** (`string`, _required_)
  - Values: `"balance_unavailable"`, `"content_policy"`, `"configuration_timeout"`, `"generation_failed"`, `"generation_timeout"`, `"immutable_settings"`, `"initialization_timeout"`, `"invalid_initial_image"`, `"invalid_initial_audio"`, `"invalid_initial_script"`, `"invalid_input"`, `"invalid_message"`, `"not_configured"`, `"stale_prompt_version"`
- **`error`** (`string`, _required_)

**Example**

```json
{
  "type": "error",
  "code": "balance_unavailable",
  "error": "string"
}
```

### Server pong message

- **Wire type**: `pong`
- **Payload**: `object`

- **`client_ts`** (`number`, _required_)
- **`type`** (`string`, _required_)
  - Constant: `"pong"`

**Example**

```json
{
  "client_ts": 0,
  "type": "pong"
}
```

### Server prompt_applied message

- **Wire type**: `prompt_applied`
- **Payload**: `object`

A direction (or script) is admitted; the script fields report scripts.

``script_origin_chunk_index`` is the first chunk generated under the
script when that is already fixed (a replacement binds to the next
undispatched chunk); an appended script reports ``None`` until it starts.

- **`script_origin_chunk_index`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=0
    2. `null`
- **`script_mode`** (`string | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `string`
      - Values: `"replace"`, `"append"`
    2. `null`
- **`script_beats`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=1
    2. `null`
- **`type`** (`string`, _required_)
  - Constant: `"prompt_applied"`
- **`script_queued`** (`integer | null`, _optional_)
  - Default: `null`
  - anyOf alternatives:
    1. `integer`
      - Constraints: minimum=0
    2. `null`
- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1

**Example**

```json
{
  "type": "prompt_applied",
  "prompt_version": 1
}
```

### Server prompt_pending message

- **Wire type**: `prompt_pending`
- **Payload**: `object`

- **`type`** (`string`, _required_)
  - Constant: `"prompt_pending"`
- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1

**Example**

```json
{
  "type": "prompt_pending",
  "prompt_version": 1
}
```

### Server prompt_rejected message

- **Wire type**: `prompt_rejected`
- **Payload**: `object`

- **`reason`** (`string`, _required_)
  - Values: `"content_policy"`, `"preparation_failed"`, `"stale_prompt_version"`, `"invalid_script"`, `"infeasible_timing"`, `"invalid_audio"`, `"invalid_image"`, `"queue_full"`
- **`type`** (`string`, _required_)
  - Constant: `"prompt_rejected"`
- **`prompt_version`** (`integer`, _required_)
  - Constraints: minimum=1
- **`error`** (`string | null`, _optional_)
  - Default: `null`

**Example**

```json
{
  "reason": "content_policy",
  "type": "prompt_rejected",
  "prompt_version": 1
}
```

### Server reference_state message

- **Wire type**: `reference_state`
- **Payload**: `object`

- **`references`** (`object[]`, _required_)
  - Array items: `object`
    - **`name`** (`string | null`, _optional_)
      - Default: `null`
    - **`id`** (`string`, _required_)
    - **`kind`** (`string`, _required_)
      - Values: `"image"`, `"audio"`
    - **`revision`** (`integer`, _required_)
- **`type`** (`string`, _required_)
  - Constant: `"reference_state"`
- **`prompt_version`** (`integer`, _required_)

**Example**

```json
{
  "references": [],
  "type": "reference_state",
  "prompt_version": 0
}
```

### Server session_info message

- **Wire type**: `session_info`
- **Payload**: `object`

- **`billable_units_per_playable_second`** (`object`, _optional_)
  - Default: `{"704p":1.15,"544p":1.04,"640p":1.1,"1080p":1.2,"1440p":1.2,"768p":1.2,"480p":1}`
  - Additional properties: number
- **`protocol_version`** (`integer`, _optional_)
  - Constant: `1`
- **`fps`** (`integer`, _optional_)
  - Default: `24`
- **`app`** (`string`, _optional_)
  - Constant: `"minimax-h3-max-reference-director"`
- **`forced_cut_max_segments`** (`integer`, _optional_)
- **`max_chunk_duration`** (`integer`, _optional_)
  - Default: `15`
- **`resolutions`** (`string[]`, _optional_)
  - Default: `["480p","544p","640p","704p","768p","1080p","1440p"]`
  - Array items: `string`
    - Values: `"480p"`, `"544p"`, `"640p"`, `"704p"`, `"768p"`, `"1080p"`, `"1440p"`
- **`scripts`** (`boolean`, _optional_)
  - Default: `true`
- **`dialogue_words_per_second`** (`number`, _optional_)
  - Default: `2`
- **`accelerations`** (`string[]`, _optional_)
  - Default: `["none"]`
  - Array items: `string`
- **`pricing_status`** (`string`, _optional_)
  - Default: `"provisional-experiment"`
- **`one_session_per_machine`** (`boolean`, _optional_)
  - Default: `true`
- **`prompt_deck_size`** (`integer`, _optional_)
  - Default: `6`
- **`max_session_seconds`** (`number | null`, _optional_)
  - Default: `null`
- **`continuation_context_frames`** (`integer`, _optional_)
  - Default: `39`
- **`session_limit_scope`** (`string`, _optional_)
  - Default: `"configured"`
- **`type`** (`string`, _required_)
  - Constant: `"session_info"`
- **`reference_audio_mode`** (`string`, _optional_)
  - Default: `"sequential-content-windows"`
- **`default_pin_audio`** (`boolean`, _optional_)
  - Default: `false`
- **`max_num_inference_steps`** (`integer`, _optional_)
  - Default: `50`
- **`default_chunk_duration`** (`integer`, _optional_)
  - Default: `10`
- **`default_num_inference_steps`** (`integer`, _optional_)
  - Default: `10`
- **`min_num_inference_steps`** (`integer`, _optional_)
  - Default: `1`
- **`controller_machine_type`** (`string`, _optional_)
  - Default: `"GPU-B300/GPU-B200 (8 GPUs)"`
- **`backend_selection`** (`string`, _optional_)
  - Default: `"resident-k10-reference"`
- **`min_chunk_duration`** (`integer`, _optional_)
  - Default: `5`
- **`default_resolution`** (`string`, _optional_)
  - Default: `"480p"`
  - Values: `"480p"`, `"544p"`, `"640p"`, `"704p"`, `"768p"`, `"1080p"`, `"1440p"`

**Example**

```json
{
  "protocol_version": 1,
  "app": "minimax-h3-max-reference-director",
  "type": "session_info"
}
```

### Server session_metrics message

- **Wire type**: `session_metrics`
- **Payload**: `object`

- **`session_wall_ms`** (`number`, _required_)
  - Constraints: minimum=0
- **`history_limit`** (`integer`, _required_)
  - Constraints: minimum=1
- **`type`** (`string`, _required_)
  - Constant: `"session_metrics"`
- **`final`** (`boolean`, _optional_)
  - Default: `false`
- **`units`** (`string`, _required_)
  - Constant: `"ms"`
- **`gauges`** (`object`, _required_)
  - Additional properties: number
- **`history_size`** (`integer`, _required_)
  - Constraints: minimum=0
- **`phases`** (`object`, _required_)
  - Additional properties: object
    - **`p50_ms`** (`number`, _required_)
      - Constraints: minimum=0
    - **`total_ms`** (`number`, _required_)
      - Constraints: minimum=0
    - **`p95_ms`** (`number`, _required_)
      - Constraints: minimum=0
    - **`max_ms`** (`number`, _required_)
      - Constraints: minimum=0
    - **`count`** (`integer`, _required_)
      - Constraints: minimum=0

**Example**

```json
{
  "session_wall_ms": 0,
  "history_limit": 1,
  "type": "session_metrics",
  "units": "ms",
  "gauges": {},
  "history_size": 0,
  "phases": {}
}
```

### Server stream_exhausted message

- **Wire type**: `stream_exhausted`
- **Payload**: `object`

- **`reason`** (`string`, _required_)
  - Values: `"stopped"`, `"session_limit"`
- **`chunks`** (`integer`, _required_)
  - Constraints: minimum=0
- **`type`** (`string`, _required_)
  - Constant: `"stream_exhausted"`

**Example**

```json
{
  "reason": "stopped",
  "chunks": 0,
  "type": "stream_exhausted"
}
```

## Additional Resources

### Documentation

- [Model Playground](https://fal.ai/models/minimax/h3-max/reference-director/start-session)
- [API Documentation](https://fal.ai/models/minimax/h3-max/reference-director/start-session/api)
- [OpenAPI Schema](https://fal.ai/api/openapi/queue/openapi.json?endpoint_id=minimax/h3-max/reference-director/start-session)
- [AsyncAPI Contract](https://fal.ai/api/apps/fal-ai/minimax-h3-max-reference-director/asyncapi.json)

### fal.ai Platform

- [Platform Documentation](https://fal.ai/docs/documentation)
- [Python Client](https://fal.ai/docs/api-reference/client-libraries/python)
- [JavaScript Client](https://fal.ai/docs/api-reference/client-libraries/javascript)

### Other agent-readable surfaces

This file covers one model. To find anything else:

- [Platform overview](https://fal.ai/llms.txt): Entry points and representative endpoint IDs
- [Documentation index](https://fal.ai/docs/llms.txt): Every documentation page
- [Full documentation text](https://fal.ai/docs/llms-full.txt): The whole documentation inlined
- Any other model: `https://fal.ai/models/<endpoint-id>/llms.txt`