Direct continuous, realtime video streams with live prompts while preserving characters, settings, and story continuity.

minimax/h3-max/director

Direct continuous, realtime video streams with live prompts while preserving characters, settings, and story continuity.

Learn more about H3 Max

Inference

Commercial use

Schema AsyncAPI

LLMs

Playground API

About

Start Session

1. Calling the API#

Install the client#

The client provides a convenient way to interact with the model API.

npm install @fal-ai/client@alpha @fal-ai/server-proxy@alpha

Experimental realtime API

fal.realtime.open is currently available on the alpha release and may change in a minor version.

Set up the server proxy#

Keep FAL_KEY on your server and expose an authenticated proxy route to your browser. For a Next.js App Router application, create app/api/fal/proxy/route.ts:

import { route } from "@fal-ai/server-proxy/nextjs";

export const { GET, POST, PUT } = route;

Set FAL_KEY in the server environment, and protect this route with your application's authentication before deploying it.

Real-time via WebRTC#

This deployment publishes a WMA WebRTC contract. The example uses fal.realtime.open; its media handlers, capture constraints, and control message are generated from the linked AsyncAPI document.

import { createFalClient } from "@fal-ai/client";
import { wma } from "@fal-ai/client/realtime";

const fal = createFalClient({ proxyUrl: "/api/fal/proxy" });

const session = fal.realtime.open(wma("minimax/h3-max/director"), {
  receive: [
    "video",
    "audio"
  ],
  onMedia: (stream) => {
      const element = document.querySelector("video");
      if (element) element.srcObject = stream;
    },
  onData: (raw) => {
      const message = JSON.parse(raw);
      if (message?.type === "configured" && message?.["prompt_version"] === 1) {
        console.log("Session configured. You can now send text updates.");
      }
      console.log(message);
    },
  onState: (state) => console.log("realtime:", state),
  onError: (error) => console.error(error),
});


// Send setup once. The client buffers this until the control channel opens.
// Wait for the matching acknowledgement in onData before sending updates.
session.send({
  "prompt": "A continuous original live-action American sitcom produced in 1994, following the same ensemble of adult roommates, coworkers, neighbors, and rivals. Preserve appearances, apartment and workplace layouts, relationships, jobs, secrets, running jokes, and unresolved storylines. Advance through dialogue, entrances, misunderstandings, escalating attempts to hide mistakes, reversals, and warm character payoffs. Avoid references to existing sitcoms or actors.",
  "type": "configure",
  "protocol_version": 1,
  "resolution": "768p",
  "aspect_ratio": "16:9",
  "prompt_version": 1,
  "memory": 3
});

// Later, release the peer connection and network resources:
// await session.close();

Session flow#

Configure once, wait for the model to acknowledge setup, then update the running session. The transport being live does not mean the model is configured.

1. Configure the session#

Send configure once with /prompt_version set to 1. These field paths are JSON Pointers into message payloads. Use the setup message reference below for all configuration fields.

{
  "prompt": "A continuous original live-action American sitcom produced in 1994, following the same ensemble of adult roommates, coworkers, neighbors, and rivals. Preserve appearances, apartment and workplace layouts, relationships, jobs, secrets, running jokes, and unresolved storylines. Advance through dialogue, entrances, misunderstandings, escalating attempts to hide mistakes, reversals, and warm character payoffs. Avoid references to existing sitcoms or actors.",
  "type": "configure",
  "protocol_version": 1,
  "resolution": "768p",
  "aspect_ratio": "16:9",
  "prompt_version": 1,
  "memory": 3
}

2. Wait for acknowledgement#

Wait for configured whose /prompt_version matches the version you sent in setup. Only that acknowledgement enables text updates. Pending, applied, or rejected input events do not confirm session readiness.

3. Update the running session#

Send prompt with your text at /prompt. Start from {"type":"prompt"}, add the text and the next /prompt_version, and validate the complete payload against the message schema. The first update uses 2; increase by 1 for each subsequent submission.

{
  "type": "prompt",
  "prompt": "They follow a narrow path down to the harbor.",
  "prompt_version": 2
}

This action sends only the template, text, and version fields. Other options in the raw message schema are outside this text-update flow. If you use them too, coordinate the same session-wide version sequence.

Track each update#

Match the version in each event to the submission it describes, including the initial setup input. Several updates can have different outcomes while the session stays ready.

Pending (prompt_pending): The model is preparing this input. Match the submitted version at /prompt_version.
Applied (prompt_applied): Accepted for generation. The result may not be generating or visible yet. Match the submitted version at /prompt_version.
Rejected (prompt_rejected): This input was not accepted. An earlier applied input can remain active. Match the submitted version at /prompt_version.

You can send another update while preparation is pending. Newer updates can replace older pending preparation, and the older input may never receive a final event. A missing acknowledgement means the outcome is unknown; it does not prove rejection.

Handle errors#

For error, read the code at /code and the explanation at /error. Match input failures using /prompt_version.

Session failure: Ends this session attempt, even without a matching version. Codes: configuration_timeout, initialization_timeout, invalid_initial_image, invalid_initial_audio, invalid_initial_script, invalid_input, balance_unavailable, content_policy, generation_timeout, generation_failed.
Input failure: Fails only the submission with a known matching version. An absent or unknown version is diagnostic. Codes: stale_prompt_version.
Diagnostic: Provides information without declaring the session or a submission failed. Codes: invalid_message, not_configured, immutable_settings.

Treat unknown codes as diagnostics. An input failure without a known matching submission is also diagnostic; do not guess which input failed.

End or reconnect#

stream_exhausted ends this session attempt. A closed or failed connection also ends the attempt. Later messages must not reopen it. A session failure does not need to emit a separate end event.

Do not automatically replay setup or updates. Start a new sequence at 1 for a new session. Gaps are allowed; never reuse a version after uncertain delivery, and keep versions within the message schema and JavaScript's safe integer range.

2. Authentication#

The browser connects through your server proxy, which reads FAL_KEY from the server environment. Never put that key in browser code.

API Key#

Authenticate users before allowing them to call your proxy route so other people cannot spend against your account.

Protect your API Key

When running code on the client-side (e.g. in a browser, mobile app or GUI applications), make sure to not expose your FAL_KEY. Instead, use a server-side proxy to make requests to the API. For more information, check out our server-side integration guide.

3. Schema#

Media#

Client contract for the WebRTC session created by the linked OpenAPI operation.

Tracks are described from the browser's perspective. Send tracks are captured by the browser and sent to the model; receive tracks come back from the model.

Send

No send media tracks.

Receive

video

optional

{
  "frameRate": 24
}

audio

optional

Client messages#

Client configure message

type: "configure"

payload: object

Correlation field: /prompt_version

Additional properties: not allowed

promptstring* required

At least 1 character · At most 50000 characters

resolutionstring

Default: "768p"

Values: "480p", "768p", "1080p"

aspect_ratiostring

Default: "16:9"

Values: "16:9", "9:16", "1:1"

image_urlstring | null

URL of the image to use as the exact first frame. The opening prompt expansion also sees this image, so the first segment's prompt is grounded in it.

Default: null

At least 1 character when not null

end_image_urlstring | null

One-shot exact final frame for the first chunk. Director jointly plans that arrival and the following checkpoint continuation.

Default: null

At least 1 character when not null

audio_urlstring | null

Optional startup soundtrack. The stream's audio is pinned to this recording from the first chunk as FL2VA target audio, not a Ref2VA reference: every chunk is conditioned on the next window of it (plus the regenerated seam) until it ends, and the source PCM itself is what plays. Live prompt messages can replace or queue more audio at any time.

Default: null

At least 1 character when not null

memoryinteger

Number of prior segment prompts retained as context for future prompt expansion.

Default: 12

minimum: 1 · maximum: 50

audio_bitrateinteger | null

Session audio target in bits/s: 96000, 128000, or 192000. Explicit values use Opus audio mode on direct WebRTC; LiveKit uses its native mode with the same bitrate target. Null preserves transport defaults (WebRTC: 96000, voip). Immutable; reconnect to compare.

Default: null

When not null:

Values: 96000, 128000, 192000

seedinteger | null

Default: null

typestring* required

Constant: "configure"

protocol_versioninteger

Constant: 1

prompt_versioninteger* required

minimum: 1

scriptobject[] | null

Optional upfront script: beats at whole-second offsets from the first generated video. prompt stays the series premise; a beat prompt directs from its offset on. Cannot be combined with end_image_url or audio_url (place them in the script).

Default: null

Must match at least one of the following:

Alternative 1

object[]

minItems: 1 · maxItems: 64

Array item

object

A direction on the associated video's clock, never the stream clock.

Additional properties: not allowed

promptstring | null

Direction that starts at this offset and persists until the next text beat; it also appears as a timed span inside the chunk.

Default: null

At least 1 character when not null · At most 50000 characters when not null

audio_urlstring | null

Audio that starts playing exactly at this offset as FL2VA target audio; overlapping sources are mixed. The source PCM itself plays.

Default: null

At least 1 character when not null

offsetinteger* required

Whole seconds from the start of the first video generated under this script. Text and audio placed here start at this second; an end image here is the exact final frame at this second.

minimum: 0

end_image_urlstring | null

Exact final frame of the chunk that ends at this offset. Offsets of successive end images must be at least three seconds apart.

Default: null

At least 1 character when not null

Alternative 2

null

Example

Drop a fixed ensemble into audience-created misunderstandings.

{
  "prompt": "A continuous original live-action American sitcom produced in 1994, following the same ensemble of adult roommates, coworkers, neighbors, and rivals. Preserve appearances, apartment and workplace layouts, relationships, jobs, secrets, running jokes, and unresolved storylines. Advance through dialogue, entrances, misunderstandings, escalating attempts to hide mistakes, reversals, and warm character payoffs. Avoid references to existing sitcoms or actors.",
  "type": "configure",
  "protocol_version": 1,
  "resolution": "768p",
  "aspect_ratio": "16:9",
  "prompt_version": 1,
  "memory": 3
}

Client ping message

type: "ping"

payload: object

Additional properties: not allowed

typestring* required

Constant: "ping"

tsnumber* required

{
  "type": "ping",
  "ts": 0
}

Client prompt message

type: "prompt"

payload: object

Correlation field: /prompt_version

Additional properties: not allowed

promptstring | null

Default: null

At least 1 character when not null · At most 50000 characters when not null

end_image_urlstring | null

Default: null

At least 1 character when not null

audio_urlstring | null

FL2VA target audio for future chunks; this is not a Ref2VA reference. With audio_behavior 'replace' (default) it starts at the next undispatched chunk and drops any queued audio; with 'queue' it plays after every previously accepted source ends, sample-exact.

Default: null

At least 1 character when not null

audio_behaviorstring

'replace' cuts to this audio at the next chunk; 'queue' appends it.

Default: "replace"

Values: "replace", "queue"

replanboolean

When a prompt is included: true (default) busts the planned prompt queue so the new direction applies at the next undispatched chunk; false appends the direction after the already-planned chunks.

Default: true

typestring* required

Constant: "prompt"

script_modestring

'replace' cuts to the script at the next chunk; 'append' queues it.

Default: "replace"

Values: "replace", "append"

scriptobject[] | null

A new script: beats at whole-second offsets from the first video generated under it. Exclusive with prompt/end_image_url/audio_url. With script_mode 'replace' (default) it replaces the pending and current direction at the next undispatched chunk; 'append' starts at the chunk boundary after the running script's last text beat is represented and its audio tails have finished (queued behind earlier appended scripts). Beats without a prompt keep the current text direction.

Default: null

Must match at least one of the following:

Alternative 1

object[]

minItems: 1 · maxItems: 64

Array item

object

A direction on the associated video's clock, never the stream clock.

Additional properties: not allowed

promptstring | null

Direction that starts at this offset and persists until the next text beat; it also appears as a timed span inside the chunk.

Default: null

At least 1 character when not null · At most 50000 characters when not null

audio_urlstring | null

Audio that starts playing exactly at this offset as FL2VA target audio; overlapping sources are mixed. The source PCM itself plays.

Default: null

At least 1 character when not null

offsetinteger* required

Whole seconds from the start of the first video generated under this script. Text and audio placed here start at this second; an end image here is the exact final frame at this second.

minimum: 0

end_image_urlstring | null

Exact final frame of the chunk that ends at this offset. Offsets of successive end images must be at least three seconds apart.

Default: null

At least 1 character when not null

Alternative 2

null

prompt_versioninteger* required

minimum: 1

Example

Send a new direction after configuration is acknowledged

{
  "script_mode": "replace",
  "type": "prompt",
  "prompt": "They follow a narrow path down to the harbor.",
  "audio_behavior": "replace",
  "prompt_version": 2,
  "replan": true
}

Client stop message

type: "stop"

payload: object

Additional properties: not allowed

typestring* required

Constant: "stop"

{
  "type": "stop"
}

Server messages#

Server audio_applied message

type: "audio_applied"

payload: object

An audio update was accepted; it conditions the next eligible chunk.

starts_at_chunk_index is the first chunk that can carry it (the next undispatched chunk for replace; for queue the chunk where the preceding sources end, if already known).

remaining_secondsnumber* required

minimum: 0

typestring* required

Constant: "audio_applied"

duration_secondsnumber* required

exclusiveMinimum: 0

queued_sourcesinteger* required

minimum: 0

transcribedboolean* required

behaviorstring* required

Values: "replace", "queue"

prompt_versioninteger* required

minimum: 1

sourcestring* required

{
  "remaining_seconds": 0,
  "type": "audio_applied",
  "duration_seconds": 5e-324,
  "queued_sources": 0,
  "transcribed": true,
  "behavior": "replace",
  "prompt_version": 1,
  "source": "string"
}

Server audio_exhausted message

type: "audio_exhausted"

payload: object

Accepted audio ran out inside this chunk; the rest is silence.

Later chunks continue without audio conditioning (checkpoint continuation on the same family) until new audio is accepted.

chunk_indexinteger* required

minimum: 0

typestring* required

Constant: "audio_exhausted"

silent_secondsnumber* required

minimum: 0

source_versioninteger | null

Default: null

minimum: 0 when not null

{
  "chunk_index": 0,
  "type": "audio_exhausted",
  "silent_seconds": 0
}

Server audio_pending message

type: "audio_pending"

payload: object

typestring* required

Constant: "audio_pending"

prompt_versioninteger* required

minimum: 1

behaviorstring* required

Values: "replace", "queue"

{
  "type": "audio_pending",
  "prompt_version": 1,
  "behavior": "replace"
}

Server audio_rejected message

type: "audio_rejected"

payload: object

typestring* required

Constant: "audio_rejected"

errorstring* required

reasonstring* required

Values: "invalid_audio", "content_policy", "preparation_failed", "queue_full", "stale_prompt_version"

prompt_versioninteger* required

minimum: 1

{
  "type": "audio_rejected",
  "error": "string",
  "reason": "invalid_audio",
  "prompt_version": 1
}

Server chunk message

type: "chunk"

payload: object

scheduling_lead_msnumber* required

typestring* required

Constant: "chunk"

script_end_keyframeboolean | null

Default: null

requested_duration_secondsinteger* required

minimum: 5 · maximum: 15

buffer_depth_chunksinteger* required

minimum: 0

dispatchobject* required

wall_msnumber* required

minimum: 0

overhead_msnumber* required

minimum: 0

phases_msobject* required

Additional properties: number

number

classified_msnumber* required

minimum: 0

prompt_versioninteger* required

minimum: 1

next_generation_estimate_secondsnumber* required

minimum: 0

presented_frame_countinteger | null

Default: null

exclusiveMinimum: 0 when not null

trimmed_context_framesinteger* required

minimum: 0

hard_cutboolean

Default: false

native_playable_frame_countinteger | null

Default: null

exclusiveMinimum: 0 when not null

script_versioninteger | null

Default: null

minimum: 1 when not null

routestring* required

Values: "gorgonea", "betelgeuse", "regulus", "unknown"

scheduling_slack_msnumber* required

buffer_depth_secondsnumber* required

minimum: 0

playback_secondsnumber* required

exclusiveMinimum: 0

generated_frame_countinteger* required

exclusiveMinimum: 0

script_offset_secondsinteger | null

Default: null

minimum: 0 when not null

generation_secondsnumber* required

minimum: 0

chunk_indexinteger* required

minimum: 0

{
  "scheduling_lead_ms": 0,
  "type": "chunk",
  "requested_duration_seconds": 5,
  "buffer_depth_chunks": 0,
  "dispatch": {
    "wall_ms": 0,
    "overhead_ms": 0,
    "phases_ms": {},
    "classified_ms": 0
  },
  "prompt_version": 1,
  "next_generation_estimate_seconds": 0,
  "trimmed_context_frames": 0,
  "route": "gorgonea",
  "scheduling_slack_ms": 0,
  "buffer_depth_seconds": 0,
  "playback_seconds": 5e-324,
  "generated_frame_count": 1,
  "generation_seconds": 0,
  "chunk_index": 0
}

Server chunk_metrics message

type: "chunk_metrics"

payload: object

chunk_consumable_ready_msnumber | null

Default: null

minimum: 0 when not null

typestring* required

Constant: "chunk_metrics"

routestring* required

Values: "gorgonea", "betelgeuse", "regulus", "unknown"

unitsstring* required

Constant: "ms"

chunk_consumable_interval_msnumber | null

Default: null

minimum: 0 when not null

phases_msobject* required

Additional properties: number

number

gaugesobject* required

Additional properties: number

number

chunk_indexinteger* required

minimum: 0

{
  "type": "chunk_metrics",
  "route": "gorgonea",
  "units": "ms",
  "phases_ms": {},
  "gauges": {},
  "chunk_index": 0
}

Server configured message

type: "configured"

payload: object

Correlation field: /prompt_version

has_initial_imageboolean | null

Default: null

typestring* required

Constant: "configured"

enable_safety_checkerboolean* required

chunk_durationinteger | null

Default: null

minimum: 5 when not null · maximum: 15 when not null

memoryinteger | null

Default: null

minimum: 1 when not null · maximum: 50 when not null

prompt_versioninteger* required

minimum: 1

resolutionstring | string | null

Default: null

Must match at least one of the following:

Alternative 1

string

Values: "480p", "544p", "640p", "704p", "768p"

Alternative 2

string

Values: "480p", "768p", "1080p"

Alternative 3

null

audio_bitrateinteger | null

Default: null

When not null:

Values: 96000, 128000, 192000

aspect_ratiostring | null

Default: null

When not null:

Values: "16:9", "9:16", "1:1"

has_initial_audioboolean | null

Default: null

accelerationstring | null

Default: null

When not null:

Values: "none", "regular"

{
  "type": "configured",
  "enable_safety_checker": true,
  "prompt_version": 1
}

Server deadline_missed message

type: "deadline_missed"

payload: object

typestring* required

Constant: "deadline_missed"

late_by_secondsnumber* required

minimum: 0

behaviorstring* required

Constant: "freeze_video_and_silence_audio_until_ready"

chunk_indexinteger* required

minimum: 0

{
  "type": "deadline_missed",
  "late_by_seconds": 0,
  "behavior": "freeze_video_and_silence_audio_until_ready",
  "chunk_index": 0
}

Server error message

type: "error"

payload: object

Correlation field: /prompt_version

typestring* required

Constant: "error"

codestring* required

Values: "balance_unavailable", "content_policy", "configuration_timeout", "generation_failed", "generation_timeout", "immutable_settings", "initialization_timeout", "invalid_initial_image", "invalid_initial_audio", "invalid_initial_script", "invalid_input", "invalid_message", "not_configured", "stale_prompt_version"

errorstring* required

prompt_versioninteger | null

Default: null

minimum: 1 when not null

detailobject[] | null

Default: null

Must match at least one of the following:

Alternative 1

object[]

Array item

object

Alternative 2

null

{
  "type": "error",
  "code": "balance_unavailable",
  "error": "string"
}

Server pong message

type: "pong"

payload: object

typestring* required

Constant: "pong"

client_tsnumber* required

{
  "type": "pong",
  "client_ts": 0
}

Server prompt_applied message

type: "prompt_applied"

payload: object

Correlation field: /prompt_version

A direction (or script) is admitted; the script fields report scripts.

script_origin_chunk_index is the first chunk generated under the script when that is already fixed (a replacement binds to the next undispatched chunk); an appended script reports None until it starts.

script_modestring | null

Default: null

When not null:

Values: "replace", "append"

typestring* required

Constant: "prompt_applied"

script_queuedinteger | null

Default: null

minimum: 0 when not null

script_origin_chunk_indexinteger | null

Default: null

minimum: 0 when not null

script_beatsinteger | null

Default: null

minimum: 1 when not null

prompt_versioninteger* required

minimum: 1

{
  "type": "prompt_applied",
  "prompt_version": 1
}

Server prompt_pending message

type: "prompt_pending"

payload: object

Correlation field: /prompt_version

typestring* required

Constant: "prompt_pending"

prompt_versioninteger* required

minimum: 1

{
  "type": "prompt_pending",
  "prompt_version": 1
}

Server prompt_rejected message

type: "prompt_rejected"

payload: object

Correlation field: /prompt_version

typestring* required

Constant: "prompt_rejected"

errorstring | null

Default: null

reasonstring* required

Values: "content_policy", "preparation_failed", "stale_prompt_version", "invalid_script", "infeasible_timing", "invalid_audio", "invalid_image", "queue_full"

prompt_versioninteger* required

minimum: 1

{
  "type": "prompt_rejected",
  "reason": "content_policy",
  "prompt_version": 1
}

Server session_info message

type: "session_info"

payload: object

audio_conditioningboolean

Constant: true

typestring* required

Constant: "session_info"

script_max_decoded_audio_bytesinteger

Default: 67108864

max_audio_source_secondsnumber

Default: 600

script_max_audio_beatsinteger

Constant: 8

resolutionsstring[]

Default: ["480p","768p","1080p"]

Array item

string

Values: "480p", "768p", "1080p"

continuation_context_framesinteger

Constant: 39

min_chunk_durationinteger

Constant: 5

default_memoryinteger

Constant: 12

min_memoryinteger

Constant: 1

script_modesstring[]

Default: ["replace","append"]

Array item

string

Values: "replace", "append"

max_memoryinteger

Constant: 50

aspect_ratiosstring[]

Default: ["16:9","9:16","1:1"]

Array item

string

Values: "16:9", "9:16", "1:1"

appstring

Constant: "minimax-h3-max-director"

audio_sample_rateinteger

Constant: 48000

script_max_beatsinteger

Constant: 64

opening_reservedboolean

Default: false

script_max_pendinginteger

Constant: 4

script_min_end_image_spacing_secondsinteger

Constant: 3

audio_bitratesinteger[]

Default: [96000,128000,192000]

Array item

integer

Values: 96000, 128000, 192000

protocol_versioninteger

Constant: 1

default_audio_bitrateinteger | null

Default: null

When not null:

Values: 96000, 128000, 192000

accelerationsstring[]

Default: ["none","regular"]

Array item

string

Values: "none", "regular"

default_accelerationstring

Constant: "regular"

script_min_chunk_secondsinteger

Constant: 3

script_min_opening_chunk_secondsinteger

Constant: 5

backend_selectionstring

Constant: "minimax-h3-turbo-balancer"

conditioning_audio_sample_rateinteger

Constant: 32000

prompt_expanderstring

Constant: "fast"

max_session_secondsnumber | null

Default: null

session_limit_scopestring

Default: "configured"

Values: "configured", "effective"

scriptsboolean

Constant: true

client_message_typesstring[]

Default: ["configure","ping","prompt","stop"]

Array item

string

Values: "configure", "ping", "prompt", "stop"

prompt_context_segmentsinteger

Constant: 12

script_session_max_decoded_audio_bytesinteger

Default: 335544320

max_chunk_durationinteger

Constant: 15

controller_machine_typestring

Constant: "XL"

script_max_queuedinteger

Constant: 4

script_max_end_imagesinteger

Constant: 16

one_session_per_machineboolean

Constant: true

default_chunk_durationinteger

Constant: 10

prompt_deck_sizeinteger

Constant: 6

continuation_playback_secondsnumber

Default: 8.5

fpsinteger

Constant: 24

server_message_typesstring[]

Default: ["audio_applied","audio_exhausted","audio_pending","audio_rejected","chunk","chunk_metrics","configured","deadline_missed","error","pong","prompt_applied","prompt_pending","prompt_rejected","session_info","session_metrics","stream_exhausted"]

Array item

string

Values: "audio_applied", "audio_exhausted", "audio_pending", "audio_rejected", "chunk", "chunk_metrics", "configured", "deadline_missed", "error", "pong", "prompt_applied", "prompt_pending", "prompt_rejected", "session_info", "session_metrics", "stream_exhausted"

audio_behaviorsstring[]

Default: ["replace","queue"]

Array item

string

Values: "replace", "queue"

chunk_secondsinteger

Constant: 10

{
  "audio_conditioning": true,
  "type": "session_info",
  "script_max_audio_beats": 8,
  "continuation_context_frames": 39,
  "min_chunk_duration": 5,
  "default_memory": 12,
  "min_memory": 1,
  "max_memory": 50,
  "app": "minimax-h3-max-director",
  "audio_sample_rate": 48000,
  "script_max_beats": 64,
  "script_max_pending": 4,
  "script_min_end_image_spacing_seconds": 3,
  "protocol_version": 1,
  "default_acceleration": "regular",
  "script_min_chunk_seconds": 3,
  "script_min_opening_chunk_seconds": 5,
  "backend_selection": "minimax-h3-turbo-balancer",
  "conditioning_audio_sample_rate": 32000,
  "prompt_expander": "fast",
  "scripts": true,
  "prompt_context_segments": 12,
  "max_chunk_duration": 15,
  "controller_machine_type": "XL",
  "script_max_queued": 4,
  "script_max_end_images": 16,
  "one_session_per_machine": true,
  "default_chunk_duration": 10,
  "prompt_deck_size": 6,
  "fps": 24,
  "chunk_seconds": 10
}

Server session_metrics message

type: "session_metrics"

payload: object

finalboolean

Default: false

typestring* required

Constant: "session_metrics"

phasesobject* required

Additional properties: object

object

p50_msnumber* required

minimum: 0

total_msnumber* required

minimum: 0

max_msnumber* required

minimum: 0

countinteger* required

minimum: 0

p95_msnumber* required

minimum: 0

unitsstring* required

Constant: "ms"

history_limitinteger* required

minimum: 1

gaugesobject* required

Additional properties: number

number

session_wall_msnumber* required

minimum: 0

history_sizeinteger* required

minimum: 0

{
  "type": "session_metrics",
  "phases": {},
  "units": "ms",
  "history_limit": 1,
  "gauges": {},
  "session_wall_ms": 0,
  "history_size": 0
}

Server stream_exhausted message

type: "stream_exhausted"

payload: object

typestring* required

Constant: "stream_exhausted"

chunksinteger* required

minimum: 0

reasonstring* required

Values: "stopped", "session_limit"

{
  "type": "stream_exhausted",
  "chunks": 0,
  "reason": "stopped"
}

Related Models

1. Calling the API

2. Authentication

API Key

3. Schema

About

Start Session

1. Calling the API#

Install the client#

The client provides a convenient way to interact with the model API.

npm install @fal-ai/client@alpha @fal-ai/server-proxy@alpha

Experimental realtime API

fal.realtime.open is currently available on the alpha release and may change in a minor version.

Set up the server proxy#

Keep FAL_KEY on your server and expose an authenticated proxy route to your browser. For a Next.js App Router application, create app/api/fal/proxy/route.ts:

import { route } from "@fal-ai/server-proxy/nextjs";

export const { GET, POST, PUT } = route;

Set FAL_KEY in the server environment, and protect this route with your application's authentication before deploying it.

Real-time via WebRTC#

This deployment publishes a WMA WebRTC contract. The example uses fal.realtime.open; its media handlers, capture constraints, and control message are generated from the linked AsyncAPI document.

import { createFalClient } from "@fal-ai/client";
import { wma } from "@fal-ai/client/realtime";

const fal = createFalClient({ proxyUrl: "/api/fal/proxy" });

const session = fal.realtime.open(wma("minimax/h3-max/director"), {
  receive: [
    "video",
    "audio"
  ],
  onMedia: (stream) => {
      const element = document.querySelector("video");
      if (element) element.srcObject = stream;
    },
  onData: (raw) => {
      const message = JSON.parse(raw);
      if (message?.type === "configured" && message?.["prompt_version"] === 1) {
        console.log("Session configured. You can now send text updates.");
      }
      console.log(message);
    },
  onState: (state) => console.log("realtime:", state),
  onError: (error) => console.error(error),
});


// Send setup once. The client buffers this until the control channel opens.
// Wait for the matching acknowledgement in onData before sending updates.
session.send({
  "prompt": "A continuous original live-action American sitcom produced in 1994, following the same ensemble of adult roommates, coworkers, neighbors, and rivals. Preserve appearances, apartment and workplace layouts, relationships, jobs, secrets, running jokes, and unresolved storylines. Advance through dialogue, entrances, misunderstandings, escalating attempts to hide mistakes, reversals, and warm character payoffs. Avoid references to existing sitcoms or actors.",
  "type": "configure",
  "protocol_version": 1,
  "resolution": "768p",
  "aspect_ratio": "16:9",
  "prompt_version": 1,
  "memory": 3
});

// Later, release the peer connection and network resources:
// await session.close();

Session flow#

Configure once, wait for the model to acknowledge setup, then update the running session. The transport being live does not mean the model is configured.

1. Configure the session#

Send configure once with /prompt_version set to 1. These field paths are JSON Pointers into message payloads. Use the setup message reference below for all configuration fields.

{
  "prompt": "A continuous original live-action American sitcom produced in 1994, following the same ensemble of adult roommates, coworkers, neighbors, and rivals. Preserve appearances, apartment and workplace layouts, relationships, jobs, secrets, running jokes, and unresolved storylines. Advance through dialogue, entrances, misunderstandings, escalating attempts to hide mistakes, reversals, and warm character payoffs. Avoid references to existing sitcoms or actors.",
  "type": "configure",
  "protocol_version": 1,
  "resolution": "768p",
  "aspect_ratio": "16:9",
  "prompt_version": 1,
  "memory": 3
}

2. Wait for acknowledgement#

3. Update the running session#

{
  "type": "prompt",
  "prompt": "They follow a narrow path down to the harbor.",
  "prompt_version": 2
}

Track each update#

Match the version in each event to the submission it describes, including the initial setup input. Several updates can have different outcomes while the session stays ready.

Pending (prompt_pending): The model is preparing this input. Match the submitted version at /prompt_version.
Applied (prompt_applied): Accepted for generation. The result may not be generating or visible yet. Match the submitted version at /prompt_version.
Rejected (prompt_rejected): This input was not accepted. An earlier applied input can remain active. Match the submitted version at /prompt_version.

Handle errors#

For error, read the code at /code and the explanation at /error. Match input failures using /prompt_version.

Session failure: Ends this session attempt, even without a matching version. Codes: configuration_timeout, initialization_timeout, invalid_initial_image, invalid_initial_audio, invalid_initial_script, invalid_input, balance_unavailable, content_policy, generation_timeout, generation_failed.
Input failure: Fails only the submission with a known matching version. An absent or unknown version is diagnostic. Codes: stale_prompt_version.
Diagnostic: Provides information without declaring the session or a submission failed. Codes: invalid_message, not_configured, immutable_settings.

Treat unknown codes as diagnostics. An input failure without a known matching submission is also diagnostic; do not guess which input failed.

End or reconnect#

stream_exhausted ends this session attempt. A closed or failed connection also ends the attempt. Later messages must not reopen it. A session failure does not need to emit a separate end event.

2. Authentication#

The browser connects through your server proxy, which reads FAL_KEY from the server environment. Never put that key in browser code.

API Key#

Authenticate users before allowing them to call your proxy route so other people cannot spend against your account.

Protect your API Key

3. Schema#

Media#

Client contract for the WebRTC session created by the linked OpenAPI operation.

Tracks are described from the browser's perspective. Send tracks are captured by the browser and sent to the model; receive tracks come back from the model.

Send

No send media tracks.

Receive

video

optional

{
  "frameRate": 24
}

audio

optional

Client messages#

Client configure message

type: "configure"

payload: object

Correlation field: /prompt_version

Additional properties: not allowed

promptstring* required

At least 1 character · At most 50000 characters

resolutionstring

Default: "768p"

Values: "480p", "768p", "1080p"

aspect_ratiostring

Default: "16:9"

Values: "16:9", "9:16", "1:1"

image_urlstring | null

URL of the image to use as the exact first frame. The opening prompt expansion also sees this image, so the first segment's prompt is grounded in it.

Default: null

At least 1 character when not null

end_image_urlstring | null

One-shot exact final frame for the first chunk. Director jointly plans that arrival and the following checkpoint continuation.

Default: null

At least 1 character when not null

audio_urlstring | null

Default: null

At least 1 character when not null

memoryinteger

Number of prior segment prompts retained as context for future prompt expansion.

Default: 12

minimum: 1 · maximum: 50

audio_bitrateinteger | null

Default: null

When not null:

Values: 96000, 128000, 192000

seedinteger | null

Default: null

typestring* required

Constant: "configure"

protocol_versioninteger

Constant: 1

prompt_versioninteger* required

minimum: 1

scriptobject[] | null

Default: null

Must match at least one of the following:

Alternative 1

object[]

minItems: 1 · maxItems: 64

Array item

object

A direction on the associated video's clock, never the stream clock.

Additional properties: not allowed

promptstring | null

Direction that starts at this offset and persists until the next text beat; it also appears as a timed span inside the chunk.

Default: null

At least 1 character when not null · At most 50000 characters when not null

audio_urlstring | null

Audio that starts playing exactly at this offset as FL2VA target audio; overlapping sources are mixed. The source PCM itself plays.

Default: null

At least 1 character when not null

offsetinteger* required

Whole seconds from the start of the first video generated under this script. Text and audio placed here start at this second; an end image here is the exact final frame at this second.

minimum: 0

end_image_urlstring | null

Exact final frame of the chunk that ends at this offset. Offsets of successive end images must be at least three seconds apart.

Default: null

At least 1 character when not null

Alternative 2

null

Example

Drop a fixed ensemble into audience-created misunderstandings.

{
  "prompt": "A continuous original live-action American sitcom produced in 1994, following the same ensemble of adult roommates, coworkers, neighbors, and rivals. Preserve appearances, apartment and workplace layouts, relationships, jobs, secrets, running jokes, and unresolved storylines. Advance through dialogue, entrances, misunderstandings, escalating attempts to hide mistakes, reversals, and warm character payoffs. Avoid references to existing sitcoms or actors.",
  "type": "configure",
  "protocol_version": 1,
  "resolution": "768p",
  "aspect_ratio": "16:9",
  "prompt_version": 1,
  "memory": 3
}

Client ping message

type: "ping"

payload: object

Additional properties: not allowed

typestring* required

Constant: "ping"

tsnumber* required

{
  "type": "ping",
  "ts": 0
}

Client prompt message

type: "prompt"

payload: object

Correlation field: /prompt_version

Additional properties: not allowed

promptstring | null

Default: null

At least 1 character when not null · At most 50000 characters when not null

end_image_urlstring | null

Default: null

At least 1 character when not null

audio_urlstring | null

Default: null

At least 1 character when not null

audio_behaviorstring

'replace' cuts to this audio at the next chunk; 'queue' appends it.

Default: "replace"

Values: "replace", "queue"

replanboolean

When a prompt is included: true (default) busts the planned prompt queue so the new direction applies at the next undispatched chunk; false appends the direction after the already-planned chunks.

Default: true

typestring* required

Constant: "prompt"

script_modestring

'replace' cuts to the script at the next chunk; 'append' queues it.

Default: "replace"

Values: "replace", "append"

scriptobject[] | null

Default: null

Must match at least one of the following:

Alternative 1

object[]

minItems: 1 · maxItems: 64

Array item

object

A direction on the associated video's clock, never the stream clock.

Additional properties: not allowed

promptstring | null

Direction that starts at this offset and persists until the next text beat; it also appears as a timed span inside the chunk.

Default: null

At least 1 character when not null · At most 50000 characters when not null

audio_urlstring | null

Audio that starts playing exactly at this offset as FL2VA target audio; overlapping sources are mixed. The source PCM itself plays.

Default: null

At least 1 character when not null

offsetinteger* required

Whole seconds from the start of the first video generated under this script. Text and audio placed here start at this second; an end image here is the exact final frame at this second.

minimum: 0

end_image_urlstring | null

Exact final frame of the chunk that ends at this offset. Offsets of successive end images must be at least three seconds apart.

Default: null

At least 1 character when not null

Alternative 2

null

prompt_versioninteger* required

minimum: 1

Example

Send a new direction after configuration is acknowledged

{
  "script_mode": "replace",
  "type": "prompt",
  "prompt": "They follow a narrow path down to the harbor.",
  "audio_behavior": "replace",
  "prompt_version": 2,
  "replan": true
}

Client stop message

type: "stop"

payload: object

Additional properties: not allowed

typestring* required

Constant: "stop"

{
  "type": "stop"
}

Server messages#

Server audio_applied message

type: "audio_applied"

payload: object

An audio update was accepted; it conditions the next eligible chunk.

starts_at_chunk_index is the first chunk that can carry it (the next undispatched chunk for replace; for queue the chunk where the preceding sources end, if already known).

remaining_secondsnumber* required

minimum: 0

typestring* required

Constant: "audio_applied"

duration_secondsnumber* required

exclusiveMinimum: 0

queued_sourcesinteger* required

minimum: 0

transcribedboolean* required

behaviorstring* required

Values: "replace", "queue"

prompt_versioninteger* required

minimum: 1

sourcestring* required

{
  "remaining_seconds": 0,
  "type": "audio_applied",
  "duration_seconds": 5e-324,
  "queued_sources": 0,
  "transcribed": true,
  "behavior": "replace",
  "prompt_version": 1,
  "source": "string"
}

Server audio_exhausted message

type: "audio_exhausted"

payload: object

Accepted audio ran out inside this chunk; the rest is silence.

Later chunks continue without audio conditioning (checkpoint continuation on the same family) until new audio is accepted.

chunk_indexinteger* required

minimum: 0

typestring* required

Constant: "audio_exhausted"

silent_secondsnumber* required

minimum: 0

source_versioninteger | null

Default: null

minimum: 0 when not null

{
  "chunk_index": 0,
  "type": "audio_exhausted",
  "silent_seconds": 0
}

Server audio_pending message

type: "audio_pending"

payload: object

typestring* required

Constant: "audio_pending"

prompt_versioninteger* required

minimum: 1

behaviorstring* required

Values: "replace", "queue"

{
  "type": "audio_pending",
  "prompt_version": 1,
  "behavior": "replace"
}

Server audio_rejected message

type: "audio_rejected"

payload: object

typestring* required

Constant: "audio_rejected"

errorstring* required

reasonstring* required

Values: "invalid_audio", "content_policy", "preparation_failed", "queue_full", "stale_prompt_version"

prompt_versioninteger* required

minimum: 1

{
  "type": "audio_rejected",
  "error": "string",
  "reason": "invalid_audio",
  "prompt_version": 1
}

Server chunk message

type: "chunk"

payload: object

scheduling_lead_msnumber* required

typestring* required

Constant: "chunk"

script_end_keyframeboolean | null

Default: null

requested_duration_secondsinteger* required

minimum: 5 · maximum: 15

buffer_depth_chunksinteger* required

minimum: 0

dispatchobject* required

wall_msnumber* required

minimum: 0

overhead_msnumber* required

minimum: 0

phases_msobject* required

Additional properties: number

number

classified_msnumber* required

minimum: 0

prompt_versioninteger* required

minimum: 1

next_generation_estimate_secondsnumber* required

minimum: 0

presented_frame_countinteger | null

Default: null

exclusiveMinimum: 0 when not null

trimmed_context_framesinteger* required

minimum: 0

hard_cutboolean

Default: false

native_playable_frame_countinteger | null

Default: null

exclusiveMinimum: 0 when not null

script_versioninteger | null

Default: null

minimum: 1 when not null

routestring* required

Values: "gorgonea", "betelgeuse", "regulus", "unknown"

scheduling_slack_msnumber* required

buffer_depth_secondsnumber* required

minimum: 0

playback_secondsnumber* required

exclusiveMinimum: 0

generated_frame_countinteger* required

exclusiveMinimum: 0

script_offset_secondsinteger | null

Default: null

minimum: 0 when not null

generation_secondsnumber* required

minimum: 0

chunk_indexinteger* required

minimum: 0

{
  "scheduling_lead_ms": 0,
  "type": "chunk",
  "requested_duration_seconds": 5,
  "buffer_depth_chunks": 0,
  "dispatch": {
    "wall_ms": 0,
    "overhead_ms": 0,
    "phases_ms": {},
    "classified_ms": 0
  },
  "prompt_version": 1,
  "next_generation_estimate_seconds": 0,
  "trimmed_context_frames": 0,
  "route": "gorgonea",
  "scheduling_slack_ms": 0,
  "buffer_depth_seconds": 0,
  "playback_seconds": 5e-324,
  "generated_frame_count": 1,
  "generation_seconds": 0,
  "chunk_index": 0
}

Server chunk_metrics message

type: "chunk_metrics"

payload: object

chunk_consumable_ready_msnumber | null

Default: null

minimum: 0 when not null

typestring* required

Constant: "chunk_metrics"

routestring* required

Values: "gorgonea", "betelgeuse", "regulus", "unknown"

unitsstring* required

Constant: "ms"

chunk_consumable_interval_msnumber | null

Default: null

minimum: 0 when not null

phases_msobject* required

Additional properties: number

number

gaugesobject* required

Additional properties: number

number

chunk_indexinteger* required

minimum: 0

{
  "type": "chunk_metrics",
  "route": "gorgonea",
  "units": "ms",
  "phases_ms": {},
  "gauges": {},
  "chunk_index": 0
}

Server configured message

type: "configured"

payload: object

Correlation field: /prompt_version

has_initial_imageboolean | null

Default: null

typestring* required

Constant: "configured"

enable_safety_checkerboolean* required

chunk_durationinteger | null

Default: null

minimum: 5 when not null · maximum: 15 when not null

memoryinteger | null

Default: null

minimum: 1 when not null · maximum: 50 when not null

prompt_versioninteger* required

minimum: 1

resolutionstring | string | null

Default: null

Must match at least one of the following:

Alternative 1

string

Values: "480p", "544p", "640p", "704p", "768p"

Alternative 2

string

Values: "480p", "768p", "1080p"

Alternative 3

null

audio_bitrateinteger | null

Default: null

When not null:

Values: 96000, 128000, 192000

aspect_ratiostring | null

Default: null

When not null:

Values: "16:9", "9:16", "1:1"

has_initial_audioboolean | null

Default: null

accelerationstring | null

Default: null

When not null:

Values: "none", "regular"

{
  "type": "configured",
  "enable_safety_checker": true,
  "prompt_version": 1
}

Server deadline_missed message

type: "deadline_missed"

payload: object

typestring* required

Constant: "deadline_missed"

late_by_secondsnumber* required

minimum: 0

behaviorstring* required

Constant: "freeze_video_and_silence_audio_until_ready"

chunk_indexinteger* required

minimum: 0

{
  "type": "deadline_missed",
  "late_by_seconds": 0,
  "behavior": "freeze_video_and_silence_audio_until_ready",
  "chunk_index": 0
}

Server error message

type: "error"

payload: object

Correlation field: /prompt_version

typestring* required

Constant: "error"

codestring* required

errorstring* required

prompt_versioninteger | null

Default: null

minimum: 1 when not null

detailobject[] | null

Default: null

Must match at least one of the following:

Alternative 1

object[]

Array item

object

Alternative 2

null

{
  "type": "error",
  "code": "balance_unavailable",
  "error": "string"
}

Server pong message

type: "pong"

payload: object

typestring* required

Constant: "pong"

client_tsnumber* required

{
  "type": "pong",
  "client_ts": 0
}

Server prompt_applied message

type: "prompt_applied"

payload: object

Correlation field: /prompt_version

A direction (or script) is admitted; the script fields report scripts.

script_modestring | null

Default: null

When not null:

Values: "replace", "append"

typestring* required

Constant: "prompt_applied"

script_queuedinteger | null

Default: null

minimum: 0 when not null

script_origin_chunk_indexinteger | null

Default: null

minimum: 0 when not null

script_beatsinteger | null

Default: null

minimum: 1 when not null

prompt_versioninteger* required

minimum: 1

{
  "type": "prompt_applied",
  "prompt_version": 1
}

Server prompt_pending message

type: "prompt_pending"

payload: object

Correlation field: /prompt_version

typestring* required

Constant: "prompt_pending"

prompt_versioninteger* required

minimum: 1

{
  "type": "prompt_pending",
  "prompt_version": 1
}

Server prompt_rejected message

type: "prompt_rejected"

payload: object

Correlation field: /prompt_version

typestring* required

Constant: "prompt_rejected"

errorstring | null

Default: null

reasonstring* required

Values: "content_policy", "preparation_failed", "stale_prompt_version", "invalid_script", "infeasible_timing", "invalid_audio", "invalid_image", "queue_full"

prompt_versioninteger* required

minimum: 1

{
  "type": "prompt_rejected",
  "reason": "content_policy",
  "prompt_version": 1
}

Server session_info message

type: "session_info"

payload: object

audio_conditioningboolean

Constant: true

typestring* required

Constant: "session_info"

script_max_decoded_audio_bytesinteger

Default: 67108864

max_audio_source_secondsnumber

Default: 600

script_max_audio_beatsinteger

Constant: 8

resolutionsstring[]

Default: ["480p","768p","1080p"]

Array item

string

Values: "480p", "768p", "1080p"

continuation_context_framesinteger

Constant: 39

min_chunk_durationinteger

Constant: 5

default_memoryinteger

Constant: 12

min_memoryinteger

Constant: 1

script_modesstring[]

Default: ["replace","append"]

Array item

string

Values: "replace", "append"

max_memoryinteger

Constant: 50

aspect_ratiosstring[]

Default: ["16:9","9:16","1:1"]

Array item

string

Values: "16:9", "9:16", "1:1"

appstring

Constant: "minimax-h3-max-director"

audio_sample_rateinteger

Constant: 48000

script_max_beatsinteger

Constant: 64

opening_reservedboolean

Default: false

script_max_pendinginteger

Constant: 4

script_min_end_image_spacing_secondsinteger

Constant: 3

audio_bitratesinteger[]

Default: [96000,128000,192000]

Array item

integer

Values: 96000, 128000, 192000

protocol_versioninteger

Constant: 1

default_audio_bitrateinteger | null

Default: null

When not null:

Values: 96000, 128000, 192000

accelerationsstring[]

Default: ["none","regular"]

Array item

string

Values: "none", "regular"

default_accelerationstring

Constant: "regular"

script_min_chunk_secondsinteger

Constant: 3

script_min_opening_chunk_secondsinteger

Constant: 5

backend_selectionstring

Constant: "minimax-h3-turbo-balancer"

conditioning_audio_sample_rateinteger

Constant: 32000

prompt_expanderstring

Constant: "fast"

max_session_secondsnumber | null

Default: null

session_limit_scopestring

Default: "configured"

Values: "configured", "effective"

scriptsboolean

Constant: true

client_message_typesstring[]

Default: ["configure","ping","prompt","stop"]

Array item

string

Values: "configure", "ping", "prompt", "stop"

prompt_context_segmentsinteger

Constant: 12

script_session_max_decoded_audio_bytesinteger

Default: 335544320

max_chunk_durationinteger

Constant: 15

controller_machine_typestring

Constant: "XL"

script_max_queuedinteger

Constant: 4

script_max_end_imagesinteger

Constant: 16

one_session_per_machineboolean

Constant: true

default_chunk_durationinteger

Constant: 10

prompt_deck_sizeinteger

Constant: 6

continuation_playback_secondsnumber

Default: 8.5

fpsinteger

Constant: 24

server_message_typesstring[]

Array item

string

audio_behaviorsstring[]

Default: ["replace","queue"]

Array item

string

Values: "replace", "queue"

chunk_secondsinteger

Constant: 10

{
  "audio_conditioning": true,
  "type": "session_info",
  "script_max_audio_beats": 8,
  "continuation_context_frames": 39,
  "min_chunk_duration": 5,
  "default_memory": 12,
  "min_memory": 1,
  "max_memory": 50,
  "app": "minimax-h3-max-director",
  "audio_sample_rate": 48000,
  "script_max_beats": 64,
  "script_max_pending": 4,
  "script_min_end_image_spacing_seconds": 3,
  "protocol_version": 1,
  "default_acceleration": "regular",
  "script_min_chunk_seconds": 3,
  "script_min_opening_chunk_seconds": 5,
  "backend_selection": "minimax-h3-turbo-balancer",
  "conditioning_audio_sample_rate": 32000,
  "prompt_expander": "fast",
  "scripts": true,
  "prompt_context_segments": 12,
  "max_chunk_duration": 15,
  "controller_machine_type": "XL",
  "script_max_queued": 4,
  "script_max_end_images": 16,
  "one_session_per_machine": true,
  "default_chunk_duration": 10,
  "prompt_deck_size": 6,
  "fps": 24,
  "chunk_seconds": 10
}

Server session_metrics message

type: "session_metrics"

payload: object

finalboolean

Default: false

typestring* required

Constant: "session_metrics"

phasesobject* required

Additional properties: object

object

p50_msnumber* required

minimum: 0

total_msnumber* required

minimum: 0

max_msnumber* required

minimum: 0

countinteger* required

minimum: 0

p95_msnumber* required

minimum: 0

unitsstring* required

Constant: "ms"

history_limitinteger* required

minimum: 1

gaugesobject* required

Additional properties: number

number

session_wall_msnumber* required

minimum: 0

history_sizeinteger* required

minimum: 0

{
  "type": "session_metrics",
  "phases": {},
  "units": "ms",
  "history_limit": 1,
  "gauges": {},
  "session_wall_ms": 0,
  "history_size": 0
}

Server stream_exhausted message

type: "stream_exhausted"

payload: object

typestring* required

Constant: "stream_exhausted"

chunksinteger* required

minimum: 0

reasonstring* required

Values: "stopped", "session_limit"

{
  "type": "stream_exhausted",
  "chunks": 0,
  "reason": "stopped"
}

minimax/h3-max/director

Table of contents

Table of contents

minimax/h3-max/director

Table of contents

1. Calling the API

2. Authentication

3. Schema

About

1. Calling the API#

Install the client#

Experimental realtime API

Set up the server proxy#

Real-time via WebRTC#

Session flow#

1. Configure the session#

2. Wait for acknowledgement#

3. Update the running session#

Track each update#

Handle errors#

End or reconnect#

2. Authentication#

API Key#

Protect your API Key

3. Schema#

Media#

Send

Receive

Client messages#

Client configure message

Client ping message

Client prompt message

Client stop message

Server messages#

Server audio_applied message

Server audio_exhausted message

Server audio_pending message

Server audio_rejected message

Server chunk message

Server chunk_metrics message

Server configured message

Server deadline_missed message

Server error message

Server pong message

Server prompt_applied message

Server prompt_pending message

Server prompt_rejected message

Server session_info message

Server session_metrics message

Server stream_exhausted message

Related Models

Table of contents

Table of contents

1. Calling the API

2. Authentication

3. Schema

About

1. Calling the API#

Install the client#

Experimental realtime API

Set up the server proxy#

Real-time via WebRTC#

Session flow#

1. Configure the session#

2. Wait for acknowledgement#

3. Update the running session#

Track each update#

Handle errors#

End or reconnect#

2. Authentication#

API Key#

Protect your API Key

3. Schema#

Media#

Send

Receive

Client messages#

Client configure message

Client ping message

Client prompt message