Minimax logo
minimax/h3-max/director

Direct continuous, realtime video streams with live prompts while preserving characters, settings, and story continuity.
Inference
Commercial use

About

Start Session

1. Calling the API#

Install the client#

The client provides a convenient way to interact with the model API.

npm install @fal-ai/client@alpha @fal-ai/server-proxy@alpha

Set up the server proxy#

Keep FAL_KEY on your server and expose an authenticated proxy route to your browser. For a Next.js App Router application, create app/api/fal/proxy/route.ts:

import { route } from "@fal-ai/server-proxy/nextjs";

export const { GET, POST, PUT } = route;

Set FAL_KEY in the server environment, and protect this route with your application's authentication before deploying it.

Real-time via WebRTC#

This deployment publishes a WMA WebRTC contract. The example uses fal.realtime.open; its media handlers, capture constraints, and control message are generated from the linked AsyncAPI document.

import { createFalClient } from "@fal-ai/client";
import { wma } from "@fal-ai/client/realtime";

const fal = createFalClient({ proxyUrl: "/api/fal/proxy" });

const session = fal.realtime.open(wma("minimax/h3-max/director"), {
  receive: [
    "video",
    "audio"
  ],
  onMedia: (stream) => {
      const element = document.querySelector("video");
      if (element) element.srcObject = stream;
    },
  onData: (raw) => {
      const message = JSON.parse(raw);
      if (message?.type === "configured" && message?.["prompt_version"] === 1) {
        console.log("Session configured. You can now send text updates.");
      }
      console.log(message);
    },
  onState: (state) => console.log("realtime:", state),
  onError: (error) => console.error(error),
});


// Send setup once. The client buffers this until the control channel opens.
// Wait for the matching acknowledgement in onData before sending updates.
session.send({
  "prompt_version": 1,
  "resolution": "768p",
  "aspect_ratio": "16:9",
  "type": "configure",
  "prompt": "A continuous original live-action American sitcom produced in 1994, following the same ensemble of adult roommates, coworkers, neighbors, and rivals. Preserve appearances, apartment and workplace layouts, relationships, jobs, secrets, running jokes, and unresolved storylines. Advance through dialogue, entrances, misunderstandings, escalating attempts to hide mistakes, reversals, and warm character payoffs. Avoid references to existing sitcoms or actors.",
  "protocol_version": 1,
  "memory": 3
});

// Later, release the peer connection and network resources:
// await session.close();

Session flow#

Configure once, wait for the model to acknowledge setup, then update the running session. The transport being live does not mean the model is configured.

1. Configure the session#

Send configure once with /prompt_version set to 1. These field paths are JSON Pointers into message payloads. Use the setup message reference below for all configuration fields.

{
  "prompt_version": 1,
  "resolution": "768p",
  "aspect_ratio": "16:9",
  "type": "configure",
  "prompt": "A continuous original live-action American sitcom produced in 1994, following the same ensemble of adult roommates, coworkers, neighbors, and rivals. Preserve appearances, apartment and workplace layouts, relationships, jobs, secrets, running jokes, and unresolved storylines. Advance through dialogue, entrances, misunderstandings, escalating attempts to hide mistakes, reversals, and warm character payoffs. Avoid references to existing sitcoms or actors.",
  "protocol_version": 1,
  "memory": 3
}

2. Wait for acknowledgement#

Wait for configured whose /prompt_version matches the version you sent in setup. Only that acknowledgement enables text updates. Pending, applied, or rejected input events do not confirm session readiness.

3. Update the running session#

Send prompt with your text at /prompt. Start from {"type":"prompt"}, add the text and the next /prompt_version, and validate the complete payload against the message schema. The first update uses 2; increase by 1 for each subsequent submission.

{
  "type": "prompt",
  "prompt": "They follow a narrow path down to the harbor.",
  "prompt_version": 2
}

This action sends only the template, text, and version fields. Other options in the raw message schema are outside this text-update flow. If you use them too, coordinate the same session-wide version sequence.

Track each update#

Match the version in each event to the submission it describes, including the initial setup input. Several updates can have different outcomes while the session stays ready.

  • Pending (prompt_pending): The model is preparing this input. Match the submitted version at /prompt_version.
  • Applied (prompt_applied): Accepted for generation. The result may not be generating or visible yet. Match the submitted version at /prompt_version.
  • Rejected (prompt_rejected): This input was not accepted. An earlier applied input can remain active. Match the submitted version at /prompt_version.

You can send another update while preparation is pending. Newer updates can replace older pending preparation, and the older input may never receive a final event. A missing acknowledgement means the outcome is unknown; it does not prove rejection.

Handle errors#

For error, read the code at /code and the explanation at /error. Match input failures using /prompt_version.

  • Session failure: Ends this session attempt, even without a matching version. Codes: configuration_timeout, initialization_timeout, invalid_initial_image, invalid_initial_audio, invalid_initial_script, invalid_input, balance_unavailable, content_policy, generation_timeout, generation_failed.
  • Input failure: Fails only the submission with a known matching version. An absent or unknown version is diagnostic. Codes: stale_prompt_version.
  • Diagnostic: Provides information without declaring the session or a submission failed. Codes: invalid_message, not_configured, immutable_settings.

Treat unknown codes as diagnostics. An input failure without a known matching submission is also diagnostic; do not guess which input failed.

End or reconnect#

stream_exhausted ends this session attempt. A closed or failed connection also ends the attempt. Later messages must not reopen it. A session failure does not need to emit a separate end event.

Do not automatically replay setup or updates. Start a new sequence at 1 for a new session. Gaps are allowed; never reuse a version after uncertain delivery, and keep versions within the message schema and JavaScript's safe integer range.

2. Authentication#

The browser connects through your server proxy, which reads FAL_KEY from the server environment. Never put that key in browser code.

API Key#

Authenticate users before allowing them to call your proxy route so other people cannot spend against your account.

3. Schema#

Media#

Client contract for the WebRTC session created by the linked OpenAPI operation.

Tracks are described from the browser's perspective. Send tracks are captured by the browser and sent to the model; receive tracks come back from the model.

Send

No send media tracks.

Receive

video
optional
{
  "frameRate": 24
}
audio
optional

Client messages#

Client configure message

type: "configure"
payload: object

Correlation field: /prompt_version

Additional properties: not allowed

promptstring* required

At least 1 character · At most 50000 characters

resolutionstring

Default: "768p"

Values: "480p", "768p", "1080p"

aspect_ratiostring

Default: "16:9"

Values: "16:9", "9:16", "1:1"

image_urlstring | null

URL of the image to use as the exact first frame. The opening prompt expansion also sees this image, so the first segment's prompt is grounded in it.

Default: null

At least 1 character when not null

end_image_urlstring | null

One-shot exact final frame for the first chunk. Director jointly plans that arrival and the following checkpoint continuation.

Default: null

At least 1 character when not null

audio_urlstring | null

Optional startup soundtrack. The stream's audio is pinned to this recording from the first chunk as FL2VA target audio, not a Ref2VA reference: every chunk is conditioned on the next window of it (plus the regenerated seam) until it ends, and the source PCM itself is what plays. Live prompt messages can replace or queue more audio at any time.

Default: null

At least 1 character when not null

memoryinteger

Number of prior segment prompts retained as context for future prompt expansion.

Default: 12

minimum: 1 · maximum: 50

audio_bitrateinteger | null

Session audio target in bits/s: 96000, 128000, or 192000. Explicit values use Opus audio mode on direct WebRTC; LiveKit uses its native mode with the same bitrate target. Null preserves transport defaults (WebRTC: 96000, voip). Immutable; reconnect to compare.

Default: null

When not null:

Values: 96000, 128000, 192000

seedinteger | null

Default: null

prompt_versioninteger* required

minimum: 1

protocol_versioninteger

Constant: 1

scriptobject[] | null

Optional upfront script: beats at whole-second offsets from the first generated video. prompt stays the series premise; a beat prompt directs from its offset on. Cannot be combined with end_image_url or audio_url (place them in the script).

Default: null

Must match at least one of the following:

Alternative 1
object[]

minItems: 1 · maxItems: 64

Array item
object

A direction on the associated video's clock, never the stream clock.

Additional properties: not allowed

end_image_urlstring | null

Exact final frame of the chunk that ends at this offset. Offsets of successive end images must be at least three seconds apart.

Default: null

At least 1 character when not null

offsetinteger* required

Whole seconds from the start of the first video generated under this script. Text and audio placed here start at this second; an end image here is the exact final frame at this second.

minimum: 0

audio_urlstring | null

Audio that starts playing exactly at this offset as FL2VA target audio; overlapping sources are mixed. The source PCM itself plays.

Default: null

At least 1 character when not null

promptstring | null

Direction that starts at this offset and persists until the next text beat; it also appears as a timed span inside the chunk.

Default: null

At least 1 character when not null · At most 50000 characters when not null

Alternative 2
null
typestring* required

Constant: "configure"

Drop a fixed ensemble into audience-created misunderstandings.

{
  "prompt_version": 1,
  "resolution": "768p",
  "aspect_ratio": "16:9",
  "type": "configure",
  "prompt": "A continuous original live-action American sitcom produced in 1994, following the same ensemble of adult roommates, coworkers, neighbors, and rivals. Preserve appearances, apartment and workplace layouts, relationships, jobs, secrets, running jokes, and unresolved storylines. Advance through dialogue, entrances, misunderstandings, escalating attempts to hide mistakes, reversals, and warm character payoffs. Avoid references to existing sitcoms or actors.",
  "protocol_version": 1,
  "memory": 3
}

Client ping message

type: "ping"
payload: object

Additional properties: not allowed

typestring* required

Constant: "ping"

tsnumber* required
{
  "type": "ping",
  "ts": 0
}

Client prompt message

type: "prompt"
payload: object

Correlation field: /prompt_version

Additional properties: not allowed

promptstring | null

Default: null

At least 1 character when not null · At most 50000 characters when not null

end_image_urlstring | null

Default: null

At least 1 character when not null

audio_urlstring | null

FL2VA target audio for future chunks; this is not a Ref2VA reference. With audio_behavior 'replace' (default) it starts at the next undispatched chunk and drops any queued audio; with 'queue' it plays after every previously accepted source ends, sample-exact.

Default: null

At least 1 character when not null

audio_behaviorstring

'replace' cuts to this audio at the next chunk; 'queue' appends it.

Default: "replace"

Values: "replace", "queue"

replanboolean

When a prompt is included: true (default) busts the planned prompt queue so the new direction applies at the next undispatched chunk; false appends the direction after the already-planned chunks.

Default: true

prompt_versioninteger* required

minimum: 1

script_modestring

'replace' cuts to the script at the next chunk; 'append' queues it.

Default: "replace"

Values: "replace", "append"

scriptobject[] | null

A new script: beats at whole-second offsets from the first video generated under it. Exclusive with prompt/end_image_url/audio_url. With script_mode 'replace' (default) it replaces the pending and current direction at the next undispatched chunk; 'append' starts at the chunk boundary after the running script's last text beat is represented and its audio tails have finished (queued behind earlier appended scripts). Beats without a prompt keep the current text direction.

Default: null

Must match at least one of the following:

Alternative 1
object[]

minItems: 1 · maxItems: 64

Array item
object

A direction on the associated video's clock, never the stream clock.

Additional properties: not allowed

end_image_urlstring | null

Exact final frame of the chunk that ends at this offset. Offsets of successive end images must be at least three seconds apart.

Default: null

At least 1 character when not null

offsetinteger* required

Whole seconds from the start of the first video generated under this script. Text and audio placed here start at this second; an end image here is the exact final frame at this second.

minimum: 0

audio_urlstring | null

Audio that starts playing exactly at this offset as FL2VA target audio; overlapping sources are mixed. The source PCM itself plays.

Default: null

At least 1 character when not null

promptstring | null

Direction that starts at this offset and persists until the next text beat; it also appears as a timed span inside the chunk.

Default: null

At least 1 character when not null · At most 50000 characters when not null

Alternative 2
null
typestring* required

Constant: "prompt"

Send a new direction after configuration is acknowledged

{
  "prompt_version": 2,
  "script_mode": "replace",
  "type": "prompt",
  "prompt": "They follow a narrow path down to the harbor.",
  "replan": true,
  "audio_behavior": "replace"
}

Client stop message

type: "stop"
payload: object

Additional properties: not allowed

typestring* required

Constant: "stop"

{
  "type": "stop"
}

Server messages#

Server audio_applied message

type: "audio_applied"
payload: object

An audio update was accepted; it conditions the next eligible chunk.

starts_at_chunk_index is the first chunk that can carry it (the next undispatched chunk for replace; for queue the chunk where the preceding sources end, if already known).

prompt_versioninteger* required

minimum: 1

transcribedboolean* required
remaining_secondsnumber* required

minimum: 0

duration_secondsnumber* required

exclusiveMinimum: 0

behaviorstring* required

Values: "replace", "queue"

queued_sourcesinteger* required

minimum: 0

sourcestring* required
typestring* required

Constant: "audio_applied"

{
  "prompt_version": 1,
  "transcribed": true,
  "remaining_seconds": 0,
  "duration_seconds": 5e-324,
  "behavior": "replace",
  "queued_sources": 0,
  "source": "string",
  "type": "audio_applied"
}

Server audio_exhausted message

type: "audio_exhausted"
payload: object

Accepted audio ran out inside this chunk; the rest is silence.

Later chunks continue without audio conditioning (checkpoint continuation on the same family) until new audio is accepted.

source_versioninteger | null

Default: null

minimum: 0 when not null

typestring* required

Constant: "audio_exhausted"

silent_secondsnumber* required

minimum: 0

chunk_indexinteger* required

minimum: 0

{
  "type": "audio_exhausted",
  "silent_seconds": 0,
  "chunk_index": 0
}

Server audio_pending message

type: "audio_pending"
payload: object
prompt_versioninteger* required

minimum: 1

behaviorstring* required

Values: "replace", "queue"

typestring* required

Constant: "audio_pending"

{
  "prompt_version": 1,
  "behavior": "replace",
  "type": "audio_pending"
}

Server audio_rejected message

type: "audio_rejected"
payload: object
prompt_versioninteger* required

minimum: 1

reasonstring* required

Values: "invalid_audio", "content_policy", "preparation_failed", "queue_full", "stale_prompt_version"

typestring* required

Constant: "audio_rejected"

errorstring* required
{
  "prompt_version": 1,
  "reason": "invalid_audio",
  "type": "audio_rejected",
  "error": "string"
}

Server chunk message

type: "chunk"
payload: object
prompt_versioninteger* required

minimum: 1

generation_secondsnumber* required

minimum: 0

next_generation_estimate_secondsnumber* required

minimum: 0

presented_frame_countinteger | null

Default: null

exclusiveMinimum: 0 when not null

trimmed_context_framesinteger* required

minimum: 0

native_playable_frame_countinteger | null

Default: null

exclusiveMinimum: 0 when not null

dispatchobject* required
wall_msnumber* required

minimum: 0

classified_msnumber* required

minimum: 0

overhead_msnumber* required

minimum: 0

phases_msobject* required

Additional properties: number

number
script_offset_secondsinteger | null

Default: null

minimum: 0 when not null

buffer_depth_secondsnumber* required

minimum: 0

chunk_indexinteger* required

minimum: 0

routestring* required

Values: "gorgonea", "betelgeuse", "regulus", "unknown"

scheduling_lead_msnumber* required
scheduling_slack_msnumber* required
hard_cutboolean

Default: false

script_end_keyframeboolean | null

Default: null

buffer_depth_chunksinteger* required

minimum: 0

requested_duration_secondsinteger* required

minimum: 5 · maximum: 15

typestring* required

Constant: "chunk"

playback_secondsnumber* required

exclusiveMinimum: 0

generated_frame_countinteger* required

exclusiveMinimum: 0

script_versioninteger | null

Default: null

minimum: 1 when not null

{
  "prompt_version": 1,
  "generation_seconds": 0,
  "next_generation_estimate_seconds": 0,
  "trimmed_context_frames": 0,
  "dispatch": {
    "wall_ms": 0,
    "classified_ms": 0,
    "overhead_ms": 0,
    "phases_ms": {}
  },
  "buffer_depth_seconds": 0,
  "chunk_index": 0,
  "route": "gorgonea",
  "scheduling_lead_ms": 0,
  "scheduling_slack_ms": 0,
  "buffer_depth_chunks": 0,
  "requested_duration_seconds": 5,
  "type": "chunk",
  "playback_seconds": 5e-324,
  "generated_frame_count": 1
}

Server chunk_metrics message

type: "chunk_metrics"
payload: object
routestring* required

Values: "gorgonea", "betelgeuse", "regulus", "unknown"

chunk_consumable_interval_msnumber | null

Default: null

minimum: 0 when not null

unitsstring* required

Constant: "ms"

gaugesobject* required

Additional properties: number

number
chunk_consumable_ready_msnumber | null

Default: null

minimum: 0 when not null

phases_msobject* required

Additional properties: number

number
chunk_indexinteger* required

minimum: 0

typestring* required

Constant: "chunk_metrics"

{
  "route": "gorgonea",
  "units": "ms",
  "gauges": {},
  "phases_ms": {},
  "chunk_index": 0,
  "type": "chunk_metrics"
}

Server configured message

type: "configured"
payload: object

Correlation field: /prompt_version

prompt_versioninteger* required

minimum: 1

has_initial_audioboolean | null

Default: null

aspect_ratiostring | null

Default: null

When not null:

Values: "16:9", "9:16", "1:1"

has_initial_imageboolean | null

Default: null

memoryinteger | null

Default: null

minimum: 1 when not null · maximum: 50 when not null

resolutionstring | string | null

Default: null

Must match at least one of the following:

Alternative 1
string

Values: "480p", "544p", "640p", "704p", "768p"

Alternative 2
string

Values: "480p", "768p", "1080p"

Alternative 3
null
audio_bitrateinteger | null

Default: null

When not null:

Values: 96000, 128000, 192000

accelerationstring | null

Default: null

When not null:

Values: "none", "regular"

typestring* required

Constant: "configured"

enable_safety_checkerboolean* required
chunk_durationinteger | null

Default: null

minimum: 5 when not null · maximum: 15 when not null

{
  "prompt_version": 1,
  "type": "configured",
  "enable_safety_checker": true
}

Server deadline_missed message

type: "deadline_missed"
payload: object
late_by_secondsnumber* required

minimum: 0

behaviorstring* required

Constant: "freeze_video_and_silence_audio_until_ready"

typestring* required

Constant: "deadline_missed"

chunk_indexinteger* required

minimum: 0

{
  "late_by_seconds": 0,
  "behavior": "freeze_video_and_silence_audio_until_ready",
  "type": "deadline_missed",
  "chunk_index": 0
}

Server error message

type: "error"
payload: object

Correlation field: /prompt_version

prompt_versioninteger | null

Default: null

minimum: 1 when not null

codestring* required

Values: "balance_unavailable", "content_policy", "configuration_timeout", "generation_failed", "generation_timeout", "immutable_settings", "initialization_timeout", "invalid_initial_image", "invalid_initial_audio", "invalid_initial_script", "invalid_input", "invalid_message", "not_configured", "stale_prompt_version"

typestring* required

Constant: "error"

detailobject[] | null

Default: null

Must match at least one of the following:

Alternative 1
object[]
Array item
object
Alternative 2
null
errorstring* required
{
  "code": "balance_unavailable",
  "type": "error",
  "error": "string"
}

Server pong message

type: "pong"
payload: object
typestring* required

Constant: "pong"

client_tsnumber* required
{
  "type": "pong",
  "client_ts": 0
}

Server prompt_applied message

type: "prompt_applied"
payload: object

Correlation field: /prompt_version

A direction (or script) is admitted; the script fields report scripts.

script_origin_chunk_index is the first chunk generated under the script when that is already fixed (a replacement binds to the next undispatched chunk); an appended script reports None until it starts.

prompt_versioninteger* required

minimum: 1

script_modestring | null

Default: null

When not null:

Values: "replace", "append"

typestring* required

Constant: "prompt_applied"

script_origin_chunk_indexinteger | null

Default: null

minimum: 0 when not null

script_queuedinteger | null

Default: null

minimum: 0 when not null

script_beatsinteger | null

Default: null

minimum: 1 when not null

{
  "prompt_version": 1,
  "type": "prompt_applied"
}

Server prompt_pending message

type: "prompt_pending"
payload: object

Correlation field: /prompt_version

prompt_versioninteger* required

minimum: 1

typestring* required

Constant: "prompt_pending"

{
  "prompt_version": 1,
  "type": "prompt_pending"
}

Server prompt_rejected message

type: "prompt_rejected"
payload: object

Correlation field: /prompt_version

prompt_versioninteger* required

minimum: 1

reasonstring* required

Values: "content_policy", "preparation_failed", "stale_prompt_version", "invalid_script", "infeasible_timing", "invalid_audio", "invalid_image", "queue_full"

typestring* required

Constant: "prompt_rejected"

errorstring | null

Default: null

{
  "prompt_version": 1,
  "reason": "content_policy",
  "type": "prompt_rejected"
}

Server session_info message

type: "session_info"
payload: object
script_session_max_decoded_audio_bytesinteger

Default: 335544320

fpsinteger

Constant: 24

one_session_per_machineboolean

Constant: true

max_audio_source_secondsnumber

Default: 600

audio_behaviorsstring[]

Default: ["replace","queue"]

Array item
string

Values: "replace", "queue"

scriptsboolean

Constant: true

audio_sample_rateinteger

Constant: 48000

accelerationsstring[]

Default: ["none","regular"]

Array item
string

Values: "none", "regular"

script_max_queuedinteger

Constant: 4

controller_machine_typestring

Constant: "XL"

script_modesstring[]

Default: ["replace","append"]

Array item
string

Values: "replace", "append"

script_max_audio_beatsinteger

Constant: 8

script_min_opening_chunk_secondsinteger

Constant: 5

default_memoryinteger

Constant: 12

aspect_ratiosstring[]

Default: ["16:9","9:16","1:1"]

Array item
string

Values: "16:9", "9:16", "1:1"

min_chunk_durationinteger

Constant: 5

session_limit_scopestring

Default: "configured"

Values: "configured", "effective"

script_min_end_image_spacing_secondsinteger

Constant: 3

chunk_secondsinteger

Constant: 10

opening_reservedboolean

Default: false

script_max_beatsinteger

Constant: 64

appstring

Constant: "minimax-h3-max-director"

script_min_chunk_secondsinteger

Constant: 3

prompt_context_segmentsinteger

Constant: 12

max_chunk_durationinteger

Constant: 15

prompt_deck_sizeinteger

Constant: 6

client_message_typesstring[]

Default: ["configure","ping","prompt","stop"]

Array item
string

Values: "configure", "ping", "prompt", "stop"

default_accelerationstring

Constant: "regular"

default_chunk_durationinteger

Constant: 10

backend_selectionstring

Constant: "minimax-h3-turbo-balancer"

prompt_expanderstring

Constant: "fast"

script_max_decoded_audio_bytesinteger

Default: 67108864

script_max_pendinginteger

Constant: 4

continuation_context_framesinteger

Constant: 39

min_memoryinteger

Constant: 1

resolutionsstring[]

Default: ["480p","768p","1080p"]

Array item
string

Values: "480p", "768p", "1080p"

max_session_secondsnumber | null

Default: null

protocol_versioninteger

Constant: 1

audio_bitratesinteger[]

Default: [96000,128000,192000]

Array item
integer

Values: 96000, 128000, 192000

audio_conditioningboolean

Constant: true

script_max_end_imagesinteger

Constant: 16

max_memoryinteger

Constant: 50

default_audio_bitrateinteger | null

Default: null

When not null:

Values: 96000, 128000, 192000

continuation_playback_secondsnumber

Default: 8.5

conditioning_audio_sample_rateinteger

Constant: 32000

server_message_typesstring[]

Default: ["audio_applied","audio_exhausted","audio_pending","audio_rejected","chunk","chunk_metrics","configured","deadline_missed","error","pong","prompt_applied","prompt_pending","prompt_rejected","session_info","session_metrics","stream_exhausted"]

Array item
string

Values: "audio_applied", "audio_exhausted", "audio_pending", "audio_rejected", "chunk", "chunk_metrics", "configured", "deadline_missed", "error", "pong", "prompt_applied", "prompt_pending", "prompt_rejected", "session_info", "session_metrics", "stream_exhausted"

typestring* required

Constant: "session_info"

{
  "fps": 24,
  "one_session_per_machine": true,
  "scripts": true,
  "audio_sample_rate": 48000,
  "script_max_queued": 4,
  "controller_machine_type": "XL",
  "script_max_audio_beats": 8,
  "script_min_opening_chunk_seconds": 5,
  "default_memory": 12,
  "min_chunk_duration": 5,
  "script_min_end_image_spacing_seconds": 3,
  "chunk_seconds": 10,
  "script_max_beats": 64,
  "app": "minimax-h3-max-director",
  "script_min_chunk_seconds": 3,
  "prompt_context_segments": 12,
  "max_chunk_duration": 15,
  "prompt_deck_size": 6,
  "default_acceleration": "regular",
  "default_chunk_duration": 10,
  "backend_selection": "minimax-h3-turbo-balancer",
  "prompt_expander": "fast",
  "script_max_pending": 4,
  "continuation_context_frames": 39,
  "min_memory": 1,
  "protocol_version": 1,
  "audio_conditioning": true,
  "script_max_end_images": 16,
  "max_memory": 50,
  "conditioning_audio_sample_rate": 32000,
  "type": "session_info"
}

Server session_metrics message

type: "session_metrics"
payload: object
unitsstring* required

Constant: "ms"

phasesobject* required

Additional properties: object

object
p95_msnumber* required

minimum: 0

countinteger* required

minimum: 0

max_msnumber* required

minimum: 0

p50_msnumber* required

minimum: 0

total_msnumber* required

minimum: 0

history_limitinteger* required

minimum: 1

gaugesobject* required

Additional properties: number

number
typestring* required

Constant: "session_metrics"

finalboolean

Default: false

session_wall_msnumber* required

minimum: 0

history_sizeinteger* required

minimum: 0

{
  "units": "ms",
  "phases": {},
  "history_limit": 1,
  "gauges": {},
  "type": "session_metrics",
  "session_wall_ms": 0,
  "history_size": 0
}

Server stream_exhausted message

type: "stream_exhausted"
payload: object
reasonstring* required

Values: "stopped", "session_limit"

typestring* required

Constant: "stream_exhausted"

chunksinteger* required

minimum: 0

{
  "reason": "stopped",
  "type": "stream_exhausted",
  "chunks": 0
}

Related Models