minimax/h3-max/director

Direct continuous, realtime video streams with live prompts while preserving characters, settings, and story continuity.

Inference

Commercial use

Schema AsyncAPI

LLMs

Playground API

About

Start Session

1. Calling the API#

Install the client#

The client provides a convenient way to interact with the model API.

npm install @fal-ai/client@alpha @fal-ai/server-proxy@alpha

Experimental realtime API

fal.realtime.open is currently available on the alpha release and may change in a minor version.

Set up the server proxy#

Keep FAL_KEY on your server and expose an authenticated proxy route to your browser. For a Next.js App Router application, create app/api/fal/proxy/route.ts:

import { route } from "@fal-ai/server-proxy/nextjs";

export const { GET, POST, PUT } = route;

Set FAL_KEY in the server environment, and protect this route with your application's authentication before deploying it.

Real-time via WebRTC#

This deployment publishes a WMA WebRTC contract. The example uses fal.realtime.open; its media handlers, capture constraints, and control message are generated from the linked AsyncAPI document.

import { createFalClient } from "@fal-ai/client";
import { wma } from "@fal-ai/client/realtime";

const fal = createFalClient({ proxyUrl: "/api/fal/proxy" });

const session = fal.realtime.open(wma("minimax/h3-max/director"), {
  receive: [
    "video",
    "audio"
  ],
  onMedia: (stream) => {
      const element = document.querySelector("video");
      if (element) element.srcObject = stream;
    },
  onData: (raw) => {
      const message = JSON.parse(raw);
      console.log(message);
    },
  onState: (state) => console.log("realtime:", state),
  onError: (error) => console.error(error),
});


// Send any client message described below.
session.send({
  "protocol_version": 1,
  "prompt": "A continuous original live-action stream following a group of friends as they explore a new city.",
  "type": "configure",
  "prompt_version": 1
});

// Later, release the peer connection and network resources:
// await session.close();

2. Authentication#

The browser connects through your server proxy, which reads FAL_KEY from the server environment. Never put that key in browser code.

API Key#

Authenticate users before allowing them to call your proxy route so other people cannot spend against your account.

Protect your API Key

When running code on the client-side (e.g. in a browser, mobile app or GUI applications), make sure to not expose your FAL_KEY. Instead, use a server-side proxy to make requests to the API. For more information, check out our server-side integration guide.

3. Schema#

Media#

Client contract for the WebRTC session created by the linked OpenAPI operation.

Tracks are described from the browser's perspective. Send tracks are captured by the browser and sent to the model; receive tracks come back from the model.

Send

No send media tracks.

Receive

video

optional

{
  "frameRate": 24
}

audio

optional

Client messages#

Client configure message

type: "configure"

payload: object

Additional properties: not allowed

memoryinteger

Number of prior segment prompts retained as context for future prompt expansion.

Default: 12

minimum: 1 · maximum: 50

protocol_versioninteger

Constant: 1

promptstring* required

minLength: 1 · maxLength: 50000

seedinteger | null

Default: null

typestring* required

Constant: "configure"

resolutionstring

Default: "768p"

Values: "480p", "768p"

aspect_ratiostring

Default: "16:9"

Values: "16:9", "9:16", "1:1"

prompt_versioninteger* required

minimum: 1

{
  "protocol_version": 1,
  "prompt": "A continuous original live-action stream following a group of friends as they explore a new city.",
  "type": "configure",
  "prompt_version": 1
}

Client ping message

type: "ping"

payload: object

Additional properties: not allowed

typestring* required

Constant: "ping"

tsnumber* required

{
  "type": "ping",
  "ts": 0
}

Client prompt message

type: "prompt"

payload: object

Additional properties: not allowed

promptstring* required

minLength: 1 · maxLength: 50000

prompt_versioninteger* required

minimum: 1

typestring* required

Constant: "prompt"

{
  "prompt": "string",
  "prompt_version": 1,
  "type": "prompt"
}

Client stop message

type: "stop"

payload: object

Additional properties: not allowed

typestring* required

Constant: "stop"

{
  "type": "stop"
}

Server messages#

Server chunk message

type: "chunk"

payload: object

buffer_depth_secondsnumber* required

minimum: 0

scheduling_lead_msnumber* required

playback_secondsnumber* required

exclusiveMinimum: 0

routestring* required

Values: "gorgonea", "betelgeuse", "regulus", "unknown"

typestring* required

Constant: "chunk"

prompt_versioninteger* required

minimum: 1

dispatchobject* required

phases_msobject* required

Additional properties: number

number

overhead_msnumber* required

minimum: 0

wall_msnumber* required

minimum: 0

classified_msnumber* required

minimum: 0

chunk_indexinteger* required

minimum: 0

next_generation_estimate_secondsnumber* required

minimum: 0

generation_secondsnumber* required

minimum: 0

requested_duration_secondsinteger* required

minimum: 5 · maximum: 15

scheduling_slack_msnumber* required

trimmed_context_framesinteger* required

minimum: 0

buffer_depth_chunksinteger* required

minimum: 0

generated_frame_countinteger* required

exclusiveMinimum: 0

{
  "buffer_depth_seconds": 0,
  "scheduling_lead_ms": 0,
  "playback_seconds": 5e-324,
  "route": "gorgonea",
  "type": "chunk",
  "prompt_version": 1,
  "dispatch": {
    "phases_ms": {},
    "overhead_ms": 0,
    "wall_ms": 0,
    "classified_ms": 0
  },
  "chunk_index": 0,
  "next_generation_estimate_seconds": 0,
  "generation_seconds": 0,
  "requested_duration_seconds": 5,
  "scheduling_slack_ms": 0,
  "trimmed_context_frames": 0,
  "buffer_depth_chunks": 0,
  "generated_frame_count": 1
}

Server chunk_metrics message

type: "chunk_metrics"

payload: object

chunk_indexinteger* required

minimum: 0

phases_msobject* required

Additional properties: number

number

gaugesobject* required

Additional properties: number

number

unitsstring* required

Constant: "ms"

routestring* required

Values: "gorgonea", "betelgeuse", "regulus", "unknown"

typestring* required

Constant: "chunk_metrics"

chunk_consumable_ready_msnumber | null

Default: null

anyOf option 1

number

minimum: 0

anyOf option 2

null

chunk_consumable_interval_msnumber | null

Default: null

anyOf option 1

number

minimum: 0

anyOf option 2

null

{
  "chunk_index": 0,
  "phases_ms": {},
  "gauges": {},
  "units": "ms",
  "route": "gorgonea",
  "type": "chunk_metrics"
}

Server configured message

type: "configured"

payload: object

memoryinteger | null

Default: null

anyOf option 1

integer

minimum: 1 · maximum: 50

anyOf option 2

null

chunk_durationinteger | null

Default: null

anyOf option 1

integer

minimum: 5 · maximum: 15

anyOf option 2

null

accelerationstring | null

Default: null

anyOf option 1

string

Values: "none", "regular"

anyOf option 2

null

typestring* required

Constant: "configured"

has_initial_imageboolean | null

Default: null

enable_safety_checkerboolean* required

resolutionstring | string | null

Default: null

anyOf option 1

string

Values: "480p", "544p", "640p", "704p", "768p"

anyOf option 2

string

Values: "480p", "768p"

anyOf option 3

null

prompt_versioninteger* required

minimum: 1

aspect_ratiostring | null

Default: null

anyOf option 1

string

Values: "16:9", "9:16", "1:1"

anyOf option 2

null

{
  "type": "configured",
  "enable_safety_checker": true,
  "prompt_version": 1
}

Server deadline_missed message

type: "deadline_missed"

payload: object

behaviorstring* required

Constant: "freeze_video_and_silence_audio_until_ready"

chunk_indexinteger* required

minimum: 0

typestring* required

Constant: "deadline_missed"

late_by_secondsnumber* required

minimum: 0

{
  "behavior": "freeze_video_and_silence_audio_until_ready",
  "chunk_index": 0,
  "type": "deadline_missed",
  "late_by_seconds": 0
}

Server error message

type: "error"

payload: object

detailobject[] | null

Default: null

anyOf option 1

object[]

Array item

object

anyOf option 2

null

codestring* required

Values: "content_policy", "configuration_timeout", "generation_failed", "generation_timeout", "immutable_settings", "initialization_timeout", "invalid_initial_image", "invalid_message", "not_configured", "stale_prompt_version"

prompt_versioninteger | null

Default: null

anyOf option 1

integer

minimum: 1

anyOf option 2

null

typestring* required

Constant: "error"

errorstring* required

{
  "code": "content_policy",
  "type": "error",
  "error": "string"
}

Server pong message

type: "pong"

payload: object

client_tsnumber* required

typestring* required

Constant: "pong"

{
  "client_ts": 0,
  "type": "pong"
}

Server prompt_applied message

type: "prompt_applied"

payload: object

prompt_versioninteger* required

minimum: 1

typestring* required

Constant: "prompt_applied"

{
  "prompt_version": 1,
  "type": "prompt_applied"
}

Server prompt_pending message

type: "prompt_pending"

payload: object

prompt_versioninteger* required

minimum: 1

typestring* required

Constant: "prompt_pending"

{
  "prompt_version": 1,
  "type": "prompt_pending"
}

Server prompt_rejected message

type: "prompt_rejected"

payload: object

reasonstring* required

Constant: "content_policy"

prompt_versioninteger* required

minimum: 1

typestring* required

Constant: "prompt_rejected"

{
  "reason": "content_policy",
  "prompt_version": 1,
  "type": "prompt_rejected"
}

Server session_info message

type: "session_info"

payload: object

protocol_versioninteger

Constant: 1

prompt_deck_sizeinteger

Constant: 3

chunk_secondsinteger

Constant: 10

min_chunk_durationinteger

Constant: 5

resolutionsstring[]

Default: ["480p","768p"]

Array item

string

Values: "480p", "768p"

server_message_typesstring[]

Default: ["chunk","chunk_metrics","configured","deadline_missed","error","pong","prompt_applied","prompt_pending","prompt_rejected","session_info","session_metrics","stream_exhausted"]

Array item

string

Values: "chunk", "chunk_metrics", "configured", "deadline_missed", "error", "pong", "prompt_applied", "prompt_pending", "prompt_rejected", "session_info", "session_metrics", "stream_exhausted"

appstring

Constant: "minimax-h3-max-director"

accelerationsstring[]

Default: ["none","regular"]

Array item

string

Values: "none", "regular"

typestring* required

Constant: "session_info"

continuation_playback_secondsnumber

Default: 8.5

client_message_typesstring[]

Default: ["configure","ping","prompt","stop"]

Array item

string

Values: "configure", "ping", "prompt", "stop"

default_memoryinteger

Constant: 12

max_chunk_durationinteger

Constant: 15

continuation_context_framesinteger

Constant: 39

prompt_expanderstring

Constant: "fast"

default_chunk_durationinteger

Constant: 10

controller_machine_typestring

Constant: "XL"

backend_selectionstring

Constant: "minimax-h3-turbo-balancer"

default_accelerationstring

Constant: "regular"

max_memoryinteger

Constant: 50

min_memoryinteger

Constant: 1

audio_sample_rateinteger

Constant: 32000

prompt_context_segmentsinteger

Constant: 12

one_session_per_machineboolean

Constant: true

aspect_ratiosstring[]

Default: ["16:9","9:16","1:1"]

Array item

string

Values: "16:9", "9:16", "1:1"

fpsinteger

Constant: 24

max_session_secondsnumber | null

Default: null

{
  "protocol_version": 1,
  "prompt_deck_size": 3,
  "chunk_seconds": 10,
  "min_chunk_duration": 5,
  "app": "minimax-h3-max-director",
  "type": "session_info",
  "default_memory": 12,
  "max_chunk_duration": 15,
  "continuation_context_frames": 39,
  "prompt_expander": "fast",
  "default_chunk_duration": 10,
  "controller_machine_type": "XL",
  "backend_selection": "minimax-h3-turbo-balancer",
  "default_acceleration": "regular",
  "max_memory": 50,
  "min_memory": 1,
  "audio_sample_rate": 32000,
  "prompt_context_segments": 12,
  "one_session_per_machine": true,
  "fps": 24
}

Server session_metrics message

type: "session_metrics"

payload: object

finalboolean

Default: false

gaugesobject* required

Additional properties: number

number

unitsstring* required

Constant: "ms"

history_sizeinteger* required

minimum: 0

typestring* required

Constant: "session_metrics"

history_limitinteger* required

minimum: 1

session_wall_msnumber* required

minimum: 0

phasesobject* required

Additional properties: object

object

p50_msnumber* required

minimum: 0

total_msnumber* required

minimum: 0

max_msnumber* required

minimum: 0

p95_msnumber* required

minimum: 0

countinteger* required

minimum: 0

{
  "gauges": {},
  "units": "ms",
  "history_size": 0,
  "type": "session_metrics",
  "history_limit": 1,
  "session_wall_ms": 0,
  "phases": {}
}

Server stream_exhausted message

type: "stream_exhausted"

payload: object

reasonstring* required

Values: "stopped", "session_limit"

chunksinteger* required

minimum: 0

typestring* required

Constant: "stream_exhausted"

{
  "reason": "stopped",
  "chunks": 0,
  "type": "stream_exhausted"
}

minimax/h3-max/director

Table of contents

1. Calling the API

2. Authentication

3. Schema

About

1. Calling the API#

Install the client#

Experimental realtime API

Set up the server proxy#

Real-time via WebRTC#

2. Authentication#

API Key#

Protect your API Key

3. Schema#

Media#

Send

Receive

Client messages#

Client configure message

Client ping message

Client prompt message

Client stop message

Server messages#

Server chunk message

Server chunk_metrics message

Server configured message

Server deadline_missed message

Server error message

Server pong message

Server prompt_applied message

Server prompt_pending message

Server prompt_rejected message

Server session_info message

Server session_metrics message

Server stream_exhausted message

Related Models