Eden Live 14B Video Video to Video API Docs | fal

Real-time generative rendering from any reference using VAST's Eden Live 14B

vast/eden-live-14b-video

Real-time generative rendering from any reference using VAST's Eden Live 14B

Inference

Commercial use

Schema AsyncAPI

LLMs

Playground API

About

Start Session

1. Calling the API#

Install the client#

The client provides a convenient way to interact with the model API.

npm install @fal-ai/client@alpha @fal-ai/server-proxy@alpha

Experimental realtime API

fal.realtime.open is currently available on the alpha release and may change in a minor version.

Set up the server proxy#

Keep FAL_KEY on your server and expose an authenticated proxy route to your browser. For a Next.js App Router application, create app/api/fal/proxy/route.ts:

import { route } from "@fal-ai/server-proxy/nextjs";

export const { GET, POST, PUT } = route;

Set FAL_KEY in the server environment, and protect this route with your application's authentication before deploying it.

Real-time via WebRTC#

This deployment publishes a WMA WebRTC contract. The example uses fal.realtime.open; its media handlers, capture constraints, and control message are generated from the linked AsyncAPI document.

import { createFalClient } from "@fal-ai/client";
import { wma } from "@fal-ai/client/realtime";

const fal = createFalClient({ proxyUrl: "/api/fal/proxy" });

const deviceStream = await navigator.mediaDevices.getUserMedia({
  "video": {
    "frameRate": 16,
    "height": 480,
    "width": 832
  },
  "audio": false
});
const localStream = deviceStream;

const session = fal.realtime.open(wma("vast/eden-live-14b-video"), {
  receive: [
    "video"
  ],
  localStream,
  onMedia: (stream) => {
      const element = document.querySelector("video");
      if (element) element.srcObject = stream;
    },
  onData: (raw) => {
      const message = JSON.parse(raw);
      console.log(message);
    },
  onState: (state) => console.log("realtime:", state),
  onError: (error) => console.error(error),
});


// Send any client message described below.
session.send({
  "prompt": "Change to anime style",
  "enable_prompt_expansion": true,
  "type": "configure"
});

// Later, release the peer connection and network resources:
// localStream.getTracks().forEach((track) => track.stop());
// await session.close();

2. Authentication#

The browser connects through your server proxy, which reads FAL_KEY from the server environment. Never put that key in browser code.

API Key#

Authenticate users before allowing them to call your proxy route so other people cannot spend against your account.

Protect your API Key

When running code on the client-side (e.g. in a browser, mobile app or GUI applications), make sure to not expose your FAL_KEY. Instead, use a server-side proxy to make requests to the API. For more information, check out our server-side integration guide.

3. Schema#

Media#

Client contract for the WebRTC session created by the linked OpenAPI operation.

Tracks are described from the browser's perspective. Send tracks are captured by the browser and sent to the model; receive tracks come back from the model.

Send

video

required

{
  "frameRate": 16,
  "height": 480,
  "width": 832
}

Receive

video

optional

{
  "frameRate": 16,
  "height": 480,
  "width": 832
}

Client messages#

Set conditioning

type: "configure"

payload: object

Additional properties: not allowed

Set the session's conditioning, once per connection; await configured.

Without image_url the world anchors on a reference generated from the newest input frame and the prompt (FLUX.2 [klein] 4B edit), or on that frame itself when generation fails.

promptstring | null

What the input video should look like. Defaults to a generic photoreal prompt.

Default: null

At most 2000 characters when not null

image_urlstring | null

Rendering of the input's first frame in the target look (URL, or a data URI under ~45 KB); anchors the generated world. Without it, one is generated from the newest input frame and the prompt.

Default: null

At most 60000 characters when not null

enable_prompt_expansionboolean

Rewrite the scene description (and later prompt changes) into a detailed one with an LLM, using the reference image when given. Adds about a second to session start and to each prompt change.

Default: false

seedinteger | null

Noise seed; the same inputs and seed reproduce the output.

Default: null

minimum: 0 when not null · maximum: 2147483647 when not null

typestring* required

Constant: "configure"

Example

{
  "prompt": "Change to anime style",
  "enable_prompt_expansion": true,
  "type": "configure"
}

Client ping message

type: "ping"

payload: object

Latency probe; the server echoes ts back in a pong payload.

tsnumber | null

Default: null

typestring* required

Constant: "ping"

{
  "type": "ping"
}

Update prompt

type: "prompt"

payload: object

Additional properties: not allowed

Change the scene description; applied at the next chunk boundary.

promptstring* required

Text the model renders the scene with. Keeps the current world.

At least 1 character · At most 2000 characters

typestring* required

Constant: "prompt"

{
  "prompt": "string",
  "type": "prompt"
}

Restart

type: "reset"

payload: object

Additional properties: not allowed

Restart generation from a new anchor; unset fields keep their values, and an explicit null clears image_url or seed.

The world is re-anchored on a reference generated from the newest input frame unless image_url is set.

promptstring | null

What the input video should look like. Defaults to a generic photoreal prompt.

Default: null

At most 2000 characters when not null

image_urlstring | null

Rendering of the input's first frame in the target look (URL, or a data URI under ~45 KB); anchors the generated world. Without it, one is generated from the newest input frame and the prompt.

Default: null

At most 60000 characters when not null

seedinteger | null

Noise seed; the same inputs and seed reproduce the output.

Default: null

minimum: 0 when not null · maximum: 2147483647 when not null

enable_prompt_expansionboolean | null

Default: null

typestring* required

Constant: "reset"

{
  "type": "reset"
}

End session

type: "stop"

payload: object

Additional properties: not allowed

End the session.

typestring* required

Constant: "stop"

{
  "type": "stop"
}

Server messages#

Server configured message

type: "configured"

payload: object

gameobject | null

Default: null

Must match at least one of the following:

GameInfo

object

descriptionstring* required

thumbnail_urlstring | null

Default: null

promptstring* required

instructionsstring[]* required

Array item

string

titlestring* required

controlsobject[]* required

Array item

object

keystring* required

commandstring | null

Semantic command name.

Default: null

labelstring* required

holdboolean

Default: true

actionsstring[]* required

Array item

string

idstring* required

Alternative 2

null

typestring* required

Constant: "configured"

{
  "type": "configured"
}

Server error message

type: "error"

payload: object

retryableboolean | null

Default: null

typestring* required

Constant: "error"

errorstring* required

{
  "type": "error",
  "error": "string"
}

Server pong message

type: "pong"

payload: object

Reply to a ping, echoing the client's timestamp for RTT measurement.

The echoed timestamp has two wire spellings: apps that handle ping themselves echo ts; the fallback handler in :class:registry.wma.sdk.Session replies with client_ts. Clients read whichever is present. server_ts is the (unsynchronized) runner clock; it only shows how the round trip divides once the client has an offset estimate.

client_tsnumber | null

Default: null

server_tsnumber | null

Default: null

tsnumber | null

Default: null

typestring* required

Constant: "pong"

{
  "type": "pong"
}

Server prompt_applied message

type: "prompt_applied"

payload: object

typestring* required

Constant: "prompt_applied"

{
  "type": "prompt_applied"
}

Server reset_applied message

type: "reset_applied"

payload: object

gameobject | null

Default: null

Must match at least one of the following:

GameInfo

object

descriptionstring* required

thumbnail_urlstring | null

Default: null

promptstring* required

instructionsstring[]* required

Array item

string

titlestring* required

controlsobject[]* required

Array item

object

keystring* required

commandstring | null

Semantic command name.

Default: null

labelstring* required

holdboolean

Default: true

actionsstring[]* required

Array item

string

idstring* required

Alternative 2

null

typestring* required

Constant: "reset_applied"

{
  "type": "reset_applied"
}

Server session_info message

type: "session_info"

payload: object

infoobject* required

sessionstring* required

Values: "video", "game"

typestring* required

Constant: "session_info"

{
  "info": {},
  "session": "video",
  "type": "session_info"
}

Server stats message

type: "stats"

payload: object

prompt_msnumber | null

Per-chunk prompt update time with dynamic_prompt, unchanged chunks included (moving average).

Default: null

chunkinteger* required

pipelinedboolean | null

The next chunk renders while this one generates (throughput mode).

Default: null

render_msnumber | null

Game render time per chunk (moving average).

Default: null

realtime_factornumber* required

Generated frames per second / target fps.

generation_fpsnumber* required

typestring* required

Constant: "stats"

chunk_msnumber* required

latency_msnumber | null

Input or control capture to frame handoff, median.

Default: null

input_latency_msnumber | null

Game control message arrival to the handoff of the first frame it changes, median.

Default: null

loop_lag_msnumber | null

Server event-loop lag, p95 over the stats window.

Default: null

generate_msnumber | null

Model time per chunk (moving average).

Default: null

queue_framesinteger* required

billable_unitsnumber | null

Seconds of video billed so far this session (queued for playback); the final total settles when the session closes.

Default: null

{
  "chunk": 0,
  "realtime_factor": 0,
  "generation_fps": 0,
  "type": "stats",
  "chunk_ms": 0,
  "queue_frames": 0
}

Server stream_exhausted message

type: "stream_exhausted"

payload: object

The model has no more frames to send and the current peer is closing. Clients must negotiate a new session to continue.

typestring* required

Constant: "stream_exhausted"

{
  "type": "stream_exhausted"
}

Related Models

1. Calling the API

2. Authentication

API Key

3. Schema

About

Start Session

1. Calling the API#

Install the client#

The client provides a convenient way to interact with the model API.

npm install @fal-ai/client@alpha @fal-ai/server-proxy@alpha

Experimental realtime API

fal.realtime.open is currently available on the alpha release and may change in a minor version.

Set up the server proxy#

Keep FAL_KEY on your server and expose an authenticated proxy route to your browser. For a Next.js App Router application, create app/api/fal/proxy/route.ts:

import { route } from "@fal-ai/server-proxy/nextjs";

export const { GET, POST, PUT } = route;

Set FAL_KEY in the server environment, and protect this route with your application's authentication before deploying it.

Real-time via WebRTC#

This deployment publishes a WMA WebRTC contract. The example uses fal.realtime.open; its media handlers, capture constraints, and control message are generated from the linked AsyncAPI document.

import { createFalClient } from "@fal-ai/client";
import { wma } from "@fal-ai/client/realtime";

const fal = createFalClient({ proxyUrl: "/api/fal/proxy" });

const deviceStream = await navigator.mediaDevices.getUserMedia({
  "video": {
    "frameRate": 16,
    "height": 480,
    "width": 832
  },
  "audio": false
});
const localStream = deviceStream;

const session = fal.realtime.open(wma("vast/eden-live-14b-video"), {
  receive: [
    "video"
  ],
  localStream,
  onMedia: (stream) => {
      const element = document.querySelector("video");
      if (element) element.srcObject = stream;
    },
  onData: (raw) => {
      const message = JSON.parse(raw);
      console.log(message);
    },
  onState: (state) => console.log("realtime:", state),
  onError: (error) => console.error(error),
});


// Send any client message described below.
session.send({
  "prompt": "Change to anime style",
  "enable_prompt_expansion": true,
  "type": "configure"
});

// Later, release the peer connection and network resources:
// localStream.getTracks().forEach((track) => track.stop());
// await session.close();

2. Authentication#

The browser connects through your server proxy, which reads FAL_KEY from the server environment. Never put that key in browser code.

API Key#

Authenticate users before allowing them to call your proxy route so other people cannot spend against your account.

Protect your API Key

3. Schema#

Media#

Client contract for the WebRTC session created by the linked OpenAPI operation.

Tracks are described from the browser's perspective. Send tracks are captured by the browser and sent to the model; receive tracks come back from the model.

Send

video

required

{
  "frameRate": 16,
  "height": 480,
  "width": 832
}

Receive

video

optional

{
  "frameRate": 16,
  "height": 480,
  "width": 832
}

Client messages#

Set conditioning

type: "configure"

payload: object

Additional properties: not allowed

Set the session's conditioning, once per connection; await configured.

Without image_url the world anchors on a reference generated from the newest input frame and the prompt (FLUX.2 [klein] 4B edit), or on that frame itself when generation fails.

promptstring | null

What the input video should look like. Defaults to a generic photoreal prompt.

Default: null

At most 2000 characters when not null

image_urlstring | null

Rendering of the input's first frame in the target look (URL, or a data URI under ~45 KB); anchors the generated world. Without it, one is generated from the newest input frame and the prompt.

Default: null

At most 60000 characters when not null

enable_prompt_expansionboolean

Rewrite the scene description (and later prompt changes) into a detailed one with an LLM, using the reference image when given. Adds about a second to session start and to each prompt change.

Default: false

seedinteger | null

Noise seed; the same inputs and seed reproduce the output.

Default: null

minimum: 0 when not null · maximum: 2147483647 when not null

typestring* required

Constant: "configure"

Example

{
  "prompt": "Change to anime style",
  "enable_prompt_expansion": true,
  "type": "configure"
}

Client ping message

type: "ping"

payload: object

Latency probe; the server echoes ts back in a pong payload.

tsnumber | null

Default: null

typestring* required

Constant: "ping"

{
  "type": "ping"
}

Update prompt

type: "prompt"

payload: object

Additional properties: not allowed

Change the scene description; applied at the next chunk boundary.

promptstring* required

Text the model renders the scene with. Keeps the current world.

At least 1 character · At most 2000 characters

typestring* required

Constant: "prompt"

{
  "prompt": "string",
  "type": "prompt"
}

Restart

type: "reset"

payload: object

Additional properties: not allowed

Restart generation from a new anchor; unset fields keep their values, and an explicit null clears image_url or seed.

The world is re-anchored on a reference generated from the newest input frame unless image_url is set.

promptstring | null

What the input video should look like. Defaults to a generic photoreal prompt.

Default: null

At most 2000 characters when not null

image_urlstring | null

Rendering of the input's first frame in the target look (URL, or a data URI under ~45 KB); anchors the generated world. Without it, one is generated from the newest input frame and the prompt.

Default: null

At most 60000 characters when not null

seedinteger | null

Noise seed; the same inputs and seed reproduce the output.

Default: null

minimum: 0 when not null · maximum: 2147483647 when not null

enable_prompt_expansionboolean | null

Default: null

typestring* required

Constant: "reset"

{
  "type": "reset"
}

End session

type: "stop"

payload: object

Additional properties: not allowed

End the session.

typestring* required

Constant: "stop"

{
  "type": "stop"
}

Server messages#

Server configured message

type: "configured"

payload: object

gameobject | null

Default: null

Must match at least one of the following:

GameInfo

object

descriptionstring* required

thumbnail_urlstring | null

Default: null

promptstring* required

instructionsstring[]* required

Array item

string

titlestring* required

controlsobject[]* required

Array item

object

keystring* required

commandstring | null

Semantic command name.

Default: null

labelstring* required

holdboolean

Default: true

actionsstring[]* required

Array item

string

idstring* required

Alternative 2

null

typestring* required

Constant: "configured"

{
  "type": "configured"
}

Server error message

type: "error"

payload: object

retryableboolean | null

Default: null

typestring* required

Constant: "error"

errorstring* required

{
  "type": "error",
  "error": "string"
}

Server pong message

type: "pong"

payload: object

Reply to a ping, echoing the client's timestamp for RTT measurement.

client_tsnumber | null

Default: null

server_tsnumber | null

Default: null

tsnumber | null

Default: null

typestring* required

Constant: "pong"

{
  "type": "pong"
}

Server prompt_applied message

type: "prompt_applied"

payload: object

typestring* required

Constant: "prompt_applied"

{
  "type": "prompt_applied"
}

Server reset_applied message

type: "reset_applied"

payload: object

gameobject | null

Default: null

Must match at least one of the following:

GameInfo

object

descriptionstring* required

thumbnail_urlstring | null

Default: null

promptstring* required

instructionsstring[]* required

Array item

string

titlestring* required

controlsobject[]* required

Array item

object

keystring* required

commandstring | null

Semantic command name.

Default: null

labelstring* required

holdboolean

Default: true

actionsstring[]* required

Array item

string

idstring* required

Alternative 2

null

typestring* required

Constant: "reset_applied"

{
  "type": "reset_applied"
}

Server session_info message

type: "session_info"

payload: object

infoobject* required

sessionstring* required

Values: "video", "game"

typestring* required

Constant: "session_info"

{
  "info": {},
  "session": "video",
  "type": "session_info"
}

Server stats message

type: "stats"

payload: object

prompt_msnumber | null

Per-chunk prompt update time with dynamic_prompt, unchanged chunks included (moving average).

Default: null

chunkinteger* required

pipelinedboolean | null

The next chunk renders while this one generates (throughput mode).

Default: null

render_msnumber | null

Game render time per chunk (moving average).

Default: null

realtime_factornumber* required

Generated frames per second / target fps.

generation_fpsnumber* required

typestring* required

Constant: "stats"

chunk_msnumber* required

latency_msnumber | null

Input or control capture to frame handoff, median.

Default: null

input_latency_msnumber | null

Game control message arrival to the handoff of the first frame it changes, median.

Default: null

loop_lag_msnumber | null

Server event-loop lag, p95 over the stats window.

Default: null

generate_msnumber | null

Model time per chunk (moving average).

Default: null

queue_framesinteger* required

billable_unitsnumber | null

Seconds of video billed so far this session (queued for playback); the final total settles when the session closes.

Default: null

{
  "chunk": 0,
  "realtime_factor": 0,
  "generation_fps": 0,
  "type": "stats",
  "chunk_ms": 0,
  "queue_frames": 0
}

Server stream_exhausted message

type: "stream_exhausted"

payload: object

The model has no more frames to send and the current peer is closing. Clients must negotiate a new session to continue.

typestring* required

Constant: "stream_exhausted"

{
  "type": "stream_exhausted"
}

vast/eden-live-14b-video

Table of contents

Table of contents

vast/eden-live-14b-video

Table of contents

1. Calling the API

2. Authentication

3. Schema

About

1. Calling the API#

Install the client#

Experimental realtime API

Set up the server proxy#

Real-time via WebRTC#

2. Authentication#

API Key#

Protect your API Key

3. Schema#

Media#

Send

Receive

Client messages#

Set conditioning

Client ping message

Update prompt

Restart

End session

Server messages#

Server configured message

Server error message

Server pong message

Server prompt_applied message

Server reset_applied message

Server session_info message

Server stats message

Server stream_exhausted message

Related Models

Table of contents

Table of contents

1. Calling the API

2. Authentication

3. Schema

About

1. Calling the API#

Install the client#

Experimental realtime API

Set up the server proxy#

Real-time via WebRTC#

2. Authentication#

API Key#

Protect your API Key

3. Schema#

Media#

Send

Receive

Client messages#

Set conditioning

Client ping message

Update prompt

Restart

End session

Server messages#

Server configured message

Server error message

Server pong message

Server prompt_applied message

Server reset_applied message

Server session_info message

Server stats message

Server stream_exhausted message

Related Models