Bidirectional proxy for Grok's realtime speech-to-speech agent.
Relays xAI's realtime events verbatim in both directions between the
fal client and wss://api.x.ai/v1/realtime?model=grok-voice-latest.
Connect with a fal realtime client and exchange the event dicts
documented by xAI (the client library owns the wire framing). The
upstream connection is opened as soon as the session starts;
XAI_KEY is injected server-side and never exposed.
session_timeout=3600 matches xAI's 1-hour maximum;
buffering=1 because voice agents are latency-sensitive.
The client provides a convenient way to interact with the model API.
npm install --save @fal-ai/clientThe @fal-ai/serverless-client package has been deprecated in favor of @fal-ai/client. Install the new package and update your imports — see client setup.
Set FAL_KEY as an environment variable in your runtime.
export FAL_KEY="YOUR_API_KEY"This model has a real-time mode via WebSockets, supported by fal.realtime.connect.
import { fal } from "@fal-ai/client";
const connection = fal.realtime.connect("xai/grok-voice/realtime", {
onResult: (result) => {
console.log(result);
},
onError: (error) => {
console.error(error);
},
// Fetch short-lived JWT token from your backend
tokenProvider: async (app) => {
const response = await fetch("/api/fal/realtime-token", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ app }),
});
return response.text();
},
tokenExpirationSeconds: 10,
});
connection.send({
type: "session.update"
});The API uses an API Key for authentication. It is recommended you set the FAL_KEY environment variable in your runtime when possible.
import { fal } from "@fal-ai/client";
fal.config({
credentials: "YOUR_FAL_KEY"
});When running code on the client-side (e.g. in a browser, mobile app or GUI applications), make sure to not expose your FAL_KEY. Instead, use a server-side proxy to make requests to the API. For more information, check out our server-side integration guide.
Some attributes in the API accept file URLs as input. Whenever that's the case you can pass your own URL or a Base64 data URI.
You can pass a Base64 data URI as a file input. The API will handle the file decoding for you. Keep in mind that for large files, this alternative although convenient can impact the request performance.
You can also pass your own URLs as long as they are publicly accessible. Be aware that some hosts might block cross-site requests, rate-limit, or consider the request as a bot.
We provide a convenient file storage that allows you to upload files and use them in your requests. You can upload files using the client API and use the returned URL in your requests.
import { fal } from "@fal-ai/client";
const file = new File(["Hello, World!"], "hello.txt", { type: "text/plain" });
const url = await fal.storage.upload(file);The client will auto-upload the file for you if you pass a binary object (e.g. File, Data).
Read more about file handling in our file upload guide.
voice EnumBuilt-in xAI voice for the agent's speech. Uses Grok's default voice when omitted.
Possible enum values: carina, zagan, helix, orion, luna, iris, altair, zenith, perseus, helios, lux, kepler, rigel, cosmo, celeste, ursa, sirius, lumen, castor, naksh, atlas, aurora, liora, ara, eve, leo, rex, sal
Server-side tools available to the agent. Web, X, and remote MCP tools are supported; every tool is disabled by default. An explicitly supplied object always reaches the session, so previously enabled tools can be cleared.
prompt stringSystem instructions describing the agent's persona and conversation context. Uses Grok's default persona when omitted. Default value: "null"
type string* requiredxAI realtime event type (e.g. 'session.update', 'input_audio_buffer.append', 'response.create', 'response.output_audio.delta'). Events are relayed verbatim in both directions.
Automatic turn detection: any configured object (even empty) lets the agent detect when you stop speaking and reply on its own. Set null for manual turns; omit to keep the session default.
{
"type": "session.update"
}voice EnumBuilt-in xAI voice for the agent's speech. Uses Grok's default voice when omitted.
Possible enum values: carina, zagan, helix, orion, luna, iris, altair, zenith, perseus, helios, lux, kepler, rigel, cosmo, celeste, ursa, sirius, lumen, castor, naksh, atlas, aurora, liora, ara, eve, leo, rex, sal
Server-side tools available to the agent. Web, X, and remote MCP tools are supported; every tool is disabled by default. An explicitly supplied object always reaches the session, so previously enabled tools can be cleared.
prompt stringSystem instructions describing the agent's persona and conversation context. Uses Grok's default persona when omitted. Default value: "null"
type string* requiredxAI realtime event type (e.g. 'session.update', 'input_audio_buffer.append', 'response.create', 'response.output_audio.delta'). Events are relayed verbatim in both directions.
Automatic turn detection: any configured object (even empty) lets the agent detect when you stop speaking and reply on its own. Set null for manual turns; omit to keep the session default.
{
"type": "session.update"
}xAI-managed web search; off unless enabled is set true.
xAI-managed X search; off unless enabled is set true.
Remote MCP servers that xAI connects to and executes.
silence_duration_ms integerSilence duration that ends a detected turn, in milliseconds.
prefix_padding_ms integerAudio retained before detected speech, in milliseconds.
threshold floatVAD activation threshold; higher values require louder audio.
idle_timeout_ms integerIdle time after an assistant response before xAI starts a proactive turn, in milliseconds.
Restrict results to at most five domains, without protocol or path.
Exclude results from at most five domains, without protocol or path.
enabled booleanWhether to include web_search in session.tools. Disabled by default; set true to let the agent search the web.
Optional location used to localize search results.
enable_image_understanding booleanAllow the agent to inspect images found during web search.
from_date stringEarliest post date, as ISO YYYY-MM-DD. Default value: "null"
Restrict results to posts from at most 20 X handles.
Exclude posts from at most 20 X handles.
enabled booleanWhether to include x_search in session.tools. Disabled by default; set true to let the agent search X posts.
enable_video_understanding booleanAllow the agent to inspect videos found on X.
to_date stringLatest post date, as ISO YYYY-MM-DD. Default value: "null"
enable_image_understanding booleanAllow the agent to inspect images found on X.
server_description stringDescription of the capabilities exposed by this server. Default value: "null"
Optional allow-list of tool names exposed from this server.
authorization stringSensitive authorization value forwarded only to xAI. Default value: "null"
enabled booleanFalse omits this server. Default value: true
server_label string* requiredLabel xAI uses to identify and prefix this server's tools.
server_url string* requiredCaller-selected HTTP(S) MCP endpoint. xAI makes the remote connection and supports Streaming HTTP/SSE transports. Use at your own risk; fal does not connect to or SSRF-fetch this URL.
Sensitive MCP request headers forwarded only to xAI.
timezone stringIANA timezone name, for example America/New_York. Default value: "null"
city stringCity used to localize web-search results. Default value: "null"
country stringCountry used to localize web-search results. Default value: "null"
region stringRegion used to localize web-search results. Default value: "null"