kotoba-technologies/kotoba-streaming-stt/v1/realtime
Loading API documentation
Endpoint: wss://fal.run/Kotoba-Technologies/kotoba-streaming-stt/v1/realtime
Playground: upload a clip at https://fal.ai/models/kotoba-technologies/kotoba-streaming-stt/v1/speech-to-text to get the transcript.
Overview: Open a WebSocket, stream audio in real time, and receive transcript updates continuously while the speaker talks. When the client commits the stream, the server flushes the remaining audio and acknowledges the end of the session. The protocol follows the OpenAI Realtime transcription event format, so existing realtime clients port with minimal change.
Use cases
Technical specifications
target_language (the transcript language, required), audio format and
sample rate, turn_detection (see below).turn_detection defaults to {"type": "server_vad", "silence_duration_ms": 800}.
The server ends a turn after 800 ms in which the model produces no text (rounded up to whole
80 ms chunks); it is judged from the model's output, not from audio energy. Each turn gets its
own item_id (a UUID, created at the turn's first text) on its deltas and a
conversation.item.input_audio_transcription.completed event with the turn's full transcript.
Set "turn_detection": false to disable, or pass the object with another non-negative
silence_duration_ms; true and null are rejected. The setting is fixed when the session
opens. Stopping the audio does not end a turn: send input_audio_buffer.commit to flush, which
completes the last turn and is answered by input_audio_buffer.committed.target_language to ja.Example input (turn detection disabled: one continuous transcript, ended by commit)
{"type": "transcription_session.update",
"session": {"input_audio_format": "pcm16", "input_audio_sample_rate": 24000,
"input_audio_transcription": {"target_language": "ja"},
"turn_detection": false}}
{"type": "input_audio_buffer.append", "audio": "<base64 pcm16>"}
{"type": "input_audio_buffer.commit"}Example output with turn_detection: false
{"type": "transcription_session.updated", "session": {...}}
{"type": "conversation.item.created", "event_id": "…"}
{"type": "conversation.item.input_audio_transcription.delta", "item_id": null, "delta": "おはよう"}
{"type": "conversation.item.input_audio_transcription.delta", "item_id": null, "delta": "ございます。"}
{"type": "input_audio_buffer.committed"}Example output with the default turn detection (turn_detection omitted)
{"type": "transcription_session.updated", "session": {...}}
{"type": "conversation.item.input_audio_transcription.delta", "item_id": "2f0c…", "delta": "おはよう"}
{"type": "conversation.item.input_audio_transcription.delta", "item_id": "2f0c…", "delta": "ございます。"}
{"type": "conversation.item.input_audio_transcription.completed", "item_id": "2f0c…", "transcript": "おはようございます。"}
{"type": "conversation.item.input_audio_transcription.delta", "item_id": "9b41…", "delta": "今日は"}
{"type": "input_audio_buffer.commit"}
{"type": "conversation.item.input_audio_transcription.completed", "item_id": "9b41…", "transcript": "今日は水曜日です。"}
{"type": "input_audio_buffer.committed"}Deltas are append-only and never revised; completed repeats the turn's text in one piece.
Deltas sent before the first turn starts carry item_id: null. item_id is unrelated to the
event_id a client may put on its append messages.
Example (Python SDK)
import kotoba
client = kotoba.KotobaClient() # KOTOBA_API_KEY + KOTOBA_ASR_URL from the environment
for delta in client.asr.transcribe_stream(mic_chunks(), language="ja"): # PCM16 chunks in
print(delta, end="", flush=True)
# Turn detection: turn_detection=False disables it; {"type": "server_vad", "silence_duration_ms": 1200}
# changes the threshold. Session-level API: a "final_transcript" event closes each turn.Pricing: $0.39 per hour of session (unit: second of session, $0.000108 per second).
The kotoba-sdk package (PyPI, Python 3.10+, version 0.2.3 or newer) covers all four apps.
It sends the fal key with fal's Authorization: Key scheme automatically for fal.run URLs,
so use the fal key as the API key and point the per-route URLs at the apps.
pip install kotoba-sdk
export KOTOBA_API_KEY=<fal key>
export KOTOBA_ASR_REST_URL=https://fal.run/Kotoba-Technologies/kotoba-stt/v1
export KOTOBA_ASR_URL=wss://fal.run/Kotoba-Technologies/kotoba-streaming-stt/v1/realtime
export KOTOBA_TTS_URL=wss://fal.run/Kotoba-Technologies/kotoba-tts/v2/tts/ws
export KOTOBA_S2ST_URL=wss://fal.run/Kotoba-Technologies/kotoba-sts/v1/realtime_voiceAn app that has been idle may boot a GPU runner on the first request, which can take a few
minutes. Allow for that in the request timeout or retry the first request; the SDK's default
timeouts are 30 seconds for HTTP requests (KotobaClient(timeout=...)) and 15 seconds for the
WebSocket handshake.
Any WebSocket client works with this endpoint. The examples use ws for Node.js and websockets for Python.
npm install wsSet FAL_KEY as an environment variable in your runtime.
export FAL_KEY="YOUR_API_KEY"This endpoint speaks its own message protocol over a WebSocket. Open the connection with any WebSocket client and authenticate the upgrade with the Authorization: Key header. Browsers must not hold the API key: connect through your own server, which adds the header. The message protocol itself is documented in the model's About section.
import WebSocket from "ws";
// Keep the API key on your server; a browser connects through your
// own backend, which adds the header. Use the "ws" package: Node's
// global WebSocket ignores the headers option and the upgrade gets 401.
const ws = new WebSocket("wss://fal.run/kotoba-technologies/kotoba-streaming-stt/v1/realtime", {
headers: { Authorization: "Key " + process.env.FAL_KEY },
});
ws.on("open", () => {
// Send the app's first message here.
});
ws.on("message", (data, isBinary) => {
if (isBinary) return; // binary frame, e.g. audio
console.log(data.toString()); // the app's messages, shown as text
});
ws.on("error", console.error);The API uses an API Key for authentication. Read it from the FAL_KEY environment variable on your server and send it as the Authorization: Key <FAL_KEY> header on the WebSocket upgrade, as the real-time snippet above does.
When running code on the client-side (e.g. in a browser, mobile app or GUI applications), make sure to not expose your FAL_KEY. Instead, use a server-side proxy to make requests to the API. For more information, check out our server-side integration guide.
Some attributes in the API accept file URLs as input. Whenever that's the case you can pass your own URL or a Base64 data URI.
You can pass a Base64 data URI as a file input. The API will handle the file decoding for you. Keep in mind that for large files, this alternative although convenient can impact the request performance.
You can also pass your own URLs as long as they are publicly accessible. Be aware that some hosts might block cross-site requests, rate-limit, or consider the request as a bot.
We provide a convenient file storage that allows you to upload files and use them in your requests. You can upload files using the client API and use the returned URL in your requests.
import { fal } from "@fal-ai/client";
const file = new File(["Hello, World!"], "hello.txt", { type: "text/plain" });
const url = await fal.storage.upload(file);The client will auto-upload the file for you if you pass a binary object (e.g. File, Data).
Read more about file handling in our file upload guide.