# Controlfoley

> Foley Control is a video-to-audio model that automatically generates synchronized sound effects for videos, using text prompts to shape the type of sound while matching the timing and action on screen.


## Overview

- **Endpoint**: `https://fal.run/fal-ai/controlfoley`
- **Model ID**: `fal-ai/controlfoley`
- **Category**: video-to-video
- **Kind**: inference
**Tags**: stylized, transform, lipsync



## Pricing

- **Price**: $0.015 per videos

For more details, see [fal.ai pricing](https://fal.ai/pricing).

## API Information

This model can be used via our HTTP API or more conveniently via our client libraries.
See the input and output schema below, as well as the usage examples.


### Input Schema

The API accepts the following input parameters:


- **`video_url`** (`string`, _required_):
  URL of the video to generate synchronized audio for.
  - Examples: "https://storage.googleapis.com/falserverless/model_tests/video_models/mmaudio_input.mp4"

- **`prompt`** (`string`, _optional_):
  Optional text prompt describing the desired audio. When combined with the video it provides text-controlled video-to-audio (TC-V2A) generation; leave empty for pure video-to-audio (V2A). Default value: `""`
  - Default: `""`
  - Examples: "the skateboard wheels scraping and grinding on the ground", "gentle footsteps on a wooden floor"

- **`negative_prompt`** (`string`, _optional_):
  Negative text prompt — describe audio characteristics to avoid. Default value: `""`
  - Default: `""`

- **`reference_audio_url`** (`string`, _optional_):
  Optional URL to a 2-4 second reference audio clip whose timbre should be transferred to the generated audio (AC-V2A mode). Audio shorter than 2s is zero-padded; longer is truncated.

- **`duration`** (`float`, _optional_):
  Target audio duration in seconds. Truncated to source video length when shorter. Default value: `8`
  - Default: `8`
  - Range: `1` to `30`

- **`num_inference_steps`** (`integer`, _optional_):
  Number of flow-matching ODE steps. Default value: `25`
  - Default: `25`
  - Range: `4` to `100`

- **`guidance_scale`** (`float`, _optional_):
  Classifier-free guidance strength. Default value: `4.5`
  - Default: `4.5`
  - Range: `0` to `20`

- **`mask_away_clip`** (`boolean`, _optional_):
  When True, disables the CLIP visual stream (text-driven mode).
  - Default: `false`

- **`seed`** (`integer`, _optional_):
  Random seed for reproducibility. If null, a random seed is used.



**Required Parameters Example**:

```json
{
  "video_url": "https://storage.googleapis.com/falserverless/model_tests/video_models/mmaudio_input.mp4"
}
```

**Full Example**:

```json
{
  "video_url": "https://storage.googleapis.com/falserverless/model_tests/video_models/mmaudio_input.mp4",
  "prompt": "the skateboard wheels scraping and grinding on the ground",
  "duration": 8,
  "num_inference_steps": 25,
  "guidance_scale": 4.5
}
```


### Output Schema

The API returns the following output format:

- **`video`** (`File`, _required_):
  Generated video with synchronized audio (H.264 + AAC).
  - Examples: {"url":"https://storage.googleapis.com/falserverless/model_tests/video_models/mmaudio_output.mp4"}

- **`audio`** (`File`, _required_):
  Generated audio waveform only (44.1 kHz mono WAV).

- **`seed`** (`integer`, _required_):
  The seed used for generation.



**Example Response**:

```json
{
  "video": {
    "url": "https://storage.googleapis.com/falserverless/model_tests/video_models/mmaudio_output.mp4"
  },
  "audio": {
    "url": "",
    "content_type": "image/png",
    "file_name": "z9RV14K95DvU.png",
    "file_size": 4404019
  }
}
```


## Usage Examples

### cURL

```bash
curl --request POST \
  --url https://fal.run/fal-ai/controlfoley \
  --header "Authorization: Key $FAL_KEY" \
  --header "Content-Type: application/json" \
  --data '{
     "video_url": "https://storage.googleapis.com/falserverless/model_tests/video_models/mmaudio_input.mp4"
   }'
```

### Python

Ensure you have the Python client installed:

```bash
pip install fal-client
```

Then use the API client to make requests:

```python
import fal_client

def on_queue_update(update):
    if isinstance(update, fal_client.InProgress):
        for log in update.logs:
           print(log["message"])

result = fal_client.subscribe(
    "fal-ai/controlfoley",
    arguments={
        "video_url": "https://storage.googleapis.com/falserverless/model_tests/video_models/mmaudio_input.mp4"
    },
    with_logs=True,
    on_queue_update=on_queue_update,
)
print(result)
```

### JavaScript

Ensure you have the JavaScript client installed:

```bash
npm install --save @fal-ai/client
```

Then use the API client to make requests:

```javascript
import { fal } from "@fal-ai/client";

const result = await fal.subscribe("fal-ai/controlfoley", {
  input: {
    video_url: "https://storage.googleapis.com/falserverless/model_tests/video_models/mmaudio_input.mp4"
  },
  logs: true,
  onQueueUpdate: (update) => {
    if (update.status === "IN_PROGRESS") {
      update.logs.map((log) => log.message).forEach(console.log);
    }
  },
});
console.log(result.data);
console.log(result.requestId);
```


## Additional Resources

### Documentation

- [Model Playground](https://fal.ai/models/fal-ai/controlfoley)
- [API Documentation](https://fal.ai/models/fal-ai/controlfoley/api)
- [OpenAPI Schema](https://fal.ai/api/openapi/queue/openapi.json?endpoint_id=fal-ai/controlfoley)

### fal.ai Platform

- [Platform Documentation](https://docs.fal.ai)
- [Python Client](https://docs.fal.ai/clients/python)
- [JavaScript Client](https://docs.fal.ai/clients/javascript)