Speak

Turn text + a face image + a voice into a lip-synced MP4 of that face speaking the text in that voice. One synchronous, blocking call chains the two stages for you:

1. Voice clone (TTS) — synthesize text in the cloned voice → audio.
2. Lip-sync (video) — drive the face image with that audio → an MP4.

Try it →

Generate a video

POST /api/v1/speak

Required scope: generations:write

Request body

Supply the media one of two ways — flat fields win if both are present.

(a) OpenAI-style content[] (recommended)

{
  "workspace_id": "019...",
  "text": "Hi! Thanks for trying out the avatar API.",
  "content": [
    { "type": "image_url", "image_url": { "url": "https://example.com/face.jpg" } },
    { "type": "audio_url", "audio_url": { "url": "https://example.com/voice.mp3" } }
  ]
}

(b) Flat fields (legacy, still supported)

{
  "workspace_id": "019...",
  "text": "The words to speak.",
  "image": "https://example.com/face.jpg",
  "voice_ref": "https://example.com/voice.mp3"
}

Field	Required	Description
text	Yes	The words the avatar should say.
workspace_id	Conditional	Workspace to attribute the generation to. Required if the token is not workspace-scoped.
content	Yes*	OpenAI content parts: image_url, audio_url, or input_audio (inline base64). *Required unless you pass the flat image + voice_ref.
image	Yes*	Face image as an http(s) URL or a data:image/...;base64,… URI.
voice_ref	Yes*	Voice reference to clone (3–10 s ideal), as a URL or data URI.
voice_text	No	Transcript of voice_ref. Omit it and the server auto-transcribes (Whisper) + caches. Supplying it skips transcription.
immutable_url_for_caching	No	If true, treats your media URLs as immutable cache keys → skips re-download/re-transcription/re-encode on repeats. Only set when the URL content never changes.
end_frame	No	Optional last-frame face (URL or data URI); the clip ends on this pose. Defaults to image.
prompt	No	Scene/style prompt for the video model.
num_step	No	TTS denoising steps. Default 16.
guidance_scale	No	TTS guidance. Default 2.0.
first_frame_strength	No	How strongly the first frame is pinned to the source face. Default 0.5.
last_frame_strength	No	How strongly the last frame is pinned. Default 0.0 (frees the mouth for natural lip-sync).
width	No	Output width (px). Default 352.
height	No	Output height (px). Default 480 (portrait).
frame_rate	No	Output fps. Default 24.

Media formats. image and voice_ref accept an http(s) URL (the server downloads it) or a data:<mime>;base64,… URI. Audio: wav, mp3, m4a, flac, ogg. Images: jpg, png, webp, etc.

You don’t need a transcript of the voice reference. The server transcribes it with Whisper and caches the result. Pass voice_text only to skip that step.

Response 200

Content-Type: video/mp4 — the body is the raw MP4. Timing/metadata is in headers:

Header	Meaning
X-Clone-Ms	TTS (voice-clone) time, ms
X-Gen-Ms	Video-generation time, ms
X-Total-Ms	Total server time, ms
X-Duration	Video duration, seconds
X-Box-Id	Which GPU box served it

Errors

Status	error	Meaning
400	invalid_json	Body was not valid JSON.
400	missing_fields	Missing text, image, or voice ref (the required field explains).
400	workspace_id_required	No workspace_id and the token is not workspace-scoped.
401	—	Missing/invalid token.
403	forbidden	Token lacks generations:write (or workspace membership).
429	at_capacity	All GPU cells busy. Honor Retry-After and retry.
502	render_failed	Upstream TTS/video failure; retry, report if persistent.
503	no_box_available	No healthy GPU box. Retry shortly.

Example

curl -sS -X POST https://api.casola.ai/api/v1/speak \
  -H "Authorization: Bearer avatar_abc12345.def...your-key" \
  -H "content-type: application/json" \
  -d '{
    "workspace_id": "019...",
    "text": "Hello from the avatar API.",
    "content": [
      { "type": "image_url", "image_url": { "url": "https://cdn.example.com/face.jpg" } },
      { "type": "audio_url", "audio_url": { "url": "https://cdn.example.com/voice.mp3" } }
    ],
    "immutable_url_for_caching": true
  }' --output reply.mp4 -D headers.txt

grep -i '^x-' headers.txt   # timing

Notes

• Warm latency is roughly 2–5 s per clip (≈ X-Clone-Ms + X-Gen-Ms).
• Repeat avatars are faster. Reusing the same face + voice hits the transcript, cloned-voice, and face-latent caches. For best repeat performance, host the face/voice at stable URLs and set immutable_url_for_caching: true.
• Resolution & frames. Default 352×480 portrait at 24 fps; duration is derived from the synthesized speech (you don’t set frame count directly).
• Capacity. Each GPU box has a fixed number of cells; at capacity you get a 429 with Retry-After. Back off and retry. For an interactive, multi-turn stream instead of one-shot clips, use a live session.