API Reference

The Casola REST API lives under https://api.casola.ai (versioned paths under /api/v1/). It is the control plane: minting sessions, managing avatars and keys, and running one-shot generations. Live audio/video never flows through it — clients stream directly against the GPU edge (see The data plane).

Concepts

Read this once — every endpoint below is a small move inside this model.

Workspace. Your tenant container. API keys, avatars, voices, and sessions all belong to a workspace. Workspace-bound keys never need to pass a workspace_id; it is derived from the key. Keys minted on the dashboard are not workspace-bound today, so endpoints that require a workspace — such as /api/v1/speak — answer 400 workspace_id_required unless the request body carries an explicit workspace_id. Look yours up with GET /api/v1/workspaces.

API keys. Two kinds, created on the dashboard or via /api/v1/tokens:

Kind	Format	Where it may live	What it can do
Secret	`avatar_<8hex>.<56hex>`	Your server only	Anything its scopes allow
Publishable	`pk_<8hex>.<56hex>`	Shipped to browsers	Mint stock-persona sessions only, and only from its registered web origins

Scopes are OAuth-style strings on the key (* = all): sessions:write, sessions:read, session:connect (publishable), generations:write, avatars:read, avatars:write, …. Send the key on every request:

Authorization: Bearer avatar_abc12345.def678…

Stock personas vs. custom avatars. Two ways to put a face on screen:

Stock personas — the built-in, pre-warmed catalog (mei, david, …), shared by all tenants and listed by the public /api/gallery. Addressed by name ("persona": "mei").
Custom avatars — your own: a face image + a voice sample + a personality prompt, in your workspace. An avatar is the container; its identity (face, voice) is sealed into immutable versions, and its text (prompt, backstory, tools, options) lives in append-only profiles. Mint against the avatar with avatar_id and you get whatever it currently publishes; avatar_ref names a specific version id. Nothing about a live session changes if you edit the avatar afterwards: the session snapshots everything at mint time.

Defaults vs. overrides. What you edit in the dashboard becomes the avatar’s default (a new profile). What you pass on a session mint — context.system_prompt, tools, config, the extra_* layers — is a per-session override that is never stored. Resolution is per field: session parameter, else profile, else the sealed version. Full contract: Prompt & overrides.

Tools. An avatar can call out mid-conversation — to an MCP server, to your own hook, or to a built-in that runs on the GPU box. The call happens off the critical path: the avatar keeps talking, and the result rejoins the conversation a few seconds later. Declared per session as tools; see Tool calling.

Session. One live, full-duplex conversation seat on a GPU box. POST /api/v1/sessions allocates a seat, picks a box, and returns:

connect_url — that box’s https base URL, and
session_token — a ~60-second EdDSA JWT, verified offline by the box (no callback to the platform), carrying everything the box needs — including, for a custom avatar, where to fetch its face/voice so the box can materialize it on first contact.

Connect within the token’s TTL; the conversation itself then runs up to the session cap (default ≈300 s). When every seat is busy the mint returns 202 with a queue_ticket — re-POST the same body with that ticket until you are promoted to a 201. Only when the line is full past the wait horizon (or no box is registered at all) does it return 503 fleet_busy. See Sessions.

Generations. One-shot, blocking authoring calls (no seat, no session): /api/v1/speak (text + face + voice → lip-synced MP4) and /api/v1/design (natural-language voice description → new voice audio). Use them to build assets — intro clips, previews, designed voices.

Idle media. An avatar plays short looping clips while it listens. The platform builds them in the background after an avatar is published and reports progress as assets_status on the avatar (missing → building → ready, see Avatars). Sessions never wait for it: an avatar whose media is still missing is callable immediately and simply holds a still frame between replies. POST /api/v1/prebuild remains available to warm the currently-live boxes explicitly.

Gallery — stock persona catalog

Public, no auth. Use it to build your persona picker.

GET /api/gallery                     → { "avatars": [ … ] }
GET /api/gallery/{id}/face           → image (the poster/preview face)
GET /api/gallery/{id}/intro?lang=en  → video/mp4 (short looping living-portrait, Range-capable)

Each catalog entry:

{ "id": "mei", "name": "Mei", "tagline": "…", "blurb": "…", "accent": "#e88",
  "voiceLang": "zh", "poster": "/api/gallery/mei/face", "hasIntro": true }

id is the persona key used in session mints. Intro clips are muted looping portraits (lang=en|zh, falls back to en) — play them in the picker while the user decides.

Sessions — start a live conversation

Full page: Sessions. The core call:

POST /api/v1/sessions          scope: sessions:write (secret) or session:connect (publishable)

Body — pick one of the avatar selectors:

Field	Type	Notes
`persona`	string	Stock persona key from the gallery (`"mei"`).
`avatar_id`	string	Your avatar — resolves to its current published version. Secret keys only.
`avatar_ref`	string	Custom avatar version id. Requires a workspace; secret keys only.
`voice_ref`	string	Optional voice version override. Secret keys only.
`workspace_id`	string	Only when the key is not workspace-bound.
`domain_allowlist`	string[]	Origins allowed to connect with the minted token. Publishable keys: forced to the key’s registered origins.

Plus the per-session behaviour overrides (profile_id, context.*, tools, config, extra_*) — all secret-key only, all documented in Prompt & overrides.

201 (seat granted):

{
  "status": "ready",
  "session_id": "019…",
  "connect_url": "https://box-1.casola.ai",
  "session_token": "eyJhbGciOiJFZERTQ…",
  "seat_token": "…",
  "expires_at": 1719000060
}

202 (all seats busy — you are in line):

{ "status": "queued", "queue_ticket": "8a6b…", "position": 3, "eta_seconds": 45, "retry_after": 3 }

Re-POST the same body with queue_ticket set every retry_after seconds until you get a 201. Dropping the ticket sends you to the back of the line.

503 (line full past the wait horizon, or no box registered):

{ "error": "fleet_busy" }

Terminal for this attempt — surface it rather than polling.

Session lifecycle (all sessions:write, secret keys):

GET  /api/v1/sessions/{id}             → session row (scope: sessions:read)
POST /api/v1/sessions/{id}/heartbeat   → keep an idle-held seat alive
POST /api/v1/sessions/{id}/release     → free the seat early on clean exit

Live media traffic is itself the heartbeat; you only need the explicit one for a seat you’re holding without being connected yet.

The data plane — talking to the edge

After the mint, everything is client ↔ box, directly. Two WebSockets against connect_url (swap https → wss):

Auth on the upgrade: browsers append ?token=<session_token> (they cannot set headers on WS upgrades); servers and native apps should send Authorization: Bearer <session_token>.

`/mse` — downlink (avatar video + voice)

First message (text): {"mime": "video/mp4; codecs=…"} — feed this to your decoder/MediaSource; never hardcode the codec string.
Then: binary fMP4 segments, appended in arrival order. The avatar is always animated — idle motion between replies, speech when it answers.

`/mic_stream` — uplink (your user’s voice) + transcripts

First message (text): {"op": "hello", "lang": "en"}.
Then: binary frames of 16 kHz mono PCM16, 1600 samples (100 ms) each. Send silence frames to keep cadence; a muted client sends zeroed frames.
Text messages come back on the same socket:

{ "type": "partial", "text": "so what do yo…" }
{ "type": "turn", "text": "so what do you think?", "reply": "Honestly? I love it.", "language": "en", "speech_id": "…" }
{ "type": "error", "error": "…" }

partial/turn are transcripts for your UI — the avatar’s actual reply arrives as spoken video on /mse. Barge-in is server-side: just keep streaming mic audio and speak over the avatar.

Echo cancellation is mandatory. The avatar’s voice plays out of the device speakers; without acoustic echo cancellation the mic re-captures it and the avatar starts replying to itself. In browsers: getUserMedia({ audio: { echoCancellation: true } }) (the SDK does this). On native: use the platform’s voice-processing audio unit / VOICE_COMMUNICATION source.

`/chat` — typed text input (optional)

POST {connect_url}/chat?token=<session_token> with {"text": "hello"} injects a typed user message into the same conversation (the avatar answers in video, as if spoken). The box does not set CORS headers — call it from your server or a same-origin relay, not cross-origin from the browser.

The browser side of all this is packaged as @casola/avatar-client.

Custom avatars — the customization pipeline

Full CRUD page: Avatars. This is the end-to-end recipe (all calls: secret key, avatars:write, workspace-bound):

1. Create the avatar (container):

POST /api/v1/avatars                     { "name": "Ryan" }        → { "id": "<avatarId>", … }

2. Create a version (the immutable snapshot — its id is your avatar_ref):

POST /api/v1/avatars/{avatarId}/versions
{
  "system_prompt": "You are Ryan, a laid-back surf instructor…",
  "backstory": "…",
  "native_language": "en",
  "voice_ref_text": "transcript of the voice sample, if you have it",
  "source_persona": "mia"
}
→ { "id": "<versionId>", "upload_url": "…", … }

source_persona is the no-copy path for basing a custom identity on a public stock face and voice. It makes the version ready immediately; omit it when uploading original media in steps 3–4.

3. Upload the face (proxy-PUT the raw bytes; image/jpeg, image/png, or image/webp, ≤ 10 MB):

PUT /api/v1/avatars/{avatarId}/versions/{versionId}/upload
Content-Type: image/jpeg
<binary image bytes>

4. Upload the voice sample (5–30 s of clean speech; audio/mpeg or audio/wav, ≤ 25 MB; optional X-Ref-Text header carries the transcript and skips server-side transcription):

PUT /api/v1/avatars/{avatarId}/versions/{versionId}/voice
Content-Type: audio/mpeg
X-Ref-Text: Hey, I'm Ryan — let's catch some waves.
<binary audio bytes>

No recording of the person? Design a voice from text instead — call /api/v1/design with a description (“warm, unhurried surf-bro baritone”) + a sample line, then upload the returned audio as the voice sample.

5. Warm it (recommended, fire-and-forget):

POST /api/v1/prebuild        scope: generations:write
{ "avatar_ref": "<versionId>" }
→ 202 { "ok": true, "avatar_ref": "…", "boxes": [ { "box_id": "…", "status": "building" } ] }

6. Use it — mint sessions with {"avatar_ref": "<versionId>"}. The GPU box pulls the face/voice on first contact (the session token tells it where), so a freshly created avatar is callable immediately, worldwide, on any box.

Optional extras: additional faces (PUT …/versions/{versionId}/faces), an intro clip for your picker (render one with /api/v1/speak using the same face + voice and store it wherever your app serves media).

Prebuild — pre-warm a custom avatar

POST /api/v1/prebuild { "avatar_ref": … } sweeps every live box and starts the idle-motion cache build in the background (202 per-box statuses: building, cached, persona_unavailable, prebuild_unsupported, unreachable). Skipping it is safe — the first call just starts noticeably slower while the cache builds on the connect path.

Generations

POST /api/v1/speak — text + face image + voice sample → lip-synced MP4 (synchronous; video/mp4 body; timing in X-Clone-Ms / X-Gen-Ms / X-Total-Ms). 429 at_capacity (honor Retry-After), 502 render_failed, 503 no_box_available.
POST /api/v1/design — { "description": "an excited e-sports caster", "text": "…" } → { "audio_base64", "sample_rate", "duration_seconds", "gen_ms", "rtf" }. Decode knobs (temperature, top_p, top_k, repetition_penalty, max_new_tokens) pass through.

Both require generations:write.

Building without your own backend (mobile & client-only apps)

The one call that must stay off the client is the session mint with a secret key. Everything else is client ↔ edge. Your options, in order of preference:

1. A one-endpoint backend (recommended — including for mobile). A single serverless function holding your secret key:

client → POST /start-session (your function) → POST https://api.casola.ai/api/v1/sessions
       ← { connect_url, session_token }
client ↔ edge WebSockets (direct — your function is out of the loop)

That function is your only server-side code: ~20 lines, one invocation per session, no media bandwidth. This is also where you’d gate sessions behind your own login/paywall, and the only home for avatars:write / generations:write flows (avatar customization is a server-side pipeline — never ship a key that can spend GPU time or write your workspace).

2. Publishable keys (browser web apps only). A pk_ key may ship in browser JavaScript: it can mint stock-persona sessions directly from the page, enforced by the Origin header against the key’s registered origins. This does not carry to native mobile apps — they send no browser Origin, and a key embedded in an app binary is extractable. Native apps: use option 1.

Native mobile clients — what to expect on the data plane:

You can (and should) authenticate the WebSockets with the Authorization: Bearer header — the ?token= query form exists only because browsers can’t set upgrade headers.
Playback: /mse delivers fMP4 with the codec announced in the first message. The official SDK is browser-only today, so either embed a WebView running @casola/avatar-client (fastest path, works now), or feed the segments natively (iOS AVSampleBufferDisplayLayer, Android ExoPlayer with a custom DataSource).
Microphone: resample to 16 kHz mono PCM16, send 1600-sample (100 ms) binary frames after the hello text frame — and enable the OS voice-processing / echo-cancellation mode (see data plane).
Timing: session_token lives ~60 s — mint right before connecting, not at app launch.

Endpoint index

Resource	Path	Auth
Gallery (stock catalog)	`GET /api/gallery`, `GET /api/gallery/{id}/face`, `GET /api/gallery/{id}/intro`	public
Sessions	`POST /api/v1/sessions`, `GET /api/v1/sessions/{id}`, `…/heartbeat`, `…/release`	`sessions:*` / `session:connect`
Speak	`POST /api/v1/speak`	`generations:write`
Voice design	`POST /api/v1/design`	`generations:write`
Prebuild	`POST /api/v1/prebuild`	`generations:write`
Avatars	`GET/POST /api/v1/avatars`, `…/{id}/versions`, `PUT …/versions/{vid}/upload\|voice\|faces`	`avatars:*`
Profiles	`GET/POST /api/v1/avatars/{id}/profiles`, `…/{profileId}/default`	`avatars:*`
Tool calling	`GET/POST/PATCH/DELETE /api/v1/workspaces/{wsId}/mcp-servers`, `GET/PUT …/builtin-servers`	`workspaces:write`
Voices	`GET/POST /api/v1/workspaces/{wsId}/voices`	`avatars:*`
Tokens	`GET/POST /api/v1/tokens`	authenticated
Workspaces	`GET/POST /api/v1/workspaces`	authenticated

All avatar/voice routes accept two shapes: token-relative (/api/v1/avatars/…, workspace derived from the key — the normal B2B form) and explicit (/api/v1/workspaces/{wsId}/avatars/…, for multi-workspace dashboards).

OpenAPI spec

The machine-readable spec is at /openapi.yaml — import it into Postman, Insomnia, or any OpenAPI client.

Interactive console: Try it →