API Reference

The Casola REST API lives under https://api.casola.ai (versioned paths under /api/v1/). It is the control plane: minting sessions, managing avatars and keys, and running one-shot generations. Live audio/video never flows through it — clients stream directly against the GPU edge (see The data plane).

Concepts

Read this once — every endpoint below is a small move inside this model.

Workspace. Your tenant container. API keys, avatars, voices, and sessions all belong to a workspace. Workspace-bound keys never need to pass a workspace_id; it is derived from the key.

API keys. Two kinds, created on the dashboard or via /api/v1/tokens:

Kind	Format	Where it may live	What it can do
Secret	`avatar_<8hex>.<56hex>`	Your server only	Anything its scopes allow
Publishable	`pk_<8hex>.<56hex>`	Shipped to browsers	Mint stock-persona sessions only, and only from its registered web origins

Scopes are OAuth-style strings on the key (* = all): sessions:write, sessions:read, session:connect (publishable), generations:write, avatars:read, avatars:write, …. Send the key on every request:

Authorization: Bearer avatar_abc12345.def678…

Stock personas vs. custom avatars. Two ways to put a face on screen:

Stock personas — the built-in, pre-warmed catalog (mei, david, …), shared by all tenants and listed by the public /api/gallery. Addressed by name ("persona": "mei").
Custom avatars — your own: a face image + a voice sample + a personality prompt, uploaded into your workspace. An avatar is the container; each edit creates an immutable version. The version id — called avatar_ref everywhere — is what you pass to sessions and prebuild. Nothing about a live session changes if you edit the avatar afterwards: the session snapshots its version at mint time.

Session. One live, full-duplex conversation seat on a GPU box. POST /api/v1/sessions allocates a seat, picks a box, and returns:

connect_url — that box’s https base URL, and
session_token — a ~60-second EdDSA JWT, verified offline by the box (no callback to the platform), carrying everything the box needs — including, for a custom avatar, where to fetch its face/voice so the box can materialize it on first contact.

Connect within the token’s TTL; the conversation itself then runs up to the session cap (default ≈300 s). At fleet capacity you get a queue ticket instead — hold its WebSocket for live position updates.

Generations. One-shot, blocking authoring calls (no seat, no session): /api/v1/speak (text + face + voice → lip-synced MP4) and /api/v1/design (natural-language voice description → new voice audio). Use them to build assets — intro clips, previews, designed voices.

Warm-up (prebuild). A brand-new custom avatar’s first-ever call pays a one-time cost while the fleet builds its idle-motion cache. Fire-and-forget POST /api/v1/prebuild right after creating one (while your user is still on the “creating…” screen) and the first real call starts like a stock persona.

Gallery — stock persona catalog

Public, no auth. Use it to build your persona picker.

GET /api/gallery                     → { "avatars": [ … ] }
GET /api/gallery/{id}/face           → image (the poster/preview face)
GET /api/gallery/{id}/intro?lang=en  → video/mp4 (short looping living-portrait, Range-capable)

Each catalog entry:

{ "id": "mei", "name": "Mei", "tagline": "…", "blurb": "…", "accent": "#e88",
  "voiceLang": "zh", "poster": "/api/gallery/mei/face", "hasIntro": true }

id is the persona key used in session mints. Intro clips are muted looping portraits (lang=en|zh, falls back to en) — play them in the picker while the user decides.

Sessions — start a live conversation

Full page: Sessions. The core call:

POST /api/v1/sessions          scope: sessions:write (secret) or session:connect (publishable)

Body — pick one of the avatar selectors:

Field	Type	Notes
`persona`	string	Stock persona key from the gallery (`"mei"`).
`avatar_ref`	string	Custom avatar version id. Requires a workspace; secret keys only.
`voice_ref`	string	Optional voice version override. Secret keys only.
`workspace_id`	string	Only when the key is not workspace-bound.
`domain_allowlist`	string[]	Origins allowed to connect with the minted token. Publishable keys: forced to the key’s registered origins.

201 (seat granted):

{
  "status": "ready",
  "session_id": "019…",
  "connect_url": "https://box-1.casola.ai",
  "session_token": "eyJhbGciOiJFZERTQ…",
  "seat_token": "…",
  "expires_at": 1719000060
}

202 (fleet at capacity — queued):

{ "status": "queued", "ticket": "…", "ahead": 3,
  "queue_ws": "/api/v1/sessions/queue?ticket=…" }

Open queue_ws (WebSocket) for position updates and the promotion signal, then POST /api/v1/sessions again to claim.

Session lifecycle (all sessions:write, secret keys):

GET  /api/v1/sessions/{id}             → session row (scope: sessions:read)
POST /api/v1/sessions/{id}/heartbeat   → keep an idle-held seat alive
POST /api/v1/sessions/{id}/release     → free the seat early on clean exit

Live media traffic is itself the heartbeat; you only need the explicit one for a seat you’re holding without being connected yet.

The data plane — talking to the edge

After the mint, everything is client ↔ box, directly. Two WebSockets against connect_url (swap https → wss):

Auth on the upgrade: browsers append ?token=<session_token> (they cannot set headers on WS upgrades); servers and native apps should send Authorization: Bearer <session_token>.

`/mse` — downlink (avatar video + voice)

First message (text): {"mime": "video/mp4; codecs=…"} — feed this to your decoder/MediaSource; never hardcode the codec string.
Then: binary fMP4 segments, appended in arrival order. The avatar is always animated — idle motion between replies, speech when it answers.

`/mic_stream` — uplink (your user’s voice) + transcripts

First message (text): {"op": "hello", "lang": "en"}.
Then: binary frames of 16 kHz mono PCM16, 1600 samples (100 ms) each. Send silence frames to keep cadence; a muted client sends zeroed frames.
Text messages come back on the same socket:

{ "type": "partial", "text": "so what do yo…" }
{ "type": "turn", "text": "so what do you think?", "reply": "Honestly? I love it.", "language": "en", "speech_id": "…" }
{ "type": "error", "error": "…" }

partial/turn are transcripts for your UI — the avatar’s actual reply arrives as spoken video on /mse. Barge-in is server-side: just keep streaming mic audio and speak over the avatar.

Echo cancellation is mandatory. The avatar’s voice plays out of the device speakers; without acoustic echo cancellation the mic re-captures it and the avatar starts replying to itself. In browsers: getUserMedia({ audio: { echoCancellation: true } }) (the SDK does this). On native: use the platform’s voice-processing audio unit / VOICE_COMMUNICATION source.

`/chat` — typed text input (optional)

POST {connect_url}/chat?token=<session_token> with {"text": "hello"} injects a typed user message into the same conversation (the avatar answers in video, as if spoken). The box does not set CORS headers — call it from your server or a same-origin relay, not cross-origin from the browser.

The browser side of all this is packaged as @casola/avatar-client.

Custom avatars — the customization pipeline

Full CRUD page: Avatars. This is the end-to-end recipe (all calls: secret key, avatars:write, workspace-bound):

1. Create the avatar (container):

POST /api/v1/avatars                     { "name": "Ryan" }        → { "id": "<avatarId>", … }

2. Create a version (the immutable snapshot — its id is your avatar_ref):

POST /api/v1/avatars/{avatarId}/versions
{
  "system_prompt": "You are Ryan, a laid-back surf instructor…",
  "backstory": "…",
  "native_language": "en",
  "voice_ref_text": "transcript of the voice sample, if you have it"
}
→ { "id": "<versionId>", "upload_url": "…", … }

3. Upload the face (proxy-PUT the raw bytes; image/jpeg, image/png, or image/webp, ≤ 10 MB):

PUT /api/v1/avatars/{avatarId}/versions/{versionId}/upload
Content-Type: image/jpeg
<binary image bytes>

4. Upload the voice sample (5–30 s of clean speech; audio/mpeg or audio/wav, ≤ 25 MB; optional X-Ref-Text header carries the transcript and skips server-side transcription):

PUT /api/v1/avatars/{avatarId}/versions/{versionId}/voice
Content-Type: audio/mpeg
X-Ref-Text: Hey, I'm Ryan — let's catch some waves.
<binary audio bytes>

No recording of the person? Design a voice from text instead — call /api/v1/design with a description (“warm, unhurried surf-bro baritone”) + a sample line, then upload the returned audio as the voice sample.

5. Warm it (recommended, fire-and-forget):

POST /api/v1/prebuild        scope: generations:write
{ "avatar_ref": "<versionId>" }
→ 202 { "ok": true, "avatar_ref": "…", "boxes": [ { "box_id": "…", "status": "building" } ] }

6. Use it — mint sessions with {"avatar_ref": "<versionId>"}. The GPU box pulls the face/voice on first contact (the session token tells it where), so a freshly created avatar is callable immediately, worldwide, on any box.

Optional extras: additional faces (PUT …/versions/{versionId}/faces), an intro clip for your picker (render one with /api/v1/speak using the same face + voice and store it wherever your app serves media).

Prebuild — pre-warm a custom avatar

POST /api/v1/prebuild { "avatar_ref": … } sweeps every live box and starts the idle-motion cache build in the background (202 per-box statuses: building, cached, prebuild_unsupported, unreachable). Skipping it is safe — the first call just starts noticeably slower while the cache builds on the connect path.

Generations

POST /api/v1/speak — text + face image + voice sample → lip-synced MP4 (synchronous; video/mp4 body; timing in X-Clone-Ms / X-Gen-Ms / X-Total-Ms). 429 at_capacity (honor Retry-After), 502 render_failed, 503 no_box_available.
POST /api/v1/design — { "description": "an excited e-sports caster", "text": "…" } → { "audio_base64", "sample_rate", "duration_seconds", "gen_ms", "rtf" }. Decode knobs (temperature, top_p, top_k, repetition_penalty, max_new_tokens) pass through.

Both require generations:write.

Building without your own backend (mobile & client-only apps)

The one call that must stay off the client is the session mint with a secret key. Everything else is client ↔ edge. Your options, in order of preference:

1. A one-endpoint backend (recommended — including for mobile). A single serverless function holding your secret key:

client → POST /start-session (your function) → POST https://api.casola.ai/api/v1/sessions
       ← { connect_url, session_token }
client ↔ edge WebSockets (direct — your function is out of the loop)

That function is your only server-side code: ~20 lines, one invocation per session, no media bandwidth. This is also where you’d gate sessions behind your own login/paywall, and the only home for avatars:write / generations:write flows (avatar customization is a server-side pipeline — never ship a key that can spend GPU time or write your workspace).

2. Publishable keys (browser web apps only). A pk_ key may ship in browser JavaScript: it can mint stock-persona sessions directly from the page, enforced by the Origin header against the key’s registered origins. This does not carry to native mobile apps — they send no browser Origin, and a key embedded in an app binary is extractable. Native apps: use option 1.

Native mobile clients — what to expect on the data plane:

You can (and should) authenticate the WebSockets with the Authorization: Bearer header — the ?token= query form exists only because browsers can’t set upgrade headers.
Playback: /mse delivers fMP4 with the codec announced in the first message. The official SDK is browser-only today, so either embed a WebView running @casola/avatar-client (fastest path, works now), or feed the segments natively (iOS AVSampleBufferDisplayLayer, Android ExoPlayer with a custom DataSource).
Microphone: resample to 16 kHz mono PCM16, send 1600-sample (100 ms) binary frames after the hello text frame — and enable the OS voice-processing / echo-cancellation mode (see data plane).
Timing: session_token lives ~60 s — mint right before connecting, not at app launch.

Endpoint index

Resource	Path	Auth
Gallery (stock catalog)	`GET /api/gallery`, `GET /api/gallery/{id}/face`, `GET /api/gallery/{id}/intro`	public
Sessions	`POST /api/v1/sessions`, `GET /api/v1/sessions/{id}`, `…/heartbeat`, `…/release`, `GET /api/v1/sessions/queue` (WS)	`sessions:*` / `session:connect`
Speak	`POST /api/v1/speak`	`generations:write`
Voice design	`POST /api/v1/design`	`generations:write`
Prebuild	`POST /api/v1/prebuild`	`generations:write`
Avatars	`GET/POST /api/v1/avatars`, `…/{id}/versions`, `PUT …/versions/{vid}/upload\|voice\|faces`	`avatars:*`
Voices	`GET/POST /api/v1/workspaces/{wsId}/voices`	`avatars:*`
Tokens	`GET/POST /api/v1/tokens`	authenticated
Workspaces	`GET/POST /api/v1/workspaces`	authenticated

All avatar/voice routes accept two shapes: token-relative (/api/v1/avatars/…, workspace derived from the key — the normal B2B form) and explicit (/api/v1/workspaces/{wsId}/avatars/…, for multi-workspace dashboards).

OpenAPI spec

The machine-readable spec is at /openapi.yaml — import it into Postman, Insomnia, or any OpenAPI client.

Interactive console: Try it →