Quickstart

Quickstart

Go from zero to a live, full-duplex conversation with a Casola avatar: get an API key, mint a session from your server, and connect the browser directly to the GPU edge. Under five minutes.

How it fits together (two planes — this shape matters for everything below):

1. Control plane — your server calls https://api.casola.ai with your secret key to mint a session. The response names the GPU edge to talk to (connect_url) and a short-lived signed token for it (session_token).
2. Data plane — your client connects straight to that edge over two WebSockets (video down, microphone up). Your secret key never touches the client; media never proxies through your server.

Want a video panel and nothing else?

If a stock avatar in a standard 16:9 panel is all you need, skip the server entirely — <avatar-embed> is one script tag with a publishable key, and it handles the mint, the microphone, the queue and the seat for you. Come back here when you need a custom avatar, a system prompt, or your own interface.

1. Create an account and get an API key

Sign up at the dashboard, then open API keys. Click Create key, give it a name, and copy the key — it is shown only once and looks like:

avatar_<8-hex-chars>.<56-hex-chars>

This is a secret key: keep it server-side only (env var, secret manager — never a client bundle or app binary).

When API access or custom avatars are enabled for your workspace, the dashboard shows the Developer menu with API keys, trusted devices and avatar authoring.

export CASOLA_API_KEY="avatar_abc12345.xyz..."

2. Verify the key

curl https://api.casola.ai/api/v1/tokens \
  -H "Authorization: Bearer $CASOLA_API_KEY"

A 200 with your token list confirms the key works.

3. Pick a stock avatar

The stock catalog is public — no auth:

curl https://api.casola.ai/api/gallery

{
  "avatars": [
    { "id": "mei",   "name": "Mei",   "tagline": "…", "blurb": "…", "accent": "#e88",
      "voiceLang": "zh", "poster": "/api/gallery/mei/face", "hasIntro": true },
    { "id": "david", "name": "David", "tagline": "…", "blurb": "…", "accent": "#88e",
      "voiceLang": "en", "poster": "/api/gallery/david/face", "hasIntro": true }
  ]
}

id is the persona key you pass when minting. Each persona also has a public looping intro clip (/api/gallery/{id}/intro?lang=en|zh) you can show in your picker UI while the user decides.

4. Start a session (server-side)

curl -X POST https://api.casola.ai/api/v1/sessions \
  -H "Authorization: Bearer $CASOLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"persona":"mei","protocol_versions":[2]}'

Send protocol_versions. Omitting it offers v1, and every live renderer speaks v2, so a mint without it comes back 503 protocol_unavailable.

201 — a GPU seat is yours:

{
  "status": "ready",
  "session_id": "019...",
  "connect_url": "https://box-1.casola.ai",
  "session_token": "eyJhbGciOiJFZERTQ...",
  "seat_token": "...",
  "expires_at": 1719000060,
  "cap_seconds": 300
}

• connect_url — the assigned edge’s https base URL; derive WebSocket URLs by swapping to wss.
• session_token — a short-lived (≈60 s) EdDSA JWT the edge verifies offline. Connect promptly; if it expires, mint again.
• cap_seconds — how long this session may run once connected.

202 — the fleet is full, and you are in line:

{
  "status": "queued",
  "queue_ticket": "qt_...",
  "position": 3,
  "eta_seconds": 24,
  "retry_after": 5,
  "state": "waiting"
}

A full fleet puts you in line. When the wait is inside the horizon you get a queue_ticket instead of an error. Re-POST the same body plus the ticket every retry_after seconds until a 201 comes back:

curl -X POST https://api.casola.ai/api/v1/sessions \
  -H "Authorization: Bearer $CASOLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"persona":"mei","protocol_versions":[2],"queue_ticket":"qt_..."}'

Branch on state as well as the status code. claimable means a seat is being held for this ticket: re-POST immediately and that request lands as the claim. Treat position as a rough indicator, never a countdown, since priority ordering can move it backwards.

503 — no seat, and waiting will not help:

{ "error": "fleet_busy" }

fleet_busy is the terminal case: the line is full past the wait horizon, or no renderer is registered at all. Unlike 202, this attempt is over. Tell the user it is busy and let them retry.

Never re-mint in an unbounded loop. Poll a 202 on its retry_after, and bound the total wait. A naive “keep re-minting until it works” spins forever against a full fleet and looks like a hung session.

Two other refusals cannot be fixed by retrying. 429 concurrency_limit means your plan’s concurrent-session ceiling is reached; that is a plan limit rather than platform capacity, so it never enters the queue. 402 quota_exhausted means the allowance is spent until reset_at.

Once you have a 201, hand connect_url + session_token to your client — that’s the only thing your backend needs to produce.

5. Connect and talk (browser)

npm install @casola/avatar-client

import { AvatarSession, connectViaToken } from '@casola/avatar-client'
// The mic worklet is a real, separately-served asset (see "Worklet asset" below).
// With a bundler (Vite shown), import its URL — do NOT hardcode a path that
// doesn't exist in your app, or the mic silently never streams.
import workletUrl from '@casola/avatar-client/worklet?worker&url'

async function startCall() {
  // 1. Ask for the mic FIRST — make this the first thing your click/tap handler does.
  //    Doing it before you mint means a denied/absent mic fails cleanly without burning
  //    a GPU seat, and the user gesture is what unlocks mic permission + video autoplay.
  let permittedStream
  try {
    permittedStream = await AvatarSession.ensureMicPermission()
  } catch (err) {
    // err.name tells you why: NotAllowedError/SecurityError = denied,
    // NotFoundError = no mic, NotSupportedError = insecure origin (needs https/localhost).
    showMicError(err)
    return
  }

  // 2. Your backend endpoint wrapping step 4. Handle its 503 (see §4) — don't
  //    re-mint in an unbounded loop.
  const res = await fetch('/my-backend/start-session')
  if (res.status === 503) { showBusy(); return }
  const { connect_url, session_token } = await res.json()

  // 3. Connect. Reusing permittedStream avoids a second permission prompt.
  const session = new AvatarSession({
    videoEl: document.querySelector('video'),
    connect: connectViaToken({ connectUrl: connect_url, sessionToken: session_token }),
    workletUrl,                 // the imported URL — the servable path to the worklet
    permittedStream,
    lang: 'en',
    callbacks: {
      onFirstFrame()  { console.log('avatar is live') },
      onPartial(text) { console.log('you (so far):', text) },
      onTurn(t)       { console.log('you said:', t.text, '→ avatar replies:', t.reply) },
      onClose(reason) { console.log('ended:', reason) },
      // Fires on setup failures (worklet 404, getUserMedia denial) AND on mic-uplink
      // errors mid-call — so a dead microphone is never silent. Surface it.
      onError(err)    { console.error('session error:', err) },
    },
  })

  await session.start()
}

That’s the whole conversation loop: the SDK plays the avatar’s video+voice via MSE and streams your microphone up; the avatar listens, thinks, and answers by speaking — onPartial/onTurn are transcripts for your UI, not something you need to respond to. Speak over the avatar to interrupt it (barge-in is server-side).

The SDK handles ManagedMediaSource/MediaSource differences (iOS Safari), append queueing, latency housekeeping, mic capture, 48 kHz → 16 kHz resampling, and PCM16 framing. Microphone echoCancellation is always on — required, or the avatar hears itself through your speakers.

Worklet asset. audioWorklet.addModule() fetches a real script URL — the worklet cannot be inlined into your app bundle, so workletUrl must point at something your server actually serves. Two ways to get one:

• Bundler (Vite / most setups). Import the URL and pass it straight through — this is the snippet above. Your bundler emits the asset and gives you the hashed, servable URL:

  import workletUrl from '@casola/avatar-client/worklet?worker&url'

• No bundler. Copy the worklet into your static/public directory at build time (cp node_modules/@casola/avatar-client/dist/worklet/mic-worklet.js public/) and pass workletUrl: '/mic-worklet.js'.

If the mic never works, the usual cause is a worklet URL that 404s. It shows up as an onError with a failed addModule/network error and no uplink turns. Confirm the URL loads in your browser’s network tab, and check the console — with dev: true the SDK logs mic setup (AudioContext state, sample rate, per-frame peaks) to help you tell “no audio reaching the mic” from “mic never started.”

5b. Or connect without the SDK (server / native)

Two WebSockets against connect_url. Browsers must pass the token as ?token=<session_token> (no headers on WS upgrades); native/server clients should prefer the Authorization: Bearer header.

Socket	Path	Direction	Payload
Downlink	/mse	server → client	1 text frame {"mime": "..."}, then binary fMP4 segments
Uplink	/mic_stream	client → server	1 text frame {"op":"hello","lang":"en"}, then binary 16 kHz mono PCM16 frames (1600 samples = 100 ms); JSON partial / turn events come back

See the API Reference → Data plane for the full wire protocol.

Next steps

• API Reference — concepts + every endpoint, including custom avatars (your own face + voice), voice design, and prebuild warm-up
• Embed — the one-script-tag <avatar-embed> element, its attributes, and the data-record opt-in
• Browser SDK reference — full AvatarSession API
• Authentication — scoped keys, publishable keys for browser-only embeds
• Need help? Email support@casola.ai