Public beta

Embed an Agent in your application

Connect a published Agent to a browser without exposing realtime transport, provider configuration, or permanent API keys. This guide assumes nothing beyond a published Agent.

Two packages

Choose your integration

Pick the smallest package that matches who owns rendering.

@animated-waffle/client

Framework-independent lifecycle, microphone input, text input, state, and transcripts. Your app owns the UI.

@animated-waffle/react

The client API plus React state, Agent audio, and the published Avatar renderer.

Before you code

Three things you need

  1. 1

    A published Agent

    Build and publish it in the Dashboard first — the Agents guide covers every step. Session tokens are only minted for published Agents.
  2. 2

    A workspace API key

    An Owner or Admin creates it under Integrations in the Dashboard sidebar (direct link: /operator/keys): enter a Label, select Create key, and copy the awp_… value immediately — it is shown exactly once. Full walkthrough in the API reference.
  3. 3

    The Agent’s id

    Your backend calls GET /v1/agents with the key and selects your Agent by its slug; the response’s id is the stable UUID used everywhere below.

Public beta

Install from npm

Terminal
npm install @animated-waffle/client

# React applications
npm install @animated-waffle/react

Both packages are published on npm. @animated-waffle/react already includes the client — install it alone for React applications.

Trust boundary

Mint session tokens on your backend

Permanent workspace keys belong on your backend. The browser asks your backend for one short-lived Agent token per connection; the SDK does the rest.

server.ts · your token endpoint
// Your backend. The awp_ key comes from Dashboard → Integrations
// and never leaves the server.
app.post('/api/animated-waffle/session-token', async (req, res) => {
  const user = await authenticateYourUser(req)
  if (!user) return res.status(401).end()

  const response = await fetch(
    `https://animated-waffle.narya.ai/v1/agents/${AGENT_ID}/session-tokens`,
    {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${process.env.NARYA_API_KEY}`,
        'Content-Type': 'application/json',
      },
      // end_user_id: required when the Agent has memory or Calendar
      // enabled; must be a stable UUID for this user in your system.
      body: JSON.stringify({ end_user_id: user.stableUuid }),
    },
  )
  if (!response.ok) return res.status(502).json(await response.json())

  const { token } = await response.json()
  res.json({ token })
})
What Narya returnsHTTP 201 with { token, expires_at, agent_id, capabilities }. The token is valid for 15 minutes and starts exactly one Agent conversation.
Authorize your own user firstBefore minting, authenticate the user, verify they may talk to this Agent, and apply your own rate, duration, concurrency, and spend limits.
end_user_idOptional unless the Agent has conversation memory, documents, or Calendar enabled — then it must be a stable UUID for this user from your identity system. Send {} otherwise.

Framework independent

Connect with JavaScript

Use the client package when your application already owns rendering and controls.

client.ts
import { WaffleClient } from '@animated-waffle/client'

const agentId = '22222222-2222-4222-8222-222222222222'
const client = new WaffleClient({
  agentId,
  sessionToken: async () => {
    const response = await fetch('/api/animated-waffle/session-token', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ agentId }),
    })
    if (!response.ok) throw new Error('Could not start a session')
    return (await response.json()).token
  },
})

client.on('transcript', ({ id, role, text, final }) => {
  if (final) console.log(id, role, text)
})
client.on('connectionState', (state) => console.log('connection:', state))

await client.connect({ inputMode: 'push-to-talk' })
Option / memberWhat it does
agentIdThe published Agent’s UUID from GET /v1/agents. Must be a UUID, not a slug.
sessionTokenA function (sync or async) returning a fresh token from your backend. Called on each connect. Alternatively pass startSession to start sessions through your own authenticated backend; the two options are mutually exclusive.
apiOriginOptional. Point at https://animated-waffle-dev.narya.ai for the development environment; defaults to production.
inputAudioEnabledOptional. Set false for text-triggered, output-only applications; the SDK never requests the microphone and PTT controls fail closed with input_audio_disabled. Defaults to true and stays off until the Agent is ready.
audioPlaybackOptional, default true: the client plays the Agent’s voice through its own audio element. Set false to play the remoteAudioTrack yourself — an app routing the voice through its own element or Web Audio graph must turn this off, or the voice is heard twice.
connect({ inputMode })‘push-to-talk’ (default) or ‘automatic’ for an open microphone with voice detection. Resolves with the session.
sendText(text, { audioResponse })Sends a typed message into the conversation. The Agent replies with speech by default; pass audioResponse: false to make that one turn text-only — the reply still streams as transcript, no audio is synthesized.
mutedRead/write. Silences the Agent’s voice while the session, transcript, and Avatar performance carry on. No effect when audioPlayback is false.
startPushToTalk() / commitPushToTalk() / cancelPushToTalk()Hold-to-talk controls. Throw unless connected with inputMode ‘push-to-talk’.
on(‘connectionState’ | ‘session’ | ‘transcript’ | ‘agentSpeaking’ | ‘remoteAudioTrack’ | ‘error’, fn)Event subscriptions; each on() call returns its unsubscribe function. Conversation transcripts use the same required id as persisted history.
disconnect()Ends the session and releases the microphone.

Daily, Pipecat, room credentials, and Avatar asset URLs stay inside the SDK — your code never handles them.

React

Connect and render with useWaffle

Use the React package when Animated Waffle should own Agent audio and Avatar playback.

Agent.tsx
import { AvatarStage, useWaffle } from '@animated-waffle/react'

export function Agent() {
  const agentId = '22222222-2222-4222-8222-222222222222'
  const waffle = useWaffle({
    agentId,
    sessionToken: () => fetchSessionToken(agentId),
  })

  return (
    <>
      {/* Session churn rebinds signals without removing the visible stage */}
      <AvatarStage {...waffle.avatar} style={{ width: 480, height: 480 }} />
      {waffle.avatarLoad.status === 'downloading' && (
        <progress value={waffle.avatarLoad.progress ?? undefined} />
      )}
      <button
        disabled={waffle.connectionState !== 'disconnected'}
        onClick={() => void waffle.connect()}
      >
        Start conversation
      </button>
      <button onClick={() => void waffle.disconnect()}>End</button>
    </>
  )
}

The hook returns the app-facing client state — connectionState, session, transcript, agentSpeaking, muted/setMuted, performance (the cue the Avatar is performing right now), error, connect, disconnect, sendText, and the push-to-talk controls — plus an opaque avatar binding. Spread it onto a sized AvatarStageand the published character renders itself without exposing the remote audio track. The returned action functions keep stable identity across renders. avatarLoad reports first-load progress (downloading → verifying → parsing → ready with byte counts); repeat visits come from the browser cache after hash verification, so loading is near-instant.

Disconnect and reconnect only rebind session signals. The visible renderer is destroyed when AvatarStage unmounts.

When it fails

Errors you will actually see

FailureCause and fix
Your token endpoint gets 409 “Publish the agent before requesting a session token.”The Agent is still a draft. Save & publish it in the Dashboard.
Your token endpoint gets 400 memory_identity_required, documents_identity_required, or calendar_identity_requiredThe Agent has conversation memory, documents, or Calendar enabled but the request had no end_user_id. Send a stable UUID for the user.
Your token endpoint gets 401 unauthorizedThe awp_ key is missing, wrong, or revoked — check the Integrations page and your secret store.
The SDK throws ‘agentId must be a UUID.’A slug was passed as agentId. Resolve the slug to the id through GET /v1/agents on your backend.
connect() rejects with an expired-token errorTokens live 15 minutes and the sessionToken callback returned a cached one. Mint a fresh token on every call.
Push-to-talk methods throwThe client was connected with inputMode ‘automatic’. Reconnect with ‘push-to-talk’ or rely on voice detection.

Before launch

Production checklist

Publish and test the Agent in Live conversation before wiring the SDK.
Keep the awp_ workspace key only on your backend.
Bind end_user_id to a stable user in your own product.
Apply authorization, rate, concurrency, duration, and spend limits before minting tokens.
Handle disconnected and error states in the application UI.
Size AvatarStage and show avatarLoad progress on first load.