Embed an Agent in your application
Connect a published Agent to a browser without exposing realtime transport, provider configuration, or permanent API keys. This guide assumes nothing beyond a published Agent.
Two packages
Choose your integration
Pick the smallest package that matches who owns rendering.
@animated-waffle/client
Framework-independent lifecycle, microphone input, text input, state, and transcripts. Your app owns the UI.
@animated-waffle/react
The client API plus React state, Agent audio, and the published Avatar renderer.
Before you code
Three things you need
- 1
A published Agent
Build and publish it in the Dashboard first — the Agents guide covers every step. Session tokens are only minted for published Agents. - 2
A workspace API key
An Owner or Admin creates it under Integrations in the Dashboard sidebar (direct link: /operator/keys): enter a Label, select Create key, and copy theawp_…value immediately — it is shown exactly once. Full walkthrough in the API reference. - 3
The Agent’s id
Your backend callsGET /v1/agentswith the key and selects your Agent by its slug; the response’sidis the stable UUID used everywhere below.
Public beta
Install from npm
npm install @animated-waffle/client
# React applications
npm install @animated-waffle/reactBoth packages are published on npm. @animated-waffle/react already includes the client — install it alone for React applications.
Trust boundary
Mint session tokens on your backend
Permanent workspace keys belong on your backend. The browser asks your backend for one short-lived Agent token per connection; the SDK does the rest.
// Your backend. The awp_ key comes from Dashboard → Integrations
// and never leaves the server.
app.post('/api/animated-waffle/session-token', async (req, res) => {
const user = await authenticateYourUser(req)
if (!user) return res.status(401).end()
const response = await fetch(
`https://animated-waffle.narya.ai/v1/agents/${AGENT_ID}/session-tokens`,
{
method: 'POST',
headers: {
Authorization: `Bearer ${process.env.NARYA_API_KEY}`,
'Content-Type': 'application/json',
},
// end_user_id: required when the Agent has memory or Calendar
// enabled; must be a stable UUID for this user in your system.
body: JSON.stringify({ end_user_id: user.stableUuid }),
},
)
if (!response.ok) return res.status(502).json(await response.json())
const { token } = await response.json()
res.json({ token })
})Framework independent
Connect with JavaScript
Use the client package when your application already owns rendering and controls.
import { WaffleClient } from '@animated-waffle/client'
const agentId = '22222222-2222-4222-8222-222222222222'
const client = new WaffleClient({
agentId,
sessionToken: async () => {
const response = await fetch('/api/animated-waffle/session-token', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ agentId }),
})
if (!response.ok) throw new Error('Could not start a session')
return (await response.json()).token
},
})
client.on('transcript', ({ id, role, text, final }) => {
if (final) console.log(id, role, text)
})
client.on('connectionState', (state) => console.log('connection:', state))
await client.connect({ inputMode: 'push-to-talk' })| Option / member | What it does |
|---|---|
| agentId | The published Agent’s UUID from GET /v1/agents. Must be a UUID, not a slug. |
| sessionToken | A function (sync or async) returning a fresh token from your backend. Called on each connect. Alternatively pass startSession to start sessions through your own authenticated backend; the two options are mutually exclusive. |
| apiOrigin | Optional. Point at https://animated-waffle-dev.narya.ai for the development environment; defaults to production. |
| inputAudioEnabled | Optional. Set false for text-triggered, output-only applications; the SDK never requests the microphone and PTT controls fail closed with input_audio_disabled. Defaults to true and stays off until the Agent is ready. |
| audioPlayback | Optional, default true: the client plays the Agent’s voice through its own audio element. Set false to play the remoteAudioTrack yourself — an app routing the voice through its own element or Web Audio graph must turn this off, or the voice is heard twice. |
| connect({ inputMode }) | ‘push-to-talk’ (default) or ‘automatic’ for an open microphone with voice detection. Resolves with the session. |
| sendText(text, { audioResponse }) | Sends a typed message into the conversation. The Agent replies with speech by default; pass audioResponse: false to make that one turn text-only — the reply still streams as transcript, no audio is synthesized. |
| muted | Read/write. Silences the Agent’s voice while the session, transcript, and Avatar performance carry on. No effect when audioPlayback is false. |
| startPushToTalk() / commitPushToTalk() / cancelPushToTalk() | Hold-to-talk controls. Throw unless connected with inputMode ‘push-to-talk’. |
| on(‘connectionState’ | ‘session’ | ‘transcript’ | ‘agentSpeaking’ | ‘remoteAudioTrack’ | ‘error’, fn) | Event subscriptions; each on() call returns its unsubscribe function. Conversation transcripts use the same required id as persisted history. |
| disconnect() | Ends the session and releases the microphone. |
Daily, Pipecat, room credentials, and Avatar asset URLs stay inside the SDK — your code never handles them.
React
Connect and render with useWaffle
Use the React package when Animated Waffle should own Agent audio and Avatar playback.
import { AvatarStage, useWaffle } from '@animated-waffle/react'
export function Agent() {
const agentId = '22222222-2222-4222-8222-222222222222'
const waffle = useWaffle({
agentId,
sessionToken: () => fetchSessionToken(agentId),
})
return (
<>
{/* Session churn rebinds signals without removing the visible stage */}
<AvatarStage {...waffle.avatar} style={{ width: 480, height: 480 }} />
{waffle.avatarLoad.status === 'downloading' && (
<progress value={waffle.avatarLoad.progress ?? undefined} />
)}
<button
disabled={waffle.connectionState !== 'disconnected'}
onClick={() => void waffle.connect()}
>
Start conversation
</button>
<button onClick={() => void waffle.disconnect()}>End</button>
</>
)
}The hook returns the app-facing client state — connectionState, session, transcript, agentSpeaking, muted/setMuted, performance (the cue the Avatar is performing right now), error, connect, disconnect, sendText, and the push-to-talk controls — plus an opaque avatar binding. Spread it onto a sized AvatarStageand the published character renders itself without exposing the remote audio track. The returned action functions keep stable identity across renders. avatarLoad reports first-load progress (downloading → verifying → parsing → ready with byte counts); repeat visits come from the browser cache after hash verification, so loading is near-instant.
Disconnect and reconnect only rebind session signals. The visible renderer is destroyed when AvatarStage unmounts.
When it fails
Errors you will actually see
| Failure | Cause and fix |
|---|---|
| Your token endpoint gets 409 “Publish the agent before requesting a session token.” | The Agent is still a draft. Save & publish it in the Dashboard. |
| Your token endpoint gets 400 memory_identity_required, documents_identity_required, or calendar_identity_required | The Agent has conversation memory, documents, or Calendar enabled but the request had no end_user_id. Send a stable UUID for the user. |
| Your token endpoint gets 401 unauthorized | The awp_ key is missing, wrong, or revoked — check the Integrations page and your secret store. |
| The SDK throws ‘agentId must be a UUID.’ | A slug was passed as agentId. Resolve the slug to the id through GET /v1/agents on your backend. |
| connect() rejects with an expired-token error | Tokens live 15 minutes and the sessionToken callback returned a cached one. Mint a fresh token on every call. |
| Push-to-talk methods throw | The client was connected with inputMode ‘automatic’. Reconnect with ‘push-to-talk’ or rely on voice detection. |
Before launch