More articles
Higgs Avatar now streams live into voice agent calls
Higgs Avatar now streams live into voice agent calls. A realtime session adds a face to your voice agent: your agent streams its speech to the avatar, and the avatar publishes lip-synced video and that speech into your LiveKit room.
Realtime sessions are the live counterpart of
POST /v1/videos. Video jobs render a finished clip from a still image and audio you already have, and /v1/videos/stream starts playback before the clip finishes. A realtime session uses the same higgs-avatar model and the same API key, and renders from audio that arrives while your agent is still speaking.How the stream flows
The avatar is a participant in a room you own. Your agent joins the same room and sends it speech as 24 kHz, 16-bit PCM over a LiveKit data stream, using the avatar audio protocol from LiveKit Agents. The avatar renders that speech into video and publishes video and audio tracks in sync, so the user hears the voice from the face that is saying it.
Media stays in your room. Boson’s session routes start, inspect and release the avatar, and never carry audio or video.
Start a session
One request puts the avatar in the room. It names a preset avatar and describes your room, including an access token you mint for the avatar with your own LiveKit key.
The code below is an excerpt. The complete, working script is in the documentation.
Python
# Excerpt from agent.py in the session guide. The code before and after is omitted.import osimport requests...session = requests.post("https://api.boson.ai/v1/avatar/sessions",headers={"Authorization": f"Bearer {os.environ['BOSON_API_KEY']}"},json={"avatar_id": "builtin_claire_v1","transport": {"type": "livekit","url": os.environ["LIVEKIT_URL"],"room_name": "avatar-demo","participant_token": avatar_token, # minted with your LiveKit key"participant_identity": "boson-avatar","publisher_identity": "agent",},"max_duration_seconds": 1800,},timeout=60,).json()...
The request returns once the avatar has joined. Boson never receives your LiveKit API secret, only the avatar’s token, and the avatar takes speech from one participant: the agent named in
publisher_identity.A session ends when you release it with
DELETE, when max_duration_seconds runs out, or once everyone else has left the room.Pairs with Higgs Realtime
Any agent that speaks the LiveKit Agents avatar protocol can drive the avatar. The guide pairs it with Higgs Realtime, which holds the conversation and answers in speech, so one Boson API key covers the conversation and the face. The agent passes each chunk of the model’s speech to the avatar as it arrives.
The code below is an excerpt. The complete, working script is in the documentation.
Python
# Excerpt from agent.py in the session guide. The code before and after is omitted.from livekit import rtcfrom livekit.agents.voice.avatar import DataStreamAudioOutput...to_avatar = DataStreamAudioOutput(room, destination_identity="boson-avatar", sample_rate=24000,wait_remote_track=rtc.TrackKind.KIND_AUDIO,)...async for message in ws: # the Higgs Realtime WebSocketevent = json.loads(message)if event["type"] == "response.output_audio.delta":pcm = base64.b64decode(event["delta"])await to_avatar.capture_frame(rtc.AudioFrame(pcm, 24000, 1, len(pcm) // 2))elif event["type"] == "response.done":to_avatar.flush() # end of the replyelif event["type"] == "input_audio_buffer.speech_started":to_avatar.clear_buffer() # the user cut in......
When the user starts talking over the avatar,
clear_buffer drops the speech it hasn’t played yet, and the face stops mid-word along with the voice.Availability
Realtime sessions are available today through the Boson API. Every key can use Boson’s preset avatars, such as
builtin_claire_v1, and GET /v1/avatars lists the ones yours can use. Video is 480 × 640 by default.Live minutes are billed at the same rates as rendered video: $0.07 a minute at 480 × 640. The audio your agent sends in is free.
Sessions run in LiveKit Cloud projects, and LiveKit’s free tier is enough to try them. A LiveKit Agents plugin that handles the avatar side for you is on its way.
Get started
Meet Higgs Avatar
Demos, pricing and rendered video on the product page.
Learn moreGet an API key
One key covers the avatar session and the Higgs Realtime conversation.
Workspace API keysIf you build something with Higgs Avatar, let us know: contact@boson.ai.