Speech-to-Video (S2V)
Speech-to-Video (S2V) is technology that turns an audio stream into real-time video of a person speaking it, with matching lip movement and expressions. It lets developers add a face to a voice agent without changing how the agent thinks or speaks.
How Speech-to-Video works
Your voice agent produces audio as usual. Instead of only playing it, it sends the audio to the speech-to-video model, which returns video frames of the avatar speaking those words in sync. The video streams to the user alongside the audio, typically through a real-time transport such as LiveKit.
Because S2V only handles the face, you keep full control of speech recognition, the language model, the voice and the orchestration.
How Beyond Presence does it
The Beyond Presence Speech-to-Video API renders the Avatar in about 100 ms, at 1080p and 35 FPS, on the Genesis 2.0 model. It works with any language your connected text-to-speech provider supports, 100+ in total. LiveKit is the recommended integration, with Python and JavaScript SDKs. Read the API introduction.
Frequently asked questions
What does Beyond Presence Speech-to-Video do?
It turns your agent's audio into real-time video of an Avatar speaking it, at 1080p and 35 FPS, in about 100 ms.
Do I need to change my voice agent to use Speech-to-Video?
No. Your speech recognition, LLM, voice and orchestration stay the same. Speech-to-Video adds the video layer.
Which languages does Beyond Presence Speech-to-Video support?
Any language your text-to-speech provider speaks, 100+ in total.