From the beginning, our mission has been to build a product that lets users create personalized 1:1 video content, and deliver it at scale.
We’re excited to share the next big steps in that journey, following our $18 million Series A funding led by Scale Venture Partners, Sequoia Capital, Y Combinator, HubSpot, and others.
Today, we’re launching the Public Beta of:
- Phoenix, our breakthrough model that delivers realistic digital replicas and text-to-video
- Tavus Developer Platform with our models packaged in easy-to-use APIs
Now, product teams can easily leverage our AI models and APIs to build novel engagement experiences with AI generated replica videos.
With Phoenix, end-users can generate entire videos of themselves, from just a script. No more pain of recording, or re-recording. Being able to rapidly make videos, opens up a world of experimentation and makes hyper personalization of video content possible.
Sign up to our developer platform.
The power of Phoenix
Phoenix generates realistic talking head videos from just a script, complete with natural lip, cheek, nose, eyebrow, chin movements and expressions synchronized with your script and generated voice.
Developed by our team, this novel approach bypasses traditional methods and constructs dynamic, three-dimensional facial scenes using neural radiance fields (NeRFs).
Most models out there use lip sync methods and operate in the 2D space. The Phoenix model takes a more advanced approach using 3D reconstruction, resulting in a digital replica that looks and speaks like you.
Access Phoenix via our APIs
We’re thrilled to be making our APIs available to developers and product teams on our new Developer Platform. Startups and large companies have been requesting access, and we see limitless potential for our technology's application.
Our APIs are easy to use, requiring one or few API calls and only one or a few parameters. We do everything else for you under the hood.
Today, you can access the Replica API in beta, with the rest of the suite coming soon.
Replica API (Beta)
The Replica API uses the full capability of the Phoenix model. Users can create a digital replica of themselves with just 2 minutes of training footage, and generate realistic videos from a script in minutes. Currently, the replica takes ~9 hours to process.
The model moves the entire face leading to realism, naturalness, and quality.
- Multi-language support, generate in 30 languages
- A video is generated from text in just one API call
- A replica is trained in one API call with one parameter (video URL)
- Tavus handles end-to-end pipeline
- Tavus handles privacy and security
Great for high-touch use cases where realism is important and the user is too busy to record.
Sign up to the developer platform.
Lip Sync API (Coming Soon)
The Lip Sync API is powered by a derivative of the lip sync portion of the Phoenix model, called Hummingbird. It allows a user to lip sync parts of, or an entire video.
- Lower model training time than Replica API
- Tavus handles privacy and security
Great for video editing use cases like making in-place changes to words or short phrases, or doing personalization with variables.
Sign up for waitlist.
Dubbing API (Coming Soon)
The Dubbing API uses the same technology as lip sync and a multi-language voice cloning model. A video is translated from one language to another. The final result looks and sounds like the original speaker is talking in the target language.
- Lower model training time than Replica API
- Tavus handles privacy and security
This could be used to translate any sort of video for example, training videos or TED-style talks into different languages to broaden the audience.
Sign up for waitlist.
Video Campaign API (Coming Soon)
The Video Campaign API allows developers to provide an end-to-end video campaign experience out of the box, within their solutions. It's a bundle of our Replica API, along with campaign tooling including:
- Video hosting
- Variable mapping and management
- Personalized landing pages and thumbnails
- Analytics
This is great for CRM platforms, as it provides the ability for templated video personalization that maps to CRM data directly, and gives end to end hosting and analytics solutions.
Sign up for waitlist.
What will you build?
We’re excited to see what you dream up with replica videos.
Here are some pretty cool end-user use cases we’ve already seen from developer partners:
- Airline Loyalty Program: A high-touch experience for frequent flyers. Their digital journey starts with a personalized (generated) video from the pilot on flight status, menu options, and other info based on their upcoming flight.
- Customer engagement: Customer teams generate videos for outreach, relationship management, and more. Reaching more customers with video with less resources.
- Doctors are getting into video! Doctors can make video sequences to help build a relationship with patients and improve comfort and care between appointments.
- Making training and compliance fun. Personalize compliance content with employees as stars in the content. Jack from IT, your CEO or Jackie from Design speaking in any language.
Video will just be different
The Phoenix model allows users to completely reinvent how they use video.
You can now generate 10s, 100s, or 1000s of videos of yourself, without having to record and re-record, in minutes. It’s not just volume. You can now create videos and experiment in ways that were previously impossible. For example, having you speak in a language native to the viewer. Or having busy people make a lot of video content, like pilots, CEOs, doctors, lawyers.
You only make replicas of you
Tavus is built with privacy and security front-of-mind so developers can focus on user experience.
A person is only able to create a replica of themselves. Our technology cannot be used to replicate others. As part of our APIs, we employ a suite of safety checks including voice identification and user consent to ensure only a user can make a replica of themselves and holds the keys to their likeness. The process is handled on behalf of users by Tavus.
Your feedback
We’re looking forward to hearing your feedback on the Beta.There is still much work to do, and we want to learn from you.
We’re already working on a few things including reducing the time it takes to train a replica, and continuously improving our documentation.
What are you going to build with AI video?
Sign up www.tavus.io/developer
Read documentation https://docs.tavusapi.com/sections/introduction
.gif)
