Masterwork Avatar TTS - Real-Time 3D Avatar Lip Sync for the Web
Masterwork Avatar TTS is a real-time 3D avatar lip sync engine that runs natively in the browser. Type any text, and a Character Creator 4 (CC4) avatar speaks it with accurate mouth movements, natural facial expressions, and realistic eye behavior.
What Is Masterwork Avatar TTS?
It is the first web-native implementation of the CC4 viseme system. Our proprietary CC4 Viseme Engine maps text to 15 phoneme-accurate mouth shapes using a CMU pronunciation dictionary with over 126,000 words. The result is lip sync that matches what you would expect from a game engine, but running in a standard web browser with no plugins.
Key Features
- CC4 Viseme Engine: 15 phoneme-accurate mouth shapes with intelligent coarticulation, stress-based intensity, and jaw bone driver.
- Real-Time Text-to-Speech: Supports Azure Speech Services and ElevenLabs. Audio and viseme timing generated in real time.
- Natural Expressions: Procedural blink system, eye gaze with microsaccades and saccades, speech-driven brow gestures, and a 4-state conversation machine (idle, listening, thinking, speaking).
- Any CC4 Character: Bring your own Character Creator 4 avatars. Supports realistic and stylized characters from any CC4 face profile (ExPlus, Traditional, Standard).
- Web-Native: Built with Three.js and WebGL. Runs in Chrome, Firefox, Safari, and Edge. No Unreal Engine, Unity, or plugins needed.
- Blender Pipeline: Includes a Blender plugin for CC4-to-web export with automatic material conversion, morph validation, and animation baking.
Use Cases
- Virtual Assistants: Replace text chatbots with a speaking 3D avatar for customer support, onboarding, and FAQ.
- E-Learning: Create training content with a virtual instructor. No video production, just type the script.
- Games: Add talking NPCs to web games, interactive stories, and virtual tabletops.
- Marketing: Product demos with a branded avatar spokesperson. Update scripts without re-shooting.
How It Works
- Create a character in Character Creator 4.
- Export and prepare for web using our Blender plugin.
- Connect a TTS provider (Azure Speech or ElevenLabs).
- Embed on your website. Type text, the avatar speaks.
Technology
Built on Three.js for 3D rendering, the Microsoft Cognitive Services Speech SDK for Azure TTS integration, and a custom phoneme-to-CC4-viseme pipeline using the CMU Pronouncing Dictionary. The CC4 jaw bone driver replicates Character Creator's exact Blender driver formula for accurate jaw rotation from viseme weights.
Contact
For licensing, custom integration, or questions: mitchell@masterworkstudio.com
Visit masterworkstudio.com for more.