PhonemeX Pro – Offline TTS, ASR, Lip-Sync, Avatar & Face Rig System

PhonemeX Pro Edition is a modular, fully offline Text-to-Speech, Speech Recognition, and character voice system built for Unity mobile, desktop, and web builds.

It provides real-time, on-device speech synthesis, local speech-to-text transcription, and full access to phoneme timing data at runtime, enabling speech generation, lip-sync, facial animation, and voice-driven character systems.

What’s New in Version 1.2.0 

PhonemeX Pro has evolved into a complete bidirectional voice suite. Version 1.2.0 introduces a brand-new Speech Recognition (ASR) module powered by local Whisper models, and a zero-latency Audio-Reactive Lip-Sync system for fast, FFT-based mouth movements. The entire codebase has also been deeply modularized, separating Expressions from Lip-Sync to allow for perfectly layered facial animation without conflicts. Existing projects remain fully compatible.

Core Systems Included

• On-Device Text-to-Speech System 

A cross-platform, fully offline TTS engine that runs entirely inside Unity. Generates real-time audio along with detailed phoneme timing data for runtime logic and animation.

• On-Device Speech Recognition System (New!) 

A robust ASR pipeline integrating multilingual Whisper models for native speech-to-text inference. Runs completely locally via Unity Sentis (Desktop/Mobile) and WebAssembly (WebGL). Includes a built-in Download Manager UI for easy model fetching.

• Real-Time Lip-Sync System

A dedicated lip-sync pipeline that schedules phoneme data, blends viseme weights, and drives facial motion in real time. Designed for accurate speech-to-animation synchronization.

• Audio-Reactive Lip-Sync (New!) 

A high-speed, zero-latency alternative to phoneme lip-sync. Uses real-time FFT band analysis, Automatic Gain Control (AGC), and direct jaw bone rotation for instant audio-driven mouth movement directly from an audio source.

• Modular Avatar System

PhonemeX Pro includes a flexible avatar integration layer compatible with the AvatarX character set. The avatar system applies lip-sync and expression data to fully rigged characters using blendshapes or custom drivers, enabling speech-driven facial animation with minimal setup.

The system is designed to work with a wide range of humanoid characters and animation setups, allowing developers to integrate PhonemeX into existing projects without being locked to a specific character format.

• Face Rig & Expression System

An optional facial rigging and expression framework for authoring, blending, and layering facial poses, emotions, and micro-expressions on top of lip-sync animation.

Modular Design

• Each system can be used independently depending on your project needs:
• Use only the TTS system for speech playback
• Combine TTS and lip-sync for talking characters
• Integrate the avatar and face rig systems for full facial animation control

Modules are loosely coupled and designed to be extended or replaced as your project grows.

Key Features

• Fully Offline, On-Device TTS – No internet connection or server required by default
• Real-Time Phoneme Access – Phoneme timing data available at runtime
• Cross-Platform Support – Android, iOS, WebGL, Windows, macOS, and Linux
• Lightweight & Fast – Optimized for mobile and browser-based builds
• Multi-Voice & Multi-Language ASR Support – Easily switch between models
• Multi-Voice Support – Easily switch between supported voice models
• Modular Runtime Architecture – Clean separation between voice, lip-sync, and animation systems
• Optional online phoneme services can be integrated if desired, but are not required for core functionality.
• Local AI Inference – Uses Unity’s official on-device AI inference packages
• Demo Scenes Included – Examples showcasing TTS, lip-sync, avatar integration, and face rig workflows

Supported Platforms

• Android (ARM / ARM64)
• iOS / iPadOS
• WebGL
• Windows (x86 / x64)
• macOS
• Linux

Dependencies / Requirements

• Unity 6 LTS or newer
• On-device AI inference package: Unity AI Inference Engine
• Unity Editor Coroutines package: (com.unity.editorcoroutines)
• Newtonsoft Json package: (com.unity.nuget.newtonsoft-json, included with Unity)

Required packages can be installed automatically via the PhonemeX installer.

Perfect For

• Games and visual novels
• Digital humans and virtual characters
• Interactive NPCs and AI agents
• Educational and accessibility applications
• Web-based interactive tools and simulations

Need Help or Want to Connect

Join our official Carruto Support Discord: Join Here

Documentation | Support Email

Get updates, ask questions, and share your projects with the community.

Demo Builds

You can download prebuilt demo applications to test the asset before purchasing:

Windows Demo: Download

MacOS Demo: Download

These demos showcase the real runtime behavior of the asset.

No login or registration required.

PhonemeX Pro is designed as a scalable voice and character animation foundation, from simple on-device speech to fully animated, expressive characters.

Created by Carruto – professional tools for AI-driven audio and character systems.