December 12, 2025 (compiled from Zhipu AI official site, IT Home and OSCHINA) In an era of rapid advancement in speech synthesis, Zhipu AI released the industrial‑grade text‑to‑speech system GLM‑TTS on December 11 and simultaneously open‑sourced the model weights and inference code. This innovative system can accurately clone a user’s timbre, intonation and speaking habits from a 3‑second voice sample, generating natural, fluent and emotionally rich speech. GLM‑TTS’s debut not only marks an open‑source breakthrough by a Chinese AI company in TTS, but also injects new vitality into education, voiced content and intelligent customer service. At the launch, Zhipu AI founder Tang Jie said, "GLM‑TTS is not simply about ‘speaking’; it lets AI ‘express’—conveying genuine emotion in appropriate contexts." The system is available on Hugging Face and ModelScope, quickly drawing developer community attention, and the GitHub repository received over 10,000 stars within 24 hours.
Background: From the GLM Series to a Closed Voice Ecosystem
As a model of industry–university–research collaboration with Tsinghua University, Zhipu AI has built a complete GLM model family since releasing ChatGLM‑6B in 2022, including the multimodal GLM‑4 series and the GLM‑ASR speech recognition system. The release of GLM‑TTS completes the ecosystem’s "voice output" link, aiming to address traditional TTS limitations in emotional expression and timbre restoration. Unlike earlier systems that relied on massive datasets, GLM‑TTS emphasizes "few‑shot efficiency": a 3‑second reference audio can achieve zero‑shot voice cloning—an open‑source milestone at this scale.
The timing aligns with industry trends. In 2025 the global TTS market is projected to exceed $15 billion with year‑on‑year growth around 25%, yet open‑source models remain dominated by English and lag in Chinese emotional synthesis. Zhipu AI’s decision to open‑source GLM‑TTS under the Apache 2.0 license aims to foster community collaboration while securing a domestic lead in AI voice. Official data show that after launching on Zhipu’s open platform (bigmodel.cn), the Zhipu Qingyan app and Z.ai, first‑day API calls exceeded 100,000. Zhipu also open‑sourced GLM‑ASR concurrently, forming a "recognition+synthesis" closed loop that strengthens its competitive position in speech large models.
Technical Architecture: Two‑Stage Generation + Multi‑Reward Reinforcement Learning
GLM‑TTS’s core lies in its innovative architecture and training strategy, overturning the traditional end‑to‑end TTS limitations. The system adopts a two‑stage generation paradigm: the first stage uses a large language model (LLM) to convert text into a semantic token sequence, ensuring content accuracy and contextual coherence; the second stage employs a Conditional Flow‑Matching diffusion model to predict mel spectrograms, which are then reconstructed into high‑quality waveforms by a 2D‑Vocos vocoder. This "semantic modeling + acoustic reconstruction" design reduces computational complexity and supports streaming inference—first‑frame latency can be under 400 ms, suitable for real‑time interactive scenarios.
The key breakthrough is multi‑reward reinforcement learning (RL) during training. Traditional TTS often remains at supervised fine‑tuning (SFT) and can produce "robotic" outputs. GLM‑TTS introduces a GRPO (Generalized Reward‑Pretrained Optimization)‑based multidimensional reward mechanism, including character error rate (CER), timbre similarity (SIM), emotional expression (Emotion) and paralinguistic features (such as laughter and pauses). Through dynamic sampling and gradient clipping, the model improves expressiveness and robustness while maintaining controllability. Notably, pretraining requires only 100,000 hours of data (far below the industry mainstream of millions of hours), and can be completed on a single machine in four days; LoRA fine‑tuning for high‑quality timbres and the RL stage each take only one day. In addition, Phoneme‑in technology uses a dynamic lexicon and hybrid input to achieve precise pronunciation control for polyphonic and rare characters, supporting dialect cloning and emotion switching.
The model card on Hugging Face indicates GLM‑TTS supports Chinese–English mixing, multiple reading styles (e.g., news, storytelling) and dynamic parameter adjustment (speech rate, volume, emotion intensity). Developers can quickly deploy via the GitHub repository (github.com/zai‑org/GLM‑TTS), with support for mainstream frameworks such as PyTorch.
Leading Performance: Open‑Source SOTA, Outperforming Commercial Models on Dual Metrics
In benchmarks, GLM‑TTS demonstrates dominant advantages. In open evaluations, its character error rate (CER) is only 0.89% and timbre similarity (SIM) reaches 76.4%, outperforming Seed‑TTS (CER 1.12%, SIM 79.6% but not open‑sourced). For emotional expression, the GLM‑TTS RL variant achieves a "double SOTA": emotional accuracy increased by 20%, and overall MOS (mean opinion score) exceeds commercial models at lower cost. Independent tests by 53AI show that after a 3‑second clone, the model can perfectly reproduce discourse particles (such as “aiya”) and emotional inflections, maintaining natural rhythm even in long sentences.
Compared with international competitors like Google’s WaveNet or OpenAI’s TTS‑1, GLM‑TTS offers superior Chinese support: cross‑lingual cloning is strong and light‑accent preservation exceeds 95%. OSCHINA’s evaluation notes its streaming interface leads open‑source peers by about 20% in low‑latency scenarios (e.g., smart speakers). These gains stem from Zhipu’s "efficient training" philosophy: LoRA fine‑tuning modifies only 15% of parameters to achieve full‑parameter‑level timbre customization, lowering the barrier for enterprise deployment.
Open Ecosystem and Production Adoption: From Demos to Production
Zhipu’s open‑source approach is forward‑looking. Model weights, inference scripts and sample projects are published on Hugging Face (huggingface.co/zai‑org/GLM‑TTS) and ModelScope for one‑click downloads. The Zhipu open platform provides APIs supporting non‑streaming (full‑text synthesis) and streaming (real‑time output) calls with flexible billing (per character or QPS). Online demos include Z.ai’s audio.z.ai (upload text/voice prompts to generate a personalized voice) and the Zhipu Qingyan app (multi‑style reading within conversations).
Applications are broad: in education assessment, GLM‑TTS can simulate teacher–student interactions to enhance oral practice realism; in audiobooks, it supports emotional dubbing and cuts production time by about 50%; in customer service, it enables personalized responses and reduces labor costs. Zhipu has partnered with multiple enterprises, such as in automotive voice interaction systems, where GLM‑TTS may reshape in‑car voice. In the future, combined with GLM‑4‑Voice end‑to‑end speech large models, GLM‑TTS will expand into real‑time dialogue and virtual anchors.
Market Impact: Domestic TTS Accelerates Globalization
The open‑sourcing of GLM‑TTS will accelerate TTS democratization. IDC predicts the global voice AI market will reach $30 billion by 2026, with China accounting for over 30%. As a domestic representative, Zhipu’s move lowers the barrier for developers and promotes ethical norms: the company emphasizes data privacy protection and requires user authorization for cloning. Challenges remain, such as deepfake risks, and Zhipu plans to introduce watermarking mechanisms.
The release of GLM‑TTS signals a shift of TTS from a "tool" to a "partner." Zhipu AI says it will continue iterating to support more dialects and multimodal fusion. Developer feedback has been enthusiastic: one Hugging Face user said, "3‑second cloning is incredible—I integrated it into my audio app immediately." In the AI voice race, China’s momentum is accelerating, and GLM‑TTS may become the next open‑source benchmark.
▶️ Enjoy this article? Creating content is hard work!
Follow our YouTube channel AI Rosetta, your likes and subscriptions are our greatest support! 🙏