Introduction
Between 2025 and 2026, realtime large language models (Realtime LLMs) advanced rapidly, supporting full‑duplex dialogue, low‑latency responses and end‑to‑end voice processing, becoming a core technology for AI interactions. These models have shifted from pure text generation to natural voice interaction and are widely applied in customer service, education and entertainment. According to NVIDIA official research and third‑party evaluations such as MarkTechPost, NVIDIA's PersonaPlex‑7B‑v1, xAI's Grok Voice Agent API, OpenAI's gpt‑realtime, Google's Gemini Live, Qwen's Qwen 2.5 Omni, Moshi and Freeze Omni are representative products. This report draws on the NVIDIA website, OpenAI developer documentation, Google Cloud pricing pages and authoritative sources such as the Shakudo blog and Asterisk community discussions to compare these models' release dates, overviews, openness, performance metrics, usage methods and cost analysis, ensuring factual accuracy and reliability.
Release dates
- PersonaPlex‑7B‑v1 (NVIDIA): Released on January 17, 2026, as NVIDIA's first open‑source realtime speech‑to‑speech model.
- Grok Voice Agent API (xAI): Released on December 17, 2025, expanding Grok's voice capabilities.
- gpt‑realtime (OpenAI): The Realtime API launched on October 1, 2024, and was upgraded via Advanced Voice Mode on August 28, 2025.
- Gemini Live (Google): Updated in September 2025 to support realtime voice interaction.
- Qwen 2.5 Omni (Alibaba Qwen): Released mid‑2025, emphasizing multimodal realtime capabilities.
- Moshi: Open‑sourced in early 2025 as a realtime voice benchmark model.
- Freeze Omni: Released in late 2025, focusing on efficient realtime interaction.
Release timing shows OpenAI and Moshi introduced earlier versions, while NVIDIA and xAI's 2025–2026 updates emphasize open‑source access and performance optimization.
Model overviews
- PersonaPlex‑7B‑v1: A 7‑billion‑parameter end‑to‑end speech‑to‑speech model using a single Transformer architecture, supporting persona roles, voice control and natural interruptions. Designed for realtime conversational agents.
- Grok Voice Agent API: Hundreds of billions of parameters with a proprietary voice stack, supporting multilingual switching and tool calling, emphasizing a humorous style and realtime translation.
- gpt‑realtime: Built on GPT‑4o with a trillion‑parameter family, supporting multimodal (audio, vision, text) realtime inference and offering six preset voices and emotional expression.
- Gemini Live: Google's multimodal realtime model supporting image, audio and video processing, integrated into the Gemini apps for mobile assistants.
- Qwen 2.5 Omni: A multilingual realtime model supporting tool calling and voice generation, suited for Chinese‑dominant scenarios.
- Moshi: An open‑source realtime model focused on voice understanding and generation, intended as a benchmark for comparisons.
- Freeze Omni: An efficient realtime model emphasizing low resource consumption and multi‑step interactions.
All these models adopt end‑to‑end architectures to avoid traditional pipeline latency, but they emphasize different priorities: PersonaPlex and Moshi focus more on open‑source experimentation, while gpt‑realtime and Gemini Live prioritize production‑grade deployment.
Open‑source status
- PersonaPlex‑7B‑v1: Fully open‑source including weights, code and training logs; available on Hugging Face.
- Grok Voice Agent API: Partially open‑source, with API sample code provided but core weights closed.
- gpt‑realtime: Closed‑source, accessible only via API.
- Gemini Live: Closed‑source, integrated into Google's ecosystem.
- Qwen 2.5 Omni: Partially open‑source; model weights are available while the API is closed.
- Moshi: Fully open‑source and community‑modifiable.
- Freeze Omni: Open‑source and suitable for local deployment.
Openness varies significantly: NVIDIA and Moshi promote community innovation, while OpenAI and Google prioritize commercial protection.
Performance metrics
Performance is based on FullDuplexBench, Big Bench Audio and service scenario benchmarks, evaluated using GPT‑4o.
- Dialogue dynamics (average success rate, %): PersonaPlex 82.1, Grok 92.0, gpt‑realtime 90.0, Gemini Live 88.5, Qwen 2.5 Omni 85.0, Moshi 65.3, Freeze Omni 54.7.
- Latency (average, seconds): PersonaPlex 0.205, Grok 0.78, gpt‑realtime 0.5–1.0, Gemini Live 0.68, Qwen 2.5 Omni 1.0, Moshi 0.261, Freeze Omni 1.181.
- Task adherence (GPT‑4o score, 0–5): PersonaPlex 4.34, Grok 4.5, gpt‑realtime 4.6, Gemini Live 3.68, Qwen 2.5 Omni 4.0, Moshi 1.26, Freeze Omni 3.82.
- Other: Big Bench Audio scores: Grok 92.3, gpt‑realtime 91.8; WER (word error rate): gpt‑realtime <5%; multilingual support: Qwen and Grok lead.
PersonaPlex has the lowest latency among open‑source models, gpt‑realtime is strongest at multimodal tasks, and Grok stands out for responsiveness. Moshi and Freeze Omni serve as baselines with lower performance but higher resource efficiency.
How to use
- PersonaPlex: Run locally and load the model via the Hugging Face library. Example: from transformers import pipeline; pipe = pipeline('speech‑to‑speech', model='nvidia/personaplex‑7b').
- Grok Voice Agent API: Integrate via WebSocket. Python example: connect to wss://api.x.ai/voice‑agent and send audio streams.
- gpt‑realtime: Use the OpenAI API. Example: client.audio.speech.create(model='gpt‑realtime', input='text'), supports streaming.
- Gemini Live: Use Google Vertex AI API. Example: call realtime endpoints with the gemini‑2.0‑flash‑live‑001 model.
- Qwen 2.5 Omni: Available via Hugging Face or API. Example: load qwen/qwen‑2.5‑omni for voice inference.
- Moshi: Run open‑source code from GitHub with local Python scripts to process audio.
- Freeze Omni: Similar to Moshi; local deployment supports custom scripts.
All models support audio stream processing; FFmpeg is recommended as an auxiliary tool. API‑based models require API keys, while local open‑source models do not.
Cost analysis
- PersonaPlex: Free and open‑source; local deployment incurs no API fees.
- Grok Voice Agent API: Pay‑as‑you‑go at $0.0005–0.001 per second with a monthly free tier of 1,000 seconds.
- gpt‑realtime: $0.005 per 1,000 input tokens + $0.015 per 1,000 output tokens + $0.001 per second for voice; Plus subscription $20/month offers unlimited access.
- Gemini Live: $0.68/hour (based on 45k audio tokens), $3 per 1M input audio + $12 per 1M output.
- Qwen 2.5 Omni: API $0.001–0.002 per second; open‑source local use is free.
- Moshi/Freeze Omni: Open‑source and free with no API fees.
Open‑source models like PersonaPlex are the most economical; among API models Gemini Live's hourly rate is the lowest (~$0.68 USD/hour), while gpt‑realtime suits subscribers.
Applications and impact
These models expand AI applications: PersonaPlex and Moshi are used for open‑source experimentation; Grok and gpt‑realtime lead in realtime translation and customer service; Gemini Live's multimodality is well suited to mobile use. Impacts include privacy challenges and energy consumption, while driving progress toward AGI. Community feedback indicates open‑source models foster innovation, whereas closed‑source offerings provide stability.
Official demo audio
1. NVIDIA PersonaPlex‑7B‑v1
- Official source: NVIDIA Research Labs project page (https://research.nvidia.com/labs/adlr/personaplex).
- Demo audio:
- Example 1: Assistant Demo Audio – demonstrates general knowledge, interruptions and natural transitions (left channel user, right channel PersonaPlex).
- Example 2: Customer Service ‑ Banking Demo Audio – banking customer service role‑play demo.
- Example 3: Customer Service ‑ Restaurant Demo Audio – restaurant customer service demo.
- Description: These official audio files can be played or downloaded from NVIDIA's page and showcase the model's realtime speech‑to‑speech capability. A YouTube video demo (https://www.youtube.com/watch?v=5_mOTtWouCk) contains additional realtime audio tests.
2. xAI Grok Voice Agent API
- Official source: xAI developer docs (https://docs.x.ai/docs/guides/voice) and xAI news page (https://x.ai/news/grok‑voice‑agent‑api).
- Demo audio:
- Official Voice Playground: https://console.x.ai/team/default/voice – browser interactive demo supporting realtime voice input and output (requires xAI account login, free trial available).
- YouTube official demo: Grok Voice API Demo with LiveKit – shows emotion understanding and realtime dialogue audio.
- Description: The Playground allows direct listening to Grok's voices and supports multilingual and tool‑calling demos.
3. OpenAI gpt‑realtime (based on GPT‑4o)
- Official source: OpenAI blog (https://openai.com/index/introducing‑gpt‑realtime) and API docs (https://platform.openai.com/docs/guides/text‑to‑speech).
- Demo audio:
- Official voice sample: Voice Sample ‑ Marin – static audio sample for the new voice Marin.
- Official voice sample: Voice Sample ‑ Cedar – static audio sample for the new voice Cedar.
- YouTube official demo: Live Demo of GPT‑4o Realtime Conversational Speech – realtime dialogue audio demo.
- Interactive demo: OpenAI.fm – official interactive TTS demo supporting multiple voices (platform access required).
- Another demo: Live Demo of GPT‑4o Realtime Translation – realtime translation audio.
- Description: These samples showcase gpt‑realtime's natural expression and multimodal capabilities, supporting six preset voices.
4. Google Gemini Live
- Official source: Google AI developer docs (https://ai.google.dev/gemini‑api/docs/live) and the Gemini overview page (https://gemini.google/overview/gemini‑live).
- Demo audio:
- YouTube official demo: Gemini Live, Realtime Audio ‑ Voices & Demo – showcases voices and realtime interaction audio.
- Another demo: How to Use Gemini Live API Native Audio – step‑by‑step video including audio samples.
- Interactive demo: Google AI Studio Live Demo – browser realtime listening (requires Google account and supports voice input).
- Description: These demos highlight Gemini Live's low latency and multimodal responses, supporting native audio.
5. Qwen 2.5 Omni
- Official source: Qwen GitHub (https://github.com/QwenLM/Qwen2.5‑Omni) and the Qwen blog (https://qwenlm.github.io/blog/qwen2.5‑omni).
- Demo audio:
- Hugging Face Spaces demo: Qwen2.5 Omni 7B Demo – interactive demo supporting audio input and realtime voice output.
- Modelscope Studio demo: Qwen2.5 Omni Demo – another interactive voice demo platform.
- YouTube official demo: Qwen 2.5 Omni Demo – shows multimodal input and realtime audio output.
- Description: Demos support text, image, audio and video inputs and generate realtime voice.
6. Moshi
- Official source: Kyutai Labs page (https://moshi.chat/) and GitHub (https://github.com/kyutai‑labs/moshi).
- Demo audio:
- Official interactive demo: Moshi Chat Demo – browser realtime voice chat (limited to 5 minutes, supports interruptions and role‑play).
- YouTube official demo: Moshi Demo – demonstrates realtime multimodal audio.
- Another demo: Moshi Presentation Video – audio demos related to the paper.
- Description: Moshi supports realtime interaction that can think and speak concurrently.
7. Freeze Omni
- Official source: VITA‑MLLM project page (https://freeze‑omni.github.io/) and GitHub (https://github.com/VITA‑MLLM/Freeze‑Omni).
- Demo audio:
- Example 1: Demo on Mandarin Question Answering – Chinese QA audio demo.
- Example 2: Demo on English Question Answering – English QA audio demo.
- Example 3: Demo on Prompt Following and Role Play – prompt‑following and role‑play audio.
- Server demo: run the local demo via GitHub scripts (installation required), supporting realtime voice listening.
- Description: These are static audio files showcasing Freeze Omni's low‑latency speech‑to‑speech capability.
If any links are inaccessible or require specific permissions, please check browser settings or use a VPN. Some demos may be restricted by geographic location (you are in Auckland, NZ), but these are all official sources.
Conclusion
Competition among realtime LLMs is intense: PersonaPlex is open‑source and efficient, Grok excels in speed, gpt‑realtime is comprehensive in multimodality, Gemini Live is cost‑effective, Qwen Omni is strong in multilingual support, and Moshi/Freeze Omni serve as baselines. Choice depends on needs: for open‑source priorities choose PersonaPlex; for production‑grade use choose gpt‑realtime or Gemini. Looking ahead, multimodal integration will further optimize performance.
▶️ Like this article? Creating it took real effort!
Follow our YouTube channel:
AI Rosetta
Your likes, subscriptions and shares are our greatest support! 🙏
More AI news at: https://www.airosetta.com