Search “Speech”

How to integrate the ElevenLabs text-to-speech API into an app

ElevenLabs has published a tutorial on how to integrate the text-to-speech API into an app using Python and TypeScript: create an API key, install the ElevenLabs SDK, send text to the TTS endpoint and specify a voice and model, then play the returned audio. The example uses the eleven_v4 model, and the low-latency Eleven Flash v2.5 is also available; the Voice Library offers more than 17,500 voices, and output defaults to mp3 but can be switched to ulaw or pcm.

How Acuity Health Transformed Patient Services with LiveKit Voice Agents

Acuity Health built a custom voice agent on LiveKit: 80% of incoming calls are fully handled by the AI agent, most appointments are completed within 1 minute, and 81% of appointment calls end with "thank you." Its agent typically runs on a Gemma 3 27B model optimized on LiveKit Inference, paired with turn-detection models tuned for different patient populations, supports voice and text, and connects to call center tools, task management software, analytics providers, and EMR systems. The team had previously been constrained by low-code platforms; after switching to LiveKit, they built a working prototype within a week, reached a development environment within a month, and use LiveKit Simulations to stress-test scenarios before every change goes live.

English translation is being prepared.

Ethan Mollick: Voice interfaces may put an end to the open office

Ethan Mollick believes that voice interfaces in AI applications may finally put an end to the "monster" that is the open office—or may simply make it worse. Citing his own research, he points out that multiple experiments show that background noise in open offices reduces the ability to do analytical work, increases software bugs, and weakens the ability to find bugs.

ElevenLabs launches Eleven Realtime Translation real-time speech translation model

ElevenLabs releases the real-time speech translation model Eleven Realtime Translation, currently available in alpha form to ElevenAPI enterprise customers. The model uses the low-latency speech model Eleven v4 Turbo by default, supports more than 90 languages and automatic language detection, keeps the same voice consistent across languages, and can be selected from a library of more than 17,500 voices or custom voice clones. Teams can adjust the translation startup speed, trading off between high-quality translation and low latency on complex calls; Deutsche Telekom has already built new call features based on ElevenLabs in its Magenta AI Call Assistant.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

Mistral's Voxtral 3 launches on OpenRouter, including two speech transcription models and one speech synthesis model

OpenRouter announced that Mistral's Voxtral 3 is now available. The series includes two new speech transcription models and one new speech synthesis model, covering more than 50 languages. Mistral officially states that Voxtral 3's Transcribe and Speech models power its new voice agents API.

English translation is being prepared.

Mistral Releases Voxtral 3 Speech Model with Real-Time Speaker Diarization and Turn Detection

Mistral has released Voxtral 3, comprising the Transcribe and Speech models, to power its new voice agents API. Co-founder Arthur Mensch added that Voxtral can perform speaker diarization and end-of-turn detection on the fly, and can be deployed either on your own infrastructure or via Mistral's cloud service.

HeyGen Professional Voice Clone: Trained on 30 Minutes to 2 Hours of Audio

HeyGen's Professional Voice Clone is trained on 30 minutes to 2 hours of recorded audio, preserving breaths, pauses, laughter, accents, and vowel pronunciation. The author believes this amount of data is key: short reference clips cannot capture the rhythm of a breath before a long sentence or the accent features that emerge when speech speeds up, and these subtle irregularities are exactly what listeners use to judge whether a voice belongs to a real person. HeyGen claims no one can tell the clone apart from you.

Mistral releases the Voxtral 3 speech model series, including TTS and three transcription models

Mistral releases the Voxtral 3 speech model series, including one text-to-speech model and three speech-to-text models (Small Transcribe 3, Mini Transcribe 3, Mini Transcribe Realtime 3). Artificial Analysis evaluations show that Voxtral Small Transcribe 3 ranks 3rd on the speech-to-text leaderboard with a 2.1% AA-WER Index, ahead of ElevenLabs Scribe v2 at 2.2% and Grok Voice Transcribe 2.0 at 2.3%; Voxtral 3 TTS ranks 12th (out of 98) on the Provider Voice leaderboard with an Elo of 1,218, priced at $13/1M characters.

ElevenLabs Launches Eleven Realtime Hosted Voice Agents, Requiring Developers to Give Up Control of the Underlying Model

ElevenLabs has released Eleven Realtime, offering two hosted voice agent models, Mini and Max, which integrate speech recognition, reasoning, turn-taking, and speech generation, but developers cannot individually select or configure the underlying models. Data from external testing organization Artificial Analysis shows that Max scored 98.8% on Big Bench Audio, while Mini scored 75.9%; for tool-calling task completion, Max reached 87.1% and Mini 60.3%, while Mini slightly outperformed Max on Full-Duplex-Bench with 95.8% versus 94.1%. The interface follows the OpenAI Realtime API protocol and WebSocket transport, but only supports WebSocket; WebRTC applications must first switch transport methods and handle the capture and playback of 24kHz PCM16 audio themselves.

English translation is being prepared.

ElevenLabs Releases Two Real-Time Speech-to-Speech Models, Eleven Realtime

ElevenLabs has released two speech-to-speech models, Eleven Realtime, among which v1 Max scores 98.8% on Big Bench Audio with a response time of 0.57 seconds. The smaller v1 Mini targets low-cost natural conversation and can tell whether the other party has finished speaking or only said "mm-hmm," while v1 Max targets more complex tasks such as customer service. Both models are available through the ElevenAgents platform and the OpenAI-compatible Realtime API, billed per minute plus tokens, with v1 Max priced higher than v1 Mini.

Mistral releases Voxtral 3 transcription and speech models, powering the speech agent API

Mistral releases Voxtral 3, comprising two model types, Transcribe and Speech, which the company calls the current state of the art. These two model types are used to power Mistral's new speech agent API.

Sierra launches 𝜏-i18n multilingual voice agent evaluation benchmark

Sierra releases 𝜏-i18n, extending 𝜏-Voice to five languages: Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin, retaining the same airline, retail, and telecom tasks, policies, tools, and success criteria to enable cross-language comparison. Across 2,700 main calls, task completion rates for Spanish, Hindi, and Portuguese lag English by an average of 2.4, 3.8, and 6.4 percentage points, while Korean and Mandarin lag by 22.6 and 9.1 percentage points respectively; Korean is the weakest language among the three agents. In addition to task completion, 𝜏-i18n separately reports interaction metrics (response timing, selectivity, tool-call efficiency) and generation quality (naturalness, speech fidelity), and has open-sourced the codebase and the language pack framework LanguageFactory.

Vapi Hosts Fleet Week Yacht Event: Voice AI Builders Discuss GPT Live and Natural Interaction

On October 8, Vapi chartered a yacht in San Francisco Bay to host a Fleet Week event, with more than 30 voice AI builders in attendance; Deepgram, Kouper Health, and Coval led fireside chats and technical workshops. The most talked-about topic on board was GPT Live: skipping traditional transcription/synthesis can reduce latency and make conversational turn-taking smoother, but it comes at the cost of fine-grained control over key terms and scripts. The focus of discussion has shifted from "whether voice AI can converse" to "how to make conversation more natural," specifically including backchanneling, perceived latency, and measuring naturalness.

English translation is being prepared.