Contents
- What Is AI Voice Cloning, Exactly?
- Two Approaches: Text-to-Speech Cloning vs. Voice Conversion
- How Does the Technology Actually Work?
- Instant Cloning vs. Professional Cloning: What's the Difference?
- How Much Audio Does It Actually Take?
- What Legitimate Businesses Use Voice Cloning For
- The Scam Side Most Explainers Skip
- Can You Actually Tell If a Voice Has Been Cloned?
- FAQ
AI voice cloning uses machine learning to build a digital copy of one specific person’s voice — not a generic computer voice. The system studies real audio of someone speaking and learns their tone, pitch, accent, cadence, and even breathing patterns. Once trained, it can generate brand-new speech, words that person never said, in a voice that still sounds like theirs.
What we checked ourselves
The figures below were not taken from other articles. We checked them ourselves on 2026-09-14 00:14 KST.
We called the homepages of 12 services commonly listed as 'the best AI voice cloning tools' directly today, and found that two of them (Play.ht, LOVO.ai) were completely down.
Method: sent a direct request to each service's homepage with curl -o /dev/null -s -w '%{http_code}' -L. For any that returned no response, we double-checked by looking up DNS resolution with dig/nslookup and retrying with a browser User-Agent attached. We also captured response headers (curl -sD -) to pin down the cause. Checked: 2026-09-14 00:13–00:14 KST.
| Service | URL | HTTP status | Notes |
|---|---|---|---|
| ElevenLabs | elevenlabs.io | 200 | Normal |
| Resemble AI | resemble.ai | 200 | Normal (see F2 below, though) |
| Play.ht | play.ht, playht.com | Connection failed (DNS did not resolve) | `dig play.ht` returned no response; based on the exit code, curl could not reach either domain at all |
| Murf | murf.ai | 200 | Normal (though the page is JS-rendered, so the pricing text could not be scraped automatically) |
| Descript | descript.com | 200 | Normal |
| WellSaid Labs | wellsaidlabs.com → wellsaid.io | 301→200 | Domain has moved to wellsaid.io |
| Respeecher | respeecher.com | 200 | Normal |
| Speechify | speechify.com | 200 | Normal |
| Podcastle | podcastle.ai | 200 | Normal |
| LOVO.ai | lovo.ai | 402 (Payment Required) | Response header showed `x-vercel-error: DEPLOYMENT_DISABLED` — the Vercel deployment had been suspended over a payment issue |
| Synthesys | synthesys.io | 200 | Normal |
| Voicemod | voicemod.net | 200 | Normal |
Caveat: the sample is 12 services chosen arbitrarily by the author, not an 'industry-standard list'. Even if a server is up (200), whether the page actually offers voice cloning functionality needs separate verification (see F2). Results may also vary by region (Korean IP) due to geo-blocking.
Resemble AI and WellSaid Labs, once introduced as 'voice cloning tools', do not list a voice cloning feature anywhere on their pricing pages as of today.
Method: opened each company's official homepage and /pricing page directly and compared the text. Checked resemble.ai's main headline and the feature list for each plan on wellsaid.io/pricing as they stood. Checked: 2026-09-14 00:14 KST.
| Company | What appears on today's homepage/pricing page | Mention of voice cloning |
|---|---|---|
| Resemble AI | Main headline "Deepfakes are everywhere. So are we." — deepfake detection, PerTh watermarking, and voice identity registration/authentication are placed front and centre. The pricing table (Flex $0 / Team $350 / Business $1,000 / Enterprise custom) has no cloning line item | None (the TTS product has been pushed down under "Voice AI Research") |
| WellSaid Labs (wellsaid.io) | Trial $0 · Starter $10-19/month · Pro $33-49/month · Business $160/user/month — every tier's description only mentions "hundreds of realistic AI voices" | None ("custom voice cloning" does not appear on any plan) |
| ElevenLabs (control) | Instant/Professional voice cloning explicitly stated from Creator $11/month (121,000 credits) upward | Present |
| Descript (control) | Limited "Overdub (voice cloning)" included from Hobbyist $16/month, full custom voice cloning included from Creator $24/month | Present |
Caveat: the absence of a feature from the pricing page does not rule out the possibility that it is still offered under a separate enterprise contract (both companies have an 'Enterprise: Custom' option). Page content can change at any time, so the check timestamp is the valid-until point.
What Is AI Voice Cloning, Exactly?
That’s the short version. Here’s the part worth sitting with: cloning splits a voice into two separate things. One is “who is speaking” — the timbre, pitch, and rhythm that make a voice recognizable as belonging to one person. The other is “what is being said” — the words themselves. A cloning system trains a model to capture the first part, then pairs it with a second model that generates new speech by combining that identity with new text.
This is what separates cloning from ordinary text-to-speech. Regular TTS reads text aloud in a stock synthetic voice — nobody’s voice in particular. Voice cloning goes after one real person’s specific vocal identity and reproduces it.
Personal take: that “who” versus “what” split is the single idea that makes the rest of this topic click. Everything below builds on it.
Two Approaches: Text-to-Speech Cloning vs. Voice Conversion
Two technical methods get lumped under “voice cloning,” and they’re not the same thing.
| Approach | What goes in | What comes out |
|---|---|---|
| Text-to-speech (TTS) cloning | Typed text | New speech, generated from scratch, in the cloned voice |
| Voice conversion | Existing spoken audio from a different speaker | The same words and timing, but re-voiced |
TTS cloning generates speech from typed text in the cloned voice. Voice conversion instead transforms audio that already exists — it keeps the original content and timing, and swaps who it sounds like it’s coming from. When people say “voice cloning,” they usually mean the TTS kind. Dubbing work leans more on voice conversion, since the timing has to match a performance that’s already been recorded.
How Does the Technology Actually Work?
The pipeline ElevenLabs and Resemble AI describe collapses into roughly the same steps, whether it runs in four stages or seven:
- Audio data collection — gathering recordings of the target voice
- Audio cleanup — removing noise, trimming silence, normalizing levels
- Feature extraction and model training — a “voice encoder” extracts the speaker’s vocal identity and encodes it numerically
- Speech synthesis — the trained identity is combined with new text or audio to generate output
- Optional fine-tuning — further training to sharpen accuracy
Under the hood, this runs on deep learning and neural networks. Resemble AI’s technical rundown names specific model families doing the work: TTS models like Tacotron 2, FastSpeech 2, and VALL-E (VALL-E is specifically noted for zero-shot cloning from just seconds of audio); vocoders like WaveNet, WaveGlow, and HiFi-GAN, which turn spectrograms into raw audio; GAN-based voice conversion tools like CycleGAN-VC, along with VAEs and autoencoders; and speaker-embedding systems like Deep Speaker and SV2TTS.
None of that machinery is visible to a listener. What they hear is just a voice.
Instant Cloning vs. Professional Cloning: What’s the Difference?
ElevenLabs’ two product tiers make the practical distinction concrete. Instant Voice Cloning (IVC) is a few-shot method — the reference audio acts as a live conditioning signal, the model’s underlying weights don’t change, and there’s no training step involved. Quality varies depending on how good the reference audio is.
Professional Voice Cloning (PVC) works differently. It fine-tunes the actual model parameters using the samples provided, which produces more consistent output. It also requires a paid Creator-tier plan or higher — this isn’t a free-tier feature.
That paywall is the kind of detail other guides leave out. If you’re evaluating a tool based on quality, the tier you’re on matters as much as the tool itself.
How Much Audio Does It Actually Take?
This is the number people actually search for, and it depends on the source and the quality bar you’re aiming for.
| Source | Minimum audio | Notes |
|---|---|---|
| ElevenLabs (general) | Under a minute | Quality keeps improving up to roughly 2–3 hours, then plateaus |
| ElevenLabs IVC | Under 2 minutes | Few-shot, no training step |
| ElevenLabs PVC | Approximately 30 minutes | Fine-tunes the model; paid tier required |
| Resemble AI IVC | A few minutes | Varied intonation and emotion in samples improves quality |
One fraud-focused source cites third-party figures — attributed to a 2025 McAfee report — claiming as little as 3 seconds of audio can produce a partial voice match. That number is secondhand and unconfirmed, so treat it with caution rather than as settled fact.
Taken together, the verified sources agree on a rough shape: usable cloning can start with well under a minute of clean audio, and professional-grade results want tens of minutes to a few hours.
What Legitimate Businesses Use Voice Cloning For
The use cases aren’t exotic. They’re mostly things already happening in media and customer service:
- Audiobook narration and podcast or YouTube voiceovers
- Film and video dubbing into other languages while preserving the original speaker’s vocal characteristics
- Accessibility — restoring speech for people with degenerative conditions or voice loss, such as ALS, throat cancer, or laryngectomy, using their own banked voice samples
- Multilingual e-learning and education content
- Customer support voice agents and personalized voice assistants
- Gaming NPC voices, music and vocal synthesis, and historical voice preservation
Legitimate use generally requires consent from the person whose voice is being cloned, and commercial releases typically need to follow whatever disclosure rules the distribution platform sets.
The Scam Side Most Explainers Skip
Most writeups on this topic stop at “here’s how the technology works” and treat the fraud angle as a footnote. That’s backwards.
The best-known pattern: a caller clones a family member’s or executive’s voice, then fabricates an emergency or urgent-payment scenario to pressure the target into sending money. The FTC — the US federal consumer-protection agency — formally warned consumers about exactly this, describing it as one of criminals’ newer AI-enabled tactics. In its own words: “All he needs is a short audio clip of your family member’s voice — which he could get from content posted online — and a voice-cloning program.”
Reporting has documented real losses from this pattern: one Florida woman lost $15,000, and another case nearly reached a $35,000 loss before it was stopped. A separate survey, cited in a May 2026 report, found that roughly one in four respondents said they’d experienced this kind of scam directly or knew someone who had — though that survey wasn’t checked directly, so take the figure with a grain of salt.
The FTC flags three payment channels as red flags specifically because they’re hard to reverse once money moves: wire transfers, cryptocurrency, and gift cards. Industry blog posts also cite broader fraud-loss totals — hundreds of millions to billions of dollars across 2024 and 2025, sourced from FBI, Vectra AI, and CrowdStrike reporting. Those primary reports weren’t checked directly, so read the totals as a rough indication, not a precise count.
My take: the barrier to entry for this dropped a lot faster than people’s guard did. A tool built for audiobook narration and a tool used in a fake-emergency phone scam are, technically, close to the same thing.
Can You Actually Tell If a Voice Has Been Cloned?
Security researchers agree on this point: ordinary listeners struggle to reliably tell a cloned voice from a real one by ear alone. One security-focused source states plainly that no consumer app or browser extension reliably catches a cloned voice in real time — the defenses that actually work are procedural, not technical.
The FTC’s own recommended response isn’t a detection method at all. It’s a verification habit: don’t trust the voice alone. Call the person back on a number you already know is theirs, or reach them through another family member or friend.
This piece isn’t turning that into a step-by-step scam checklist — that’s a separate topic worth its own space. But the core fact is worth remembering on its own: if you can’t verify who’s actually on the line, the voice itself isn’t proof.
FAQ
What’s the difference between voice cloning and regular text-to-speech? Regular TTS reads text aloud in a generic synthetic voice that doesn’t belong to anyone in particular. Voice cloning specifically targets one real person’s voice and reproduces their tone, pitch, and cadence.
How much audio do you need to clone a voice? It depends on the tool and the quality bar. ElevenLabs says its instant cloning works with under 2 minutes of audio, while its professional tier wants around 30 minutes for consistent results. Some fraud-focused reports claim as little as 3 seconds can produce a partial match, but that figure hasn’t been independently verified here.
Can you tell if a phone call is using a cloned voice? Not reliably. Security researchers say ordinary listeners can’t reliably distinguish a cloned voice from a real one by ear, and no consumer app is confirmed to catch this in real time either. The FTC’s own advice is to verify by calling the person back on a known number rather than trying to detect it by listening.
What’s still unclear is how fast the audio-length requirement keeps shrinking — the gap between “well under a minute” and the unverified “3 seconds” claim is exactly where the next round of scam reporting is likely to focus. Whether the industry-wide fraud-loss figures hold up once the primary reports get properly checked is also an open question. Worth watching rather than assuming either way.
Image credits
- Photo by Robinson Recalde on Unsplash
- Photo by John Taran on Pexels
- Photo by Pexels on Pixabay