ไทย

PaperEvaluation, Speech, Thai language

How clearly do voices read Thai tip alerts? A blind test of 30 voices

60 unseen alerts, two speech recognizers, 2,000 resamples: where Tipjai Voice 1 stands against 29 voices from 10 companies, and where it still loses.

Published
Authors
Tipjai Research

Listen to the summary0:00

Read in Thai by Tipjai Voice, our synthetic voice in preview. It may misread a word.

An anechoic chamber at NASA Langley: terracotta foam wedges on every wall, a white arch of microphones around a black test cylinder.
An anechoic chamber at NASA Langley Research Center, 10 April 2025.Image: NASA Licence

Summary

  • Tipjai Voice 1 was heard wrong on 6.0% of characters across 60 alerts no voice had seen: 8th of 30 voices, with a 95% interval of 4.6% to 7.5%.
  • Compared pair by pair on the same alerts, 0 voices were clearly clearer than Tipjai Voice 1, it was clearly clearer than 7, and 22 were too close to call.
  • It costs ฿0 per alert and takes 1.38 s per alert on a laptop CPU. The voices heard wrong less often cost ฿35 to ฿168 per 1,000 alerts.
  • It loses on names: 26th of 30 for hearing the supporter's name right.
  • On naturalness, one blind listener gave it 3.17 out of 5; the best voice scored 5.00. That gap is Tipjai Voice 2's main job.

The task

On a Thai live stream, every tip is read aloud the moment it arrives. An alert has three parts: the supporter's name, often a gaming handle written in Latin letters; the amount in baht; and a short message full of slang, 555, stretched letters and emoji.

The voice has to get it right the first time, live, with no second take. So the question here is narrow and measurable: when you listen, how much of what you hear matches what you should have heard?

A neon ON AIR sign built around a microphone outline, in white, pink-red, yellow and violet, in a dim room.
On air. Tip alerts are read aloud, live, in the middle of a stream.Photo: Meruyert Gonullu / Pexels Licence

The test set

The test set has 60 alerts, written separately and frozen before any voice was scored. Tipjai Voice 1 was frozen before the set was opened, and nothing was tuned after seeing results. 33 alerts have a supporter name in Latin letters and 27 in Thai; the average alert is 50 characters. No real supporter message is in the set.

The set has now been used once, for these published results, so it is released in full under Reproduce it. The next voice is measured on new sets no one has seen.

The voices

30 voices were tested: Tipjai Voice 1 and 29 voices from 10 companies. Each vendor's audio came from its own service on 5 and 6 October 2026. Every voice read all 60 alerts, except iApp ChindaTTS Kaitom, which read 39; it is left off the board, because a mean over other alerts cannot be ranked with the rest.

Prices are list prices before VAT, converted at 33.695 baht per dollar (2026-10-05, European Central Bank reference rate via api.frankfurter.dev). No vendor's audio has ever been used to train or tune Tipjai Voice.

Scoring

Two Thai speech recognizers listen to every clip: Thonburian Whisper (biodatlab/whisper-th-medium-combined) and Typhoon Whisper large-v3 (typhoon-ai). The score is the character error rate: the share of characters heard wrong, after every accepted reading of a name or loanword is mapped to one spelling. Digits, the repeat mark ๆ and stretched letters are normalised before counting. Each voice's score is the mean of the two recognizers.

95% intervals come from 2,000 bootstrap resamples of the alerts. A rank range is the 2.5th to 97.5th percentile of a voice's rank across those resamples. Paired comparisons resample the same alerts for both voices, 4,000 times (seed 6); when the interval of the difference crosses zero, we call the pair too close to call.

Results

Figure All 30 voices on one ruler. Tap a name to hear it read the same alert.

Dots are the share heard wrong; lines are 95% intervals. The shaded band is Tipjai Voice 1's interval: a voice whose line meets it cannot be told apart from ours on these alerts. Every clip is matched to -20 LUFS.

Figure Tipjai Voice 1 is clearly clearer than 7 voices; none is clearly clearer than it; 22 are too close to call.
  1. Gemini 3.8 Flash TTS Kore+0.7 [−1.3, +2.7] too close to call
  2. ElevenLabs v4 Sarah+0.7 [−1.8, +2.8] too close to call
  3. ElevenLabs v4 Charlie+0.6 [−1.4, +2.4] too close to call
  4. Cartesia Sonic 3.6 Somchai+0.4 [−1.8, +2.4] too close to call
  5. ElevenLabs v4 Turbo Sarah+0.2 [−2.3, +2.4] too close to call
  6. MiniMax Speech 2.8 HD male 1+0.2 [−1.8, +2.0] too close to call
  7. Cartesia Sonic 3.6 Supannee+0.2 [−2.6, +2.5] too close to call
  8. Gemini 3.8 Flash Lite TTS Kore †−0.2 [−2.0, +1.6] too close to call
  9. Google Neural2-C−0.2 [−2.6, +1.9] too close to call
  10. Google Standard-A−0.2 [−2.6, +1.9] too close to call
  11. Cartesia Sonic 3.6 Kanya−0.3 [−3.2, +2.2] too close to call
  12. Grok Luna−0.3 [−3.6, +2.5] too close to call
  13. Microsoft MAI-Voice-2.1 Krit−0.3 [−3.1, +2.0] too close to call
  14. Grok Ara−0.3 [−3.9, +2.5] too close to call
  15. ElevenLabs v3 Conversational Sarah−0.8 [−3.5, +1.6] too close to call
  16. MiniMax Speech 2.8 HD female 1−0.8 [−4.0, +1.8] too close to call
  17. Microsoft MAI-Voice-2.1 Flash Harper−1.1 [−3.9, +1.6] too close to call
  18. Google Chirp 3 HD Kore−1.1 [−4.5, +1.7] too close to call
  19. Google Chirp 3 HD Charon−1.5 [−4.9, +1.3] too close to call
  20. Qwen Audio 3.0 TTS Flash, loongjohn−1.7 [−4.3, +0.5] too close to call
  21. Microsoft Premwadee−2.0 [−4.5, +0.3] too close to call
  22. ElevenLabs v3 Sarah−2.2 [−5.3, +0.5] too close to call
  23. Qwen Audio 3.0 TTS Plus, longanlingxin−2.5 [−5.0, −0.3] Tipjai Voice 1 clearly clearer
  24. Inworld TTS-2 Dennis−2.7 [−5.1, −0.5] Tipjai Voice 1 clearly clearer
  25. Microsoft Niwat−3.2 [−5.3, −1.1] Tipjai Voice 1 clearly clearer
  26. Microsoft MAI-Voice-2.1 Harper−5.4 [−11.4, −0.5] Tipjai Voice 1 clearly clearer
  27. Botnoi 311 (Northern dialect)−6.4 [−8.5, −4.3] Tipjai Voice 1 clearly clearer
  28. Inworld TTS-2 Flash Dennis−10.8 [−13.3, −8.4] Tipjai Voice 1 clearly clearer
  29. Fish Audio S2.1 Pro, library voice−20.7 [−24.1, −17.5] Tipjai Voice 1 clearly clearer

Each dot is the difference in errors (Tipjai Voice 1 minus that voice, in percentage points) on the same alerts; each line is its 95% interval.

Figure The 7 voices heard wrong less often cost ฿35 to ฿168 per 1,000 alerts.
Gemini 3.8 Flash TTS KoreTipjai Voice 1Google Standard-A

List price per 1,000 alerts (log scale) against share heard wrong (clearest at the top)

No public list price, so not drawn: Botnoi 311 (Northern dialect) 12.4%

Price and share heard wrong
Gemini 3.8 Flash TTS Kore฿355.3%
ElevenLabs v4 Sarah฿1355.4%
ElevenLabs v4 Charlie฿1355.4%
Cartesia Sonic 3.6 Somchai฿845.7%
ElevenLabs v4 Turbo Sarah฿675.9%
MiniMax Speech 2.8 HD male 1฿1685.9%
Cartesia Sonic 3.6 Supannee฿845.9%
Tipjai Voice 1฿06.0%
Gemini 3.8 Flash Lite TTS Kore †฿246.2%
Google Neural2-C฿276.3%
Google Standard-A฿6.76.3%
Cartesia Sonic 3.6 Kanya฿846.3%
Grok Luna฿256.3%
Microsoft MAI-Voice-2.1 Krit฿376.4%
Grok Ara฿256.4%
ElevenLabs v3 Conversational Sarah฿676.8%
MiniMax Speech 2.8 HD female 1฿1686.9%
Microsoft MAI-Voice-2.1 Flash Harper฿257.1%
Google Chirp 3 HD Kore฿517.2%
Google Chirp 3 HD Charon฿517.5%
Qwen Audio 3.0 TTS Flash, loongjohn฿257.8%
Microsoft Premwadee฿258.0%
ElevenLabs v3 Sarah฿1358.2%
Qwen Audio 3.0 TTS Plus, longanlingxin฿348.5%
Inworld TTS-2 Dennis฿428.8%
Microsoft Niwat฿259.2%
Microsoft MAI-Voice-2.1 Harper฿3711.5%
Inworld TTS-2 Flash Dennis฿2516.8%
Fish Audio S2.1 Pro, library voice฿6726.7%

List prices before VAT, 50 characters per alert. Vertical lines are 95% intervals. Labelled: Tipjai Voice 1, the clearest voice and the cheapest.

Names

Names are Tipjai Voice 1's clearest weakness: the name was heard right in 29 and 30 of 60 alerts by the first and second recognizer, 26th of 30. It got 10.5 of 33 Latin-script names (median of all voices 13.75) and 19 of 27 Thai-script names (median 24.5).

Figure Every name, every voice. A full square: both recognizers heard the name right.
  1. Microsoft MAI-Voice-2.1 Harper44/47
  2. Grok Luna41/45
  3. Microsoft MAI-Voice-2.1 Flash Harper42/43
  4. Gemini 3.8 Flash TTS Kore41/43
  5. Grok Ara40/44
  6. Cartesia Sonic 3.6 Somchai40/43
  7. Cartesia Sonic 3.6 Kanya39/44
  8. Cartesia Sonic 3.6 Supannee39/43
  9. MiniMax Speech 2.8 HD female 139/42
  10. ElevenLabs v4 Sarah41/39
  11. MiniMax Speech 2.8 HD male 139/41
  12. Google Neural2-C38/41
  13. Google Standard-A39/40
  14. ElevenLabs v4 Turbo Sarah39/39
  15. Google Chirp 3 HD Kore38/40
  16. Microsoft MAI-Voice-2.1 Krit35/42
  17. Google Chirp 3 HD Charon37/40
  18. Qwen Audio 3.0 TTS Flash, loongjohn39/36
  19. ElevenLabs v4 Charlie38/36
  20. Gemini 3.8 Flash Lite TTS Kore †35/36
  21. ElevenLabs v3 Conversational Sarah33/36
  22. Inworld TTS-2 Dennis35/31
  23. ElevenLabs v3 Sarah32/33
  24. Microsoft Premwadee32/31
  25. Qwen Audio 3.0 TTS Plus, longanlingxin30/32
  26. Tipjai Voice 129/30
  27. Microsoft Niwat30/29
  28. Fish Audio S2.1 Pro, library voice24/25
  29. Botnoi 311 (Northern dialect)19/18
  30. Inworld TTS-2 Flash Dennis17/18

One row per voice, most names right at the top; one square per alert. Half a square: one recognizer heard it right. Right column: recognizer 1 / 2.

Two rulers

Figure Change the ruler and the order moves: 5 voices shift 8 places or more.

Thonburian WhisperTyphoon Whisper large-v3

  1. ElevenLabs v4 Charlie1
  2. Gemini 3.8 Flash TTS Kore2
  3. ElevenLabs v4 Sarah3
  4. MiniMax Speech 2.8 HD male 14
  5. ElevenLabs v4 Turbo Sarah5
  6. Cartesia Sonic 3.6 Somchai6
  7. Grok Luna7
  8. Gemini 3.8 Flash Lite TTS Kore †8
  9. Google Standard-A9
  10. Tipjai Voice 110
  11. Google Neural2-C11
  12. Cartesia Sonic 3.6 Supannee12
  13. Microsoft MAI-Voice-2.1 Flash Harper13
  14. Cartesia Sonic 3.6 Kanya14
  15. Microsoft MAI-Voice-2.1 Krit15
  16. Grok Ara16
  17. ElevenLabs v3 Conversational Sarah17
  18. MiniMax Speech 2.8 HD female 118
  19. Google Chirp 3 HD Kore19
  20. Qwen Audio 3.0 TTS Flash, loongjohn20
  21. Google Chirp 3 HD Charon21
  22. Qwen Audio 3.0 TTS Plus, longanlingxin22
  23. Inworld TTS-2 Dennis23
  24. ElevenLabs v3 Sarah24
  25. Microsoft Premwadee25
  26. Microsoft Niwat26
  27. Microsoft MAI-Voice-2.1 Harper27
  28. Botnoi 311 (Northern dialect)28
  29. Inworld TTS-2 Flash Dennis29
  30. Fish Audio S2.1 Pro, library voice30
  1. 1Cartesia Sonic 3.6 Somchai
  2. 2Cartesia Sonic 3.6 Supannee
  3. 3ElevenLabs v4 Sarah
  4. 4ElevenLabs v4 Turbo Sarah
  5. 5Tipjai Voice 1
  6. 6Gemini 3.8 Flash TTS Kore
  7. 7Grok Ara
  8. 8Cartesia Sonic 3.6 Kanya
  9. 9Microsoft MAI-Voice-2.1 Krit
  10. 10ElevenLabs v4 Charlie
  11. 11Google Neural2-C
  12. 12Gemini 3.8 Flash Lite TTS Kore †
  13. 13Google Standard-A
  14. 14MiniMax Speech 2.8 HD male 1
  15. 15Grok Luna
  16. 16MiniMax Speech 2.8 HD female 1
  17. 17ElevenLabs v3 Conversational Sarah
  18. 18Google Chirp 3 HD Kore
  19. 19Google Chirp 3 HD Charon
  20. 20Microsoft MAI-Voice-2.1 Flash Harper
  21. 21Microsoft Premwadee
  22. 22ElevenLabs v3 Sarah
  23. 23Qwen Audio 3.0 TTS Flash, loongjohn
  24. 24Microsoft Niwat
  25. 25Qwen Audio 3.0 TTS Plus, longanlingxin
  26. 26Inworld TTS-2 Dennis
  27. 27Microsoft MAI-Voice-2.1 Harper
  28. 28Botnoi 311 (Northern dialect)
  29. 29Inworld TTS-2 Flash Dennis
  30. 30Fish Audio S2.1 Pro, library voice
12
ElevenLabs v4 Charlie110
Gemini 3.8 Flash TTS Kore26
ElevenLabs v4 Sarah33
MiniMax Speech 2.8 HD male 1414
ElevenLabs v4 Turbo Sarah54
Cartesia Sonic 3.6 Somchai61
Grok Luna715
Gemini 3.8 Flash Lite TTS Kore †812
Google Standard-A913
Tipjai Voice 1105
Google Neural2-C1111
Cartesia Sonic 3.6 Supannee122
Microsoft MAI-Voice-2.1 Flash Harper1320
Cartesia Sonic 3.6 Kanya148
Microsoft MAI-Voice-2.1 Krit159
Grok Ara167
ElevenLabs v3 Conversational Sarah1717
MiniMax Speech 2.8 HD female 11816
Google Chirp 3 HD Kore1918
Qwen Audio 3.0 TTS Flash, loongjohn2023
Google Chirp 3 HD Charon2119
Qwen Audio 3.0 TTS Plus, longanlingxin2225
Inworld TTS-2 Dennis2326
ElevenLabs v3 Sarah2422
Microsoft Premwadee2521
Microsoft Niwat2624
Microsoft MAI-Voice-2.1 Harper2727
Botnoi 311 (Northern dialect)2828
Inworld TTS-2 Flash Dennis2929
Fish Audio S2.1 Pro, library voice3030

Left: rank by the first recognizer; right: by the second. Dark lines moved 8 places or more (at most 10). Tipjai Voice 1 ranks 10 and 5. This is why the board averages both.

Naturalness

Figure 1.83 points behind the best voice
  1. ElevenLabs v4 Sarah5.00
  2. ElevenLabs v4 Turbo Sarah5.00
  3. Qwen Audio 3.0 TTS Plus, longanlingxin3.50
  4. Tipjai Voice 13.17
  5. Microsoft Premwadee2.83
  6. Fish Audio S2.1 Pro, library voice1.50

1 listener, blind, 6 alerts from blind set 2, scale 1 to 5. A small sample: it shows a direction, not a ranking.

Speed and stability

On one laptop CPU, Tipjai Voice 1 takes 1.38 s per alert (median of 20) and reads up to 76 alerts a minute in parallel. Cloud voices include network time, so they are not compared here.

Figure 400 stress inputs: no crash, no silence, no clipping.
Inputs
400
Crashed, silent or clipped
0
Same text, same audio
Every time

The longest names and messages Tipjai accepts, emoji, links, other writing systems, numbers in every shape, invisible characters, HTML and stretched letters.

Limitations

  • One listenerNaturalness comes from one listener and shows a direction only.
  • 60 alertsEnough to separate voices that differ a lot, not enough to separate the top group.
  • Recognizers have biasesThe two disagree by up to 10 places (figure 5): the score estimates what a person hears, it is not a person.
  • The market moves fastThese are each vendor's voices on 5 and 6 October 2026. Newer releases are not in these results.
  • Thai onlyThis set measures Thai tip alerts. English is not measured here.
  • One machineSpeed was measured on one laptop while other work ran.

On blind set 3, written later and scored once, Tipjai Voice 1 was heard wrong on 6.9% of characters (60 alerts), inside its 95% interval on set 2.

Reproduce it

Everything needed to measure it again: all 60 alerts, the reading a listener should hear, and every accepted spelling of each name and loanword.

Recognizers
Thonburian Whisper (biodatlab/whisper-th-medium-combined); Typhoon Whisper large-v3 (typhoon-ai)
Metric
CER after mapping each accepted reading of a name or word to one spelling; digits, ๆ and stretched letters normalised
Intervals
95% bootstrap, 2000 resamples, paired 4000, seed 6
Listening loudness
listening clips matched to -20 LUFS by one static gain
Measured
2026-10-06

Cite

@techreport{tipjai2026thaitipalerts,
  title       = {How clearly do voices read Thai tip alerts? A blind test of 30 voices},
  author      = {{Tipjai Research}},
  institution = {Tipjai},
  year        = {2026},
  month       = oct,
  url         = {https://research.tipjai.com/en/research/thai-tip-alert-benchmark/}
}

These results first appeared on the Tipjai Voice Lab (tipjai.com/voice-lab) and match it number for number, because both read the same measurement files. Gemini 3.8 Flash TTS Kore is the clearest voice on the board.