Why we're building our own text-to-speech models

It has never been easier to make a computer talk. A handful of API calls will get you a voice that sounds natural enough to pass for a person in a short clip, and for many products that is the right answer. So it is a fair question why a lab of our size would spend its time training speech models from scratch instead of wrapping someone else's. The short answer is that the properties we care about most are exactly the ones you cannot tune from outside a black box. The longer answer is what this post is about.

What an API can't give you

The first thing is latency. In a conversational product, the delay between the moment text is ready and the moment the first sound reaches someone's ear decides whether the exchange feels like talking or like waiting. When you own the model, you choose how it streams, how much context it needs before it can start, where it runs, and what it trades for speed. When you rent it, you get whatever the provider's architecture and network path happen to allow, and you inherit their priorities.

The second is prosody: the rhythm, stress and intonation that carry meaning beyond the words themselves. A question that rises at the end, a list read with the right pauses, a name said with emphasis because it matters in the sentence. Hosted systems usually expose a few knobs for speed and style, but the model's sense of where emphasis belongs was fixed by someone else's data and someone else's judgment. If we want a voice that reads technical content carefully, or that handles a long explanation without flattening into monotone, we need to be able to change what the model learns.

The third, and for us the most personal, is language coverage. Albanian is spoken by millions of people across Albania, Kosovo, North Macedonia and a large diaspora, yet it remains a lower-resource language for speech technology. Most large providers support it thinly or not at all, and where support exists, it often stumbles on the details a native listener notices immediately: the two main dialect groups, the stress patterns, the way names and loanwords are pronounced. With a team in Prishtina, we hear those failures every day. Nobody is going to fix them for us on our timeline.

Then there is cost and privacy. Per-character pricing is reasonable for occasional use and punishing for products where speech runs continuously. And some audio should simply never leave infrastructure we control. Owning the model gives us a path to run it where the data already lives.

The properties that matter most in speech are the ones you can't tune from outside a black box.

What a modern TTS system actually does

It helps to lay out the pipeline, because "text-to-speech" hides a surprising amount of structure. The first stage is text normalization: turning written text into the words someone would actually say. "Dr." becomes "doctor" or "drive" depending on context; "1990" is a year in one sentence and a quantity in another; a date, a currency amount or a URL each has its own spoken form, and those forms differ by language. This step is unglamorous and is responsible for a large share of the errors listeners actually notice.

Next comes the question of how the model sees pronunciation. Classical systems convert words into phonemes using a lexicon and rules, which gives precise control but requires a good dictionary and breaks on words it has never seen. Many recent systems learn directly from characters or subword tokens and let the network infer pronunciation from data. Albanian is an interesting case here: its spelling is fairly phonemic, which makes grapheme-based modelling more tractable than it is for a language like English, but names, borrowed words and mixed-language sentences still need careful handling.

The core is the acoustic model, which maps that linguistic representation to some description of sound. For years the target was a mel spectrogram, a compact picture of energy across frequencies over time, and a separate neural vocoder turned that picture into a waveform. That two-stage design is still common and still works well.

A newer family of approaches represents audio as discrete tokens produced by a neural audio codec. The codec compresses speech into a sequence of symbols that can be decoded back into high-quality audio, and the acoustic model becomes something close to a language model: it predicts audio tokens conditioned on text, one step at a time. This makes it natural to borrow the tools and scaling behaviour of large language models, and it lends itself well to streaming. The cost is that autoregressive generation can drift, repeat or skip words, and errors compound.

Alongside these are diffusion and flow-matching models, which learn to transform noise into speech features through a sequence of refinement steps. They tend to be robust and produce smooth, natural prosody, and they can be made fast with fewer steps, though fitting them into a strict streaming budget takes care. In practice, the most interesting systems mix these ideas, using tokens for structure and a flow or diffusion stage for detail, and much of the craft lies in choosing where each belongs.

What we're focusing on

Our work is organized around a few questions rather than a single model. The first is streaming from the start. We are designing architectures where time-to-first-audio is a primary objective, not something bolted on after the fact. That shapes everything: how much text the model needs to look ahead before committing to a sound, how the codec frames audio, and how the system recovers when the text it is given changes mid-sentence, as it does when the text is itself being generated by a language model.

The second is Albanian done properly. That means building normalization rules for the ways Albanian writes numbers, dates and abbreviations, handling both Gheg and Tosk varieties rather than averaging them into something no one speaks, and treating code-switching between Albanian and English as the norm it is in everyday speech. Our early experiments suggest that careful data curation and normalization matter at least as much here as model size, which is encouraging for a lab that cannot win on compute alone.

The third is controllable prosody. We want a model that can be told, in some structured way, that this word carries the emphasis, that this clause is an aside, that this sentence should sound like a question even without a question mark. We're exploring conditioning signals that are rich enough to be useful and simple enough that a product can supply them reliably.

The open problems

Several things remain genuinely unsolved, for us and for the field. Long-form stability is one: models that sound excellent for a sentence can wander in pitch, pace or voice identity over several minutes. Robustness is another: token-based systems in particular can occasionally hallucinate, repeating a phrase or dropping a word, and in a product even a rare failure of that kind erodes trust quickly.

Data for lower-resource languages is a third. Clean, consented, well-transcribed speech in Albanian is scarce, and the easy shortcuts, such as scraping whatever audio can be found, conflict with the consent principles we hold ourselves to. We would rather move more slowly on data we have a right to use. Evaluation is a fourth; a single listening score tells you surprisingly little, and we have written separately about how we try to measure what matters.

Finally, there is the question of how far a small model can go. Much of the progress in speech has come from scale, but real-time products reward models that are compact enough to run cheaply and close to the user. Finding the point where a smaller model is good enough, and knowing precisely where it isn't, is one of the more practical research problems we face.

Why it's worth it

None of this means we will stop using hosted speech where it serves people well; Quad Chat exists precisely because different models are good at different things. But building our own gives us something an API cannot: the ability to decide what a voice should be good at, and to keep improving it in the directions our users and our languages actually need. We will share what we learn as the work matures, including the parts that do not go to plan.