Teaching machines to speak Albanian

Most speech technology is built for a handful of languages and then stretched, with varying success, to cover the rest. For Albanian, the stretch shows. Synthetic voices stumble on names, mispronounce ë, read dates as strings of digits and sound as though they learned the language from a phrasebook. Speech recognition does somewhat better on careful, standard speech and noticeably worse on the way people actually talk. We are working on both directions of the problem, teaching machines to speak Albanian and to listen to it, and this post describes what makes the language interesting, what makes it hard, and where our work currently stands.

A language on its own branch

Albanian is an Indo-European language, but it does not belong to any of the larger families that most people know. It is not Slavic, Romance, Greek or Germanic. It forms its own branch, and its closest living relative is, in a meaningful sense, nothing at all. That matters for machine learning more than it might seem. Multilingual models get a great deal of their competence in smaller languages by transfer from related, better-resourced ones: a model that has seen a lot of Spanish gets a head start on Catalan. Albanian has no such sibling to lean on. It shares vocabulary with its neighbours through centuries of contact, but its grammar, sound system and core lexicon have to be learned more or less on their own terms.

The language is also not one thing. It has two major dialect groups, Gheg in the north, which includes most speakers in Kosovo and North Macedonia, and Tosk in the south. The standard literary language, codified in the second half of the twentieth century, is based largely on Tosk. The differences are not cosmetic. Gheg keeps nasal vowels that Tosk lacks, preserves an n in many words where Tosk has r, and has an infinitive construction that the standard does not. The same verb form might be punuar in the standard and punue in Gheg. A system trained only on standard text and read speech will treat a large share of real speakers as if they were making errors.

Letters, sounds and the problem of reading aloud

Albanian spelling is, on the whole, a gift to speech engineers. The alphabet has 36 letters, and the correspondence between spelling and sound is far more regular than in English or French. But regular is not the same as simple. Nine of those letters are digraphs, dh, gj, ll, nj, rr, sh, th, xh and zh, each representing a single sound. A system that treats them as two characters will learn the wrong units. The difference between r and rr, a tap and a trill, distinguishes words. So does the difference between l and ll. Q and gj are palatal stops that many other languages do not have at all, and x and xh represent affricates that an English-trained model will not expect from those letters.

Then there is ë. It is one of the most frequent letters in written Albanian, and its pronunciation depends on position. In many words it is a clear central vowel. At the end of a word it is often barely pronounced or silent in standard speech, though it can still lengthen the vowel before it. A word like nënë, mother, contains two of them that behave differently. Text-to-speech systems that read every ë the same way sound immediately foreign to a native listener. The letter ç, an affricate close to the English ch, is simpler, but it and ë are exactly the characters that get dropped when people type quickly on keyboards without Albanian layouts. A great deal of informal written Albanian online omits the diacritics entirely, which means the same written form can correspond to different words.

Stress is the other quiet difficulty. It is mostly predictable from the stem, typically falling on its final syllable, but it is not written. Loanwords and names break the pattern, and the moment a model misplaces stress on a familiar word, listeners notice. We treat stress as something to be learned from audio and verified with native speakers, not inferred from spelling rules alone.

Morphology, loanwords and the shape of real speech

Albanian marks definiteness with a suffix rather than a separate article. Qytet is a city; qyteti is the city. Shtëpi is a house; shtëpia is the house. Nouns also inflect for case, and adjectives are linked to nouns by small particles that agree with them. The result is that a single noun can appear in many surface forms, each of which is relatively rare in any given corpus. For recognition, this inflates the vocabulary and scatters the evidence. For synthesis, it means that pronunciation lexicons built word by word will always be incomplete; the system has to generalise from stems and endings.

Real Albanian speech is also full of other languages. Centuries of contact have left layers of vocabulary from Latin, Greek, Turkish, Slavic languages and Italian, and the modern language continues to borrow. Çaj, tea, came through Turkish. Much everyday technical vocabulary arrives from English. Speakers in the diaspora switch between Albanian and German, Italian, Greek, English or Swedish mid-sentence, and speakers in Kosovo and North Macedonia may move between Albanian and Serbian or Macedonian. A recogniser that assumes a single language per utterance will fail on exactly the conversations people care most about. A voice that pronounces an English brand name with Albanian spelling rules, or an Albanian surname with English ones, sounds wrong in both directions.

Names are where speech systems reveal whether they were built for a language or merely extended to it.

Names and places deserve their own mention. Albanian personal names, town names and the names of streets and institutions are often absent from general-purpose data, and many have variant spellings across Albania, Kosovo and North Macedonia, or in transliterated form abroad. Getting them right is not a detail. It is the difference between a navigation voice people trust and one they mute.

Numbers, dates and everything that is not a word

Before a synthesiser can speak, it has to decide what the text says. This step, text normalization, is where many Albanian voices fail most visibly. Numbers are the clearest example. Albanian counting has vigesimal traces: twenty is njëzet, and forty is dyzet, literally two twenties. Some numerals agree in gender with what they count, so three is tre or tri depending on the noun. Ordinals take their own forms and agree as well. A date written as 28 nëntor 1912 has to be expanded into the right spoken sequence, with the right endings, and read as a year rather than as a quantity. Currency, times, phone numbers, abbreviations, Roman numerals and units each bring their own conventions, and the conventions are not always identical across the region.

We build normalization as an explicit, testable component rather than hoping an end-to-end model will absorb it from data. Normalization errors are systematic, easy for a listener to spot and embarrassing in production, and a small amount of careful linguistic engineering removes a large class of them. The same component also runs in reverse for recognition, turning spoken numbers and dates back into the written forms people expect to see.

Data scarcity, and the people who fill the gap

The deepest constraint is data. High-quality transcribed Albanian speech is scarce, and what exists is skewed toward read, standard, studio-like speech. Spontaneous conversation, regional accents, older and younger speakers, noisy environments and code-switched talk are all underrepresented. Scraping more audio from the web does not solve this on its own; it tends to reproduce the same skew, and it raises questions of consent that we take seriously.

Our approach is to work directly with native speakers from Albania, Kosovo, North Macedonia and the diaspora. They record, transcribe and, just as importantly, judge. Speech quality in a language is ultimately a perceptual question, and automatic metrics correlate imperfectly with what a native listener hears as natural. We ask raters to flag mispronunciations, wrong stress, unnatural prosody and normalization mistakes, and we track those judgements separately by dialect and region so that improvements for one group of speakers do not quietly come at the expense of another. Contributors know what their recordings are for, and we treat their voices as something lent to the project rather than taken.

What we are exploring

Several directions are active. On the recognition side, we are studying how far large multilingual speech models can be adapted to Albanian with modest amounts of carefully chosen data, and where they break: dialect, spontaneous speech and code-switching. Early experiments suggest that the choice of adaptation data matters more than its raw quantity, which is encouraging for a language where quantity is the scarce resource. On the synthesis side, we are working on voices that handle ë, stress and the digraph consonants reliably, that read names and numbers correctly, and that can eventually speak in more than one regional register rather than defaulting to a single idealised standard.

We are also building the evaluation to go with it: pronunciation test sets rich in names, numbers and difficult phonology; recognition tests that span dialects and recording conditions; and listening protocols for native raters. None of this is finished, and we are not claiming a finished product. But we think the path is clear. Albanian speech technology will not arrive as a side effect of work on larger languages. It has to be built deliberately, with the people who speak it, and we intend to share what we learn about how to do that well.