Machine Translation Is Coming for Languages With 4,000 Speakers

The station’s archive holds 11,400 hours of tape, most of it in Orvan, the language of an island five hours by ferry from Port Ellery.1 About 4,100 people speak it, according to the island council’s last census, and roughly 1,600 use it every day.^* Since 2022, volunteers have transcribed some 4,000 hours of those tapes, and the transcripts have become the largest part of the Orvan Text Bank, a collection of 2.3 million words that now feeds something the language has never had: a machine that can translate it, after a fashion, into English and back.

For a language this size, that is a remarkable thing to exist. It is also, depending on whom you ask on Orva, a lifeline, a nuisance or the beginning of the end. The islanders are not arguing about whether the machine works. It mostly does. They are arguing about what happens to a small language when its most prolific writer is a computer.

A corpus from the airwaves

The machine lives in a spare bedroom at the north end of the harbor, on a graphics card that Davin Mure bought secondhand from a gamer in Halbridge.2 Mure, 29, grew up on Orva, spent six years writing logistics software on the mainland and came home in 2021, he says, “for my mother’s cooking and the broadband, in that order.” He started the Text Bank the following year with a spreadsheet and an appeal on the radio, and he now coordinates 63 volunteers who transcribe, correct and translate.

The big languages are taught to machines with hundreds of billions of words gathered from the open web. Orvan has no such web. Its 2.3 million words come from wherever Mure and his volunteers could find them: the radio transcripts, which make up more than half; 40 years of the parish newsletter; the bilingual minutes of the island council; a shelf of children’s books; and a 1904 translation of the Gospels, which for a while accounted for a fifth of the whole.3 Every sentence in the bank has been paired by hand with an English translation, because that is what a translation model needs: not just Orvan, but Orvan standing beside English, line by line.

The results are better than anyone expected and worse than its fans claim. Last autumn, 12 native speakers rated a blind sample of 500 of the model’s translations. Going from Orvan into English, 61 percent were judged acceptable, meaning a reader would get the sense without being misled. Going the other way, from English into Orvan, only 38 percent were.^† That gap is the whole argument in miniature. A tool that helps you understand Orvan is fairly good. A tool that writes Orvan for you is, more often than not, writing something slightly wrong.

“It’s like having a very confident cousin who learned the language from the radio,” Mure said. “Which, to be fair, is exactly what it is.”

Seen or heard

Tesni Arden found the problem in the margins of homework. Arden, 44, teaches the Orvan-medium class at the island’s primary school: 19 children between eight and 11, most of whom speak English at home and Orvan in her classroom. In the autumn term she began to see the same mistake in essay after essay, from children who had never made it before.

Orvan verbs carry a small suffix that English lacks. Add -ach to a verb in the past tense and you are telling your listener you saw the thing yourself; add -eth and you are passing on something you were told.4 Mairn tolach means Mairn came, and I saw her; Mairn toleth means Mairn came, or so they say. The distinction is the grammatical backbone of island gossip, and of a good deal of island manners. To report a neighbor’s illness in the seen form, when you only heard about it at the shop, is faintly presumptuous.

English sentences don’t mark the difference, so when the machine translates from English into Orvan it has to guess. It nearly always guesses the seen form, because that form dominates almost every source in the bank. Children checking their homework with it absorbed the guess. By November, Arden said, about a third of her class had started using the seen form for everything, including events they had plainly not witnessed, such as a shipwreck in 1881. She asked one boy why. He showed her the screen. Her diagnosis is blunt: “The children don’t copy the machine because it’s right. They copy it because it’s confident.”

That, more than any single error, is what worries her. Correcting a child who has guessed is easy. Correcting a child who has a printout is not, because a grandparent can be wrong and a screen, to a nine-year-old, cannot. And the machine is not only doing homework. The island council has begun using it to draft leaflets and notices, to save on translators’ fees, and Mure has watched English idioms surface in them word for word. The tool renders “make a decision” with a verb for manufacturing things, where an Orvan speaker would say that the decision came to them. A careful reader notices. A child learning to read from the council’s leaflets may not.

Linguists who study small languages have a name for what Arden fears: the drift of a written standard toward whatever its most productive writer produces. On Orva, that writer was once the church, and then the radio station. Within a few years it could be the machine, turning out notices, forms, worksheets and homework in an Orvan that is grammatical and comprehensible and quietly English underneath.

Arden is not against it. She uses it herself to rough out worksheets, which she then corrects, and says it has halved the hours she spends on them. Families who moved to the island to work at the fish farm use it to read the school newsletter. Some older islanders, who never learned to read English comfortably, use it in the other direction, on letters from mainland hospitals. “It’s a gift,” Arden said. “I just don’t want it to be the one deciding what the language is.”

Who keeps the words

Last spring, the question stopped being abstract. A large mainland technology company, which Mure will not name because its offer came with a confidentiality agreement he declined to sign, approached the Text Bank with a proposal: hand over the corpus, and Orvan would be added to the translation service that already sits on most of the phones on the island. No money was offered. What was offered was reach. Orvan would appear in a menu used by hundreds of millions of people, between languages spoken by tens of millions.

The Text Bank’s committee put the question to a public meeting in the community hall in January. About 140 people came, which on Orva is a crowd, and they argued for three hours. Younger speakers, and the parents of children learning the language, mostly wanted to say yes; to them, a language that is not in the menu is a language that does not quite exist. Others asked who would correct the company’s version when it went wrong, and which way it would pull the language once it belonged to people who could not speak it. A fisherman pointed out that the radio tapes contained the names of some 200 fishing marks around the island, a few of them known to only a handful of families, and asked whether those were now going to be on everybody’s phone.

Garra spoke near the end. She had no objection to the machine learning from her, she said—it already had, from 31 years of her voice—but she objected to it learning from her somewhere she could not go and argue with it. “If it’s going to talk like me,” she said, “it can come and sit at this table and be told when it’s wrong.” The meeting voted 91 to 46 to keep the corpus on the island under a community license: open to researchers and to anyone building tools for Orvan speakers, closed to anyone who wants to sell them.

Mure voted with the majority, though not happily. He knows that a model running in his spare bedroom will never be as convenient as one on every phone, and that convenience is how languages, like tools, tend to win. “We may have just chosen to be right instead of being used,” he said. “I don’t know yet whether that was brave or stupid.” His answer, for now, is to make the island’s tool good enough to deserve the choice. The newest version includes a feature Arden asked for: when an English sentence leaves the evidence unclear, the model stops and asks whether you saw it or heard it.

It is a small change, and it makes the tool slower. Arden thinks it may be the most useful thing the machine has done, because it hands the decision back to the person who knows. In the record library, Tam Leith tried it on an English sentence of his own: My grandmother was on the radio. The screen paused and asked him which he meant. Strictly speaking, Garra retired nine years before he was born. But he has spent every Thursday since September listening to her read the news, and he decided that counted. He chose the seen form.