A closer look at AI translation errors

[liczi]

Hello everyone!

I would like to draw your attention to the AI-generated translations offered by Viki. We all know that competition in the streaming industry is fierce and Viki wants to provide subtitles in as many languages as possible and as quickly as possible. Unfortunately, it seems that volunteers are no longer meeting their expectations because they cannot work as fast as artificial intelligence, which can generate a complete translation in a few seconds.

When it comes to translating dramas into Polish, especially historical and costume dramas, AI performs very poorly. Some people claim that AI is constantly learning but unfortunately it will never be as good as a translation created by a human. Viki has already released dramas such as Dazzling, The Scarecrow, Filing For Love and The First Jasmine with Polish (and other languages) AI-generated subtitles from the very beginning. Unfortunately, these subtitles contain many issues.

Here are some of the errors that appear in all of the dramas mentioned above:

• In Polish, verbs clearly indicate gender. Quite a lot of the subtitles contain incorrect gender endings.
• The word “you” in English can refer to both a singular and a plural subject. As a result, many subtitles contain mistakes, and it is often unclear whether the speaker is addressing one person or a group.
• Literal translations and awkward calques from English.
• When a word has several possible meanings, AI does not always choose the most appropriate or natural Polish equivalent. This is especially noticeable in dialogues that convey complex emotional states but it also occurs in many other situations.

I understand that Viki wants to make subtitles available as quickly as possible, but they should also take subtitle quality into account to ensure a comfortable viewing experience. In my opinion, they should adopt an AI + human model as soon as possible. Many platforms employ translators who review and correct AI-generated subtitles so that viewers can enjoy content naturally in their own language.

Rather than replacing us, Viki should recognize us as part of a larger project. Many of us hope that we will continue to be able to provide high-quality subtitles in our native languages. We are still hoping that the dramas which I mentioned will be available for volunteers so they could correct the mistakes.

[tenji_2024]

Thank you for this important perspective on drama translations. I share the exact same sentiments, and I am honestly appalled by the quality of translations in recent dramas. It feels like platforms have completely stopped engaging volunteers, even for proofreading AI-generated subtitles. Yesterday’s viewing of The First Jasmine was a massive disappointment due to the subtitles; they were a total mess. The same terms are sometimes translated into Polish, other times left in English, and occasionally end up as a bizarre mix of both. On top of that, verb endings are completely messed up (with zero distinction between genders), and modern slang is used in a historical drama—it’s just a terribly executed translation.

I understand that viewers want subtitles as fast as possible, but in my opinion, such sloppy and at times grotesque translations seriously diminish the value of the dramas we watch. For instance, having someone say “Check, please!” in a roadside tavern, or using titles like “Acting Chancellor of War”—I mean, setting aside the fact that I’ve never even heard of a “Chancellor of War,” using “Acting” (p.o.)? At this rate, we’ll be seeing “CEO” in a historical setting any day now. I truly hope the Viki staff reconsiders their approach because, to put it plainly, this is a slippery slope.

[mirjam_465]

@liczi

In my opinion, they should adopt an AI + human model as soon as possible.

They already started doing that when they first added additional presub languages (Spanish, Portuguese, and occasionally French) two years or so ago. I don’t know if they still do it or for every single language, but the concept is certainly not new to them.
Personally, I’m not a fan of that, either. I think really good subtitles come straight from a skilled human translator, not from anything artificial, no matter how advanced. A human correcting technical errors in an AI-produced translation doesn’t result into the same rich, natural translation that we could have had if that same human had translated the work from scratch. Not to mention what it must be like for the paid translators who have been robbed of their original task and are now dealing with this kind of thing.
Also, the human touch that might be there will probably be rushed since everything needs to go fast nowadays cause otherwise, viewers will complain. And somehow, those who complain about a lack of subtitles are always louder than the few that might complain about the quality.

@liczi

Some people claim that AI is constantly learning but unfortunately it will never be as good as a translation created by a human.

I completely agree. However, in another discussion, I came to the conclusion that in a way, it actually will. No, it won’t reach the level that it should have and that a human translator used to be supposed to have. But look at how much AI (and even bad human translations) has already infiltrated our languages right now. Not just on Viki, but in general. Overall, there is less or no control on the quality of what we read cause anyone and their cat can write whatever on internet, then there are more and more translations or other texts created by AI. All of us get influenced by that to some extent. Now imagine being born right now and growing up in this situation. What is that going to do to your linguistic development? I would say that even though the AI won’t reach the original human standard, AI skills will increase while human skills will decrease and then they meet in the middle. So in the end, all of our languages get corrupted, but there will be no one left to notice.

[irmar]

@mirjam_465

A human correcting technical errors in an AI-produced translation doesn’t result into the same rich, natural translation that we could have had if that same human had translated the work from scratch.

Although what you say is true, it wholly depends on the skills of the human translator.
In my years working here I have met many subbers whom I called - back then - “Human Google Translate”, at the time when Google Translate was awful. Nowadays, those humans are worse than Google Translate. What I mean is that their output was full of those terrible calques, misunderstandings, choosing the wrong alternative in a word with two meanings etc etc, but without having resorted to automatic translators. They were “honest”, it was their own work, but they were just ignorant and sloppy, they looked up words in the dictionary but chose the wrong meaning etc.
Sometimes their subtitles were so bad that I gave up trying to fine-tune them and I just deleted the whole sub and wrote it from scratch.
One would think that the only thing where a human is always better is that a human won’t mistake female for male or plural for singular. But that is if this person watches the video. Because if s/he translates from Bulk Editor for speed, or doesn’t watch the whole episode but only the part to be translated, those mistakes will also be there. This happens a lot in the Italian subber community. Many of them take a staggering amount of dramas, they volunteer for every drama position offered in their dedicated FB group. So they obviously don’t watch the whole thing (how could anyone?), they don’t follow the story, they don’t know who the characters are or what is their relationship. There was one especially, who wrote very beautifully and very correctly (she was a high school teacher), but she took on more than 20 dramas at the same time, and of course she never watched the whole thing, only her part. I was always able to catch her by some stupid mistake like this. Assuming a university professor the characters spoke about was male while in the previous part we had seen that professor and she was a female. Things like that.

Either sloppy or ignorant, those make the majority of the subtitlers in any language. With time, of course, you come to know who are the good ones, you have a list of trusted collaborators and try to secure them, but they are few and sought after, so it happens than when you contact them they may be busy with other projects. Especially when Viki chooses the CM at the last moment and the CM chooses the moderators in a rush just before the show starts. This means that you cannot always secure a whole team of 12 people of excellent or good level, but you will also have some medium and some bad ones.
Also, my habit has always been to also welcome a couple of new, unknown people every time I form a team (to refresh and add to the pool of people). This means there will be good surprises (you discover a new gem) and bad surprises (it’s a “human GT”).

Nowadays, AI has become better than bad subbers. The good ones have become editors.

[theparrot]

We are editing two Historical Dramas (Beyoind the Curse and Rebirth). As we are already at 3/4 of our task, here are my thoughts about them.

1. Beyond has got a worse translation compared to Rebirth. There are no words to describe it
2. Rebirth is a little better. Often (but not always) AI mix up English/Italian name of places/armies, sometimes get right Your Majesty/Your Highness or formal/informal speech, but not for the entire episode (random order). Same issue for the genre. I’m waiting to see WHEN it get right the word “Master” which have many meanings in Italian. At the moment, in every Historical drama, “master” is “teacher” in Italian (also in Jasmine). Actually, we listen to the Cinese to tell the difference. These issues come and go, as if AI cannot decide which is the better option.
3. In the first episode of Jasmine, AI got right personal/impersonal form for all its lenght, but I cannot say the same for the other episodes. It really looks like AI works on probability.
4. Unfortunately most part of the public, after have been fed with bad AI translations everywhere, have lost their taste and they swallow nearly everything.

I think the Staff (maybe at the end of the broadcast) will appoint a CM for Jasmine and allow the due changes and translations for OL. Maybe all the dramas will become like the “side projects” of our past.

[lutra]

Random question, does anyone know how much data is produced. Since there is more and more talk about data-centers that put waterquantity and quality at risk. How many humans does it need to create an equal effect in that aspect. It seems too many are running behind this technic, and in the way create a world less livable for everyday people.

Well sometimes I wonder about the price human kind has to pay for modern life. It seems to me like the waste of nuclear power plants, nobody wants it in their own neighborhood, but sure it’s one of the cheapest power available. Is it?

[xylune]

AI currently cannot replace a human to check the output fully. In many shows English titles are kept, the AI does not recognize context, it has issues with some terms, and can’t really differentiate between historical and modern settings. For example, in German we use Euch/Ihr as a polite way to address in historical settings, in modern settings it’s Sie/Ihr. In historical shows this all mixed up with the informal variant which results in chaos. Similarly, some modern council member may be addressed by a rather archaic title in a modern drama - or simply in English. When I calculated how many changes in a part have to be made in average it’s up to ~80%. Surely, you could reduce these a little, but formatting that doesn’t comply with Viki rules (like multiple line breaks) will remain. It’s not necessarily better or worse than an average human translator on Viki. Let’s be real, they make some mistakes from time to time, but usually those are different. And when I look at old shows there are issues with length and overly “detailed” translations repeating long terms over and over. I can understand this had to be improved, and it somewhat did by shortening the amount of characters (even though there might remain minor issues in some cases).

As the presubbed OL subtitles here always base on the English version this can also lead to many issues. English doesn’t differentiate between formal ways to address - and many shows that received external presubs (usually Chinese, Thai) are often not as clear or cut so short that the AI has even more issues to turn them into proper presubs in other languages. While I think that the issues with English titles/names could be fixed, fixing the context or the formal speech will be difficult without someone to check and correct it based on the character relations throughout the series. From my personal experience, editing here don’t make things worse either usually. It simply combines what both (human and AI) can offer to improve the quality.

I think one of Viki’s USPs were quality subtitles, but leaving AI unattended may let this vanish aside from the advantages (faster access), so I’d think it’s better not to abolish or delay checking them accordingly. Not to forget that I find it rather questionable that companies sometimes do not care at all about these aspects. Maybe because people consume it anyway and prefer “fast” over “no access at all”? But in my opinion they shouldn’t forget that all the other platforms do it as well, so forgetting about quality assurance (or delaying it significantly) won’t lead to any advantages. Not to forget that the “AI boom” led to a severe “AI rejection” in many fields nowadays, so I’d reconsider my strategy here.

[dimghro]

Working with AI for some time already, in both job and personal plans, I can say that it’s just a matter of time for the AI to get better, even in translations. Even though I am not an AI developer myself, I can already think of a few solutions to make the AI get things correctly in the translations.

As for Viki, I always thought that their biggest perk was the cultural richness they had in their subtitles, and they still have in those worked on before the changes. And even with the 100-character limitation, the contributors still managed to preserve a considerable amount of cultural notes.

Now, I really don’t see the point of totally dropping this huge perk just to get some very bad AI subtitles without letting the contributors fix them. Honestly, I would prefer to watch series on the other streaming sites, even though they cut a ton from the original script, but at least their subtitles make sense, they are fully linked with what’s happening in the drama and its context, and they are even most of the time grammatically correct.

I never thought that Viki would take the Mango’s path (if you ever watched any of their AI subbed series ). Yes, it’s for sure not that bad, but comparing with what the volunteers, the humans, can offer for free, even as editors, it’s a pertinent comparison.

[tenji_2024]

Thank you very much for this post, as it perfectly aligns with my own observations—so I won’t repeat the points already mentioned. I am currently watching my fifth drama with AI-generated subtitles, and it’s my first costume drama (Jasmin). About 70% of the subtitles contain errors. The Polish subtitles are full of mistakes; some proper nouns are translated halfway into Polish while the second part remains in English, and the vocabulary is completely inappropriate for the era (e.g., “photographic memory”—right, so they knew about cameras back then, I’m deeply impressed). On top of that, there are modern terms like “academy director” instead of “academy master,” “managers” in the prefecture office, “cafeteria,” and “lunch”! Furthermore, there is a complete lack of any stylistic adaptation to match the genre. It wouldn’t be so bad if this were just an initial draft and the drama was then handed over for human editing, but I’ve noticed that for the past month, VikiTeam hasn’t been recruiting language editors—and that worries me deeply.

Responding to other voices in this thread—I would like to add that for most translations, we do extensive research, whether it’s regarding the naming of government offices in specific Chinese dynasties, architecture, photovoltaics, modern technologies, etc. In the teams I work with, we have genuinely always strived for both the factual accuracy of our translations and a linguistic style that best captures the mood of a given series. I am deeply saddened that this seems to mark the end of volunteer translators; it looks like Polish viewers will be left with nothing but clumsy, error-ridden AI translations. I truly wish that, despite AI, we could preserve the practice of linguistic proofreading done by humans—by native speakers of the language.

[katek4220_956]

I couldn’t agree more. I was just trying to find someone responsible for the poor Polish translation in “Filing for Love,” and it turns out there’s no one to write to. Then I found this. I’m glad I’m not the only one pissed, yet I don’t think anyone at Viki cares.

[table122000]

I understand the frustration. I still do not understand why Viki will not allow volunteers to edit the AI presubs right away after the episodes are posted. I do not understand why V is now forcing a delay (and in some cases, a permanent ban) for volunteers to be able to edit the presubs. The poor quality of the pre subs is obvious to viewers. What harm does it do to Viki to allow volunteers to improve the subs after the episodes have posted? It seems to me that Viki benefits without losing anything. The AI presubs are available for the viewers that need to watch immediately and do not care about the quality of the subtitles as long as they can understand the main points. Volunteer Edited subs can be available later that provide quality improvements which have led to Viki reputation for having the best subs. Why punish viewers that care about quality subtitles that make sense and have some cultural context and nuance by not allowing volunteers to edit the presubs immediately after posting? Why destroy the very thing (Volunteer Edited Subs) that gives you a distinct advantage in the marketplace? Again, how is letting volunteers edit presubs harming Viki? Why are they forcing volunteers to wait to edit the subs or for some programs and films stopping volunteers from editing the presubs at all? I do think that the industry is wrong to suppose that viewers don’t care about subtitle quality. From what I have observed in other forums, viewers do care about it and will pay to watch quality subs. There are always posts that ask which platform has the best subs for a particular program or where to watch for the best subs.

[lilybidulgi]

True, the thing is, most people tend to say their mind only when they’re upset about something.
It’s sad, but not new, and that’s the way it is.

I often thank people, say when I’m really happy about a product or service, being in real life or on the net.

And only once in a while, only when there’s a huge problem, do I let them know there is, and I explain why.

But let’s be honnest, most people only share their opinion when they’re not happy, so…

[table122000]

@lilybidulgi

True, the thing is, most people tend to say their mind only when they’re upset about something.

I see your point. It is unfortunate that Viki doesn’t receive the positive feedback regarding the appreciation by the viewers for Volunteer Editing of Subtitles that include things like cultural context and nuance, historical accuracy (i.e. no modern terms), clarity and flow. Also must include timing as well. I think that Viki corporate do have some idea of the value of the volunteers as I have seen them promote “Viki high quality subtitles” in their ads for purchasing subscriptions. But yes, I’m sure you are correct that Viki isn’t receiving enough positive comments from viewers on how much they love the volunteer efforts versus negative comments on things like how fast translated episodes are posted.
And then there is another issue…
And that is… well, I hate to say it, but it must be acknowledged. Volunteer subs quality depends on the quality of the volunteers. Viki has many, many dedicated volunteers who are competent and well versed in their respective languages and the source languages and culture of the Asian content. That said— I think it has been raised in this forum before, but there are some volunteers who are relying on AI tools like Ggle Trsl**e when doing their work. And there are volunteers who don’t really care much about historical accuracy or the use of modern terms, who don’t care about slang or clarity or flow, or perhaps even exact timing. And so, even with volunteer editing, the end result is perhaps not that different from the AI version, or the version available on other platforms, so viewers don’t see much difference. Sadly, I was watching an historical drama yesterday where the subs had been edited for that episode and the dialog contained modern slang terms. I found it very depressing. If we volunteers are not willing to change the AI pre subs, then what hope do we have of showing our worth to Viki corporate? And since viewers are neither praising the volunteer edited subs that have cultural context, historical accuracy, flow well, clarity, etc, nor complaining enough about those that do not, then Viki corporate feels it’s okay to use the AI tools that do take care of the issue they hear the most complaints about : I want subs available in my language immediately. And I guess they feel okay with not allowing volunteers immediate access to the episodes. Although I still do not understand why Viki is delaying or outright forbidding access to the volunteers since in the end, it still doesn’t hurt them. The pre subs will either be much improved, or at worst, stay the same.
**Update: I just saw someone mention that some volunteers have been changing the names and not following the Viki Name Guidelines and that is why Viki took away access from the volunteers. I don’t know what to think now. Maybe if volunteers signed a promise to Viki to adhere to the Naming Rules, they would let volunteers have the access and not delay it?

[lilybidulgi]

This is so sad.
I don’t know what to say.
I’ve been working so hard, almost everyday, many long hours, for quite a while, searching for the right word, checking the context on screen, subbing first before pre-sub appeared for good, always triple checking my work, even full screen, to make sure I didn’t miss anything. Then, I had no choice but to move from subbing to edition. Later, I learnt translation edition and moderation…
Anyway, many long hours of learning and of work, for the pleasure of team work, for the pleasure of a good work done, to please the viewers with quality subtitles appearing at the right timing…
But sure, if some volunteers let us all down, even if they are just a few… what can we do?
There’s a difference between taking your time because you’re a perfectionnist, and not doing any real or serious work on your project at all, I guess…
I know it’s volunteering, okay, but it’s not kindergarten playground.
If everyone could behave and at least stick to their commitment and to the rules, that would be great.

And I’m sure most volunteers do a great work going by V. guidelines.

[pirjoi_65]

I only recently started volunteering here, so there is still a lot I do not know. I subtitle in Finnish, a language so rare that it is often not mentioned at all. There are also so few Finnish subtitlers that anyone who wants to contribute may have to act as a moderator and manage the entire process alone. This makes the progress quite slow and increases the risk of poor quality, and I have seen some subtitles so poor that I felt ashamed to be a Finnish subtitler. However, I have also seen excellent work from subtitlers who do this with great dedication and ambition.

However, I wonder: if there is no subtitling team, or if the team is VikiSubTeam, as with the series mentioned earlier, The First Jasmine, does that mean no one can translate it into Finnish—possibly ever?

[lilybidulgi]

Hi dear,
I totaly understand your frustration.
I guess it’s hard for rare languages newbie to find a team.

FYI, The First Jasmine will have 40 episodes.
Today, only 32 of these are out.
So, we can’t be sure yet whether there will actually be a team or not for the channel. Let’s hope so.

I think it might be better to wait till there’s a team, or if not that at least all episodes are out before applying as mod for your language. Good luck with that.

[igiam]

I’ve truly enjoyed reading everyone’s different perspectives. They’ve made me reflect on the topic from several angles.

As a Catalan moderator, I see the situation a little differently.

Personally, I wouldn’t mind if one day AI-generated drafts were also available for Catalan. However, I still wouldn’t start translating from those drafts immediately. I would continue waiting for the English team to complete their editing process first, because their work provides context, consistency and many important corrections that help every other language produce better subtitles.

For me, AI isn’t the main concern. What matters most is preserving the human review process.

The work of the segmenters, translators and editors is still essential. They don’t just translate words—they provide context, correct inconsistencies, identify who is speaking, understand relationships between characters and refine the dialogue so it sounds natural. All of that benefits every language that comes afterwards.

Another challenge I, personally, experience is the subtitle character limit. Some languages naturally require more space to express the same idea clearly, and fitting a natural sentence into those limits can sometimes be more difficult than the translation itself.

Technology will continue to evolve, and I think it’s natural that platforms evolve with it. What I hope is that human experience continues to be part of that process because I believe the best subtitles come from combining technology with careful human revision rather than replacing it completely.

Finally, I’d like to thank all the English teams and everyone involved in the workflow. Whether we translate directly or edit existing subtitles, we all benefit from the enormous amount of work that happens before the subtitles reach our languages.

Thank you all for sharing your experiences. Reading this discussion has been very interesting. ⋱✨⋰

[mirjam_465]

@igiam

The work of the segmenters

For the segmenters, the consequences are huge. Whenever they combine or split, they now have to copy-paste presubs in up to 23 languages and counting…

[igiam]

@mirjam_465

For the segmenters, the consequences are huge. Whenever they combine or split, they now have to copy-paste presubs in up to 23 languages and counting…

I can only imagine… or rather, to be honest, I hadn’t realised how much extra work this creates for the segmenters.

They truly deserve a lot of credit. Their work requires incredible precision, patience, attention to detail and a very sharp ear.

They’re the first step in the entire workflow, and perhaps one of the most demanding ones.

[lilybidulgi]

Thank you for your message.
It’s interesting to read a different perspective.
We know the characters limit can sometimes be quite a challenge.
However, as a viewer myself, I can somehow understand this choice, even if it came all of a sudden, and that limit meant a huge change, as we had many more characters in a seg before.
I often watch dramas late, sometimes on phone, and sometimes these were not even subbed in my language yet (back then, now edited, I try to wait for the English team at least).
So, I had to read long subs, often in a foreign language (even if I’m fluent, I do know it’s harder for me to understand subs when I’m tired, especially for historical dramas) late at night.
Well, you can imagine I often gave up on these late dramas, even if they were good, because I got tired of the “play-pause” game after a while…
I never complained, but let’s be honnest, it’s hard to read three long lines of subtitles every few seconds after a long day or week of work, when you just want to watch a drama to relax…