You might need to send that question to the UK @ninjas_with_onions
I don’t know if in the English language it was always apples and oranges or if there ever was the apples and pears comparison.
2 Likes
I still use apples and oranges. Wasn’t aware that it’s changed to pears. The only time I’ve come across those two fruits appearing together is ‘apples and pears’ is Cockney rhyming slang for ‘stairs’ and slang is definitely not something to translate to!
2 Likes
It’s not “changed” to pears, it’s just different per area. In European French it’s pears, in Canadian French it’s oranges. Some compare bananas with oranges, others compare grandmothers with toads …
4 Likes
Some excellent examples, some of which I had to read a few times to fully comprehend what was going on in the sentance.
I wholeheartedly agree, even if the A.I. were able to get to that first level of translation, I would still have to make a quick assessment of the importance of the subtitle, general chatter or plot progression? Any key words that might be like what evidence was found, where, by whom, who was being accused and why, etc. Continue or pause, comprehend what was being said, restart. To always having to make those snap decisions watching the show hovering over the pause button to press at a seconds notice. It would be so disruptive to the immersion afforded them with naturalised translations into their language by humans, the very thing that got them hooked to the show in the first place and clamouring for a more speedy subtitling of the next episodes!
It really will be adding auto translate to satisfy impatience to just enable show/plot progression (in such a disruptive fashion) that the quality is actually degraded to the point that it’s detrimental to the show.
Enabling the impatient viewer a few days earlier access is just not worth the trade off in the loss of quality and that sometimes the best answer to the screaming child is “No” as it will benefit them greatly in the long run.
9 Likes
We, as active Spanish team members of the volunteer community, appreciate the fact that you are letting us know about this “AI project” instead of implementing it without reading our opinion first. As you will understand, this comes after a very unfortunate turn of events or issues such as the vikibot, the inbox change, the project finder, etc. Things that have led our team members to a lack of trust towards these type of initiatives since none of these changes have brought any improvement to our community but have caused lots of trouble for the volunteers.
When we saw the post, we were taken aback because we thought that we (the Spanish community) had already conveyed our reasons last year regarding the controversial usage of the Vikibot. The issue we are confronted with today is of the same kind. We ought to not dehumanize the incredible feeling of reading authentic translations. At least in Spanish, we have so many variations that “one size fit all” tool would render the translations completely inaccurate. Gender indicators are a core part of many languages and cannot be ignored, a machine puts aside these conventional norms (pronouns exist for a reason, right?), especially in Romance languages.
We, as a team, have thought over the past few days about the cons and pros of this journey you propose. However, we agree that the negative effects that our moderators, our active subbers, and that our editors will have to face will be far greater than the positive aspects that we can see, at least, for now.
We are aware that a lot of the complaints about lack of subs come from the Spanish speaking fans, however, we also know that an important number of those understand English too, so once an episode is completed in English they also stop asking for subs. Some of us had the opportunity of being channel managers for a long time, so we have the experience in moderating timed comments and almost 80% of those who complain, do it within the first hours of the episode being uploaded. By now, most of us have gotten over the impact of complaints about this aspect, so it is not a pressure for us anymore.
What would be a pressure on us, will be to have on screen “poor quality subs” even if those are not created by us. We will have unmotivated subbers who may not find a reason to hurry up, stressed moderators because we want to get rid of the auto subs soon, and the pressure as a team will increase. We also don’t feel it is okay to auto translate the hard work of the English team just to calm down the whiners.
We know that Viki has to be competitive in the market in order to survive as a company, but that won’t happen by lowering the quality of the subtitles. This would be possible by emphasizing or making more known how this website is a community. Highlighting how different we are from the rest and what aspects make us unique, because we are. So, we are not agreeing in the way this wants to be accomplished even though we understand the goal.
If you want to start trying out this tool, we believe it is a horrible idea to start with any of the romance languages. In the case of Spanish, we must consider the formal or informal way of speaking between characters, we consider a neutral form of speaking because Spanish is spoken in so many countries that we have already set an agreement into which slang or idioms are not allowed to be used while subbing, or which curse words are allowed etc, etc. Our rules of subbing are such that we normally do a google document per drama to clear this out for this specific case. Other teams do not make a document but they clarify in each “Go” all the vocabulary, and how each character speaks to each other. So, to summarize this part, we put a lot of thinking into the dramas in different ways. Therefore, our teams create the Spanish subs with such love that we are indeed one of the fastest languages to get to 100% after the English team releases episodes, so why do we need this tool applied to the Spanish language then?
Moreover, developing a solely performance-oriented strategy can jeopardize the core of what our communities stand for. We are committed to do our best, doing an activity that we cherish in order to broaden our audience and share the thrill of discovering such fascinating dramas. Putting bots to replace humans will deteriorate the atmosphere in which we work because human contact is essential to humanize the possibly cold virtual world. For these concerning reasons, we wish to advise you to think twice before taking major decisions in that direction. We thank you for the post, giving us an opportunity to voice our views and contribute to a project that we value greatly. From all the community, we wish to echo once more our will to find ways to work better together. What makes Viki special is its community, composed of many members that put effort, time and commitment in addition to their personal and professional responsibilities. That generosity is something that may not be measurable quantitatively at a first glance but that on the long term sustains the foundation on which Viki was built on. Once this is lost there is no turning back, therefore it calls for caution.
Something that could be done is not releasing new episodes until they are fully subtitled in English. And, as other legal sites do, establishing a releasing date in the app and website so viewers know when exactly the episodes are available in each language. In this case, the episode will be visible only from the editor to the added team in the channel and volunteers could work without pressure and the viewers could watch on an expected date without feeling “cheated” because they think that episodes come with subtitles and they don´t.
Thanks for your attention, time and concern,
Rumble95, Pilar_velasquez, Kalmendarez, Eun_soo_lee_2, Sita24, Miss_rochester, Klore6, Wendy_gutia, Henrri_11, Niennavalar.
97 Likes
I just got an idea. Instead of using AI for the subtitles, we might use it for the comment section! 
Something like this:
Each time a commenter types a word like “subtitle” (in any language, abbreviation, synonym, whatever), he/she is disabled from sending the message and instead a pop-up shows up, in which Viki clearly explains how our subbing system works.
28 Likes
I think that this might be useful for dramas where the subbing has been stopped for a few years…other than that, I agree with most people here that it’s not the best idea.
Say, there are a few Taiwanese dramas and older dramas where the subbing has stopped or the subtitles are incomplete in a few of the episodes (English and otherwise), this would be a good alternative until the moderator or channel manager gets back to the drama. A few examples:
This last drama is used for subbing and segmenting practice. Because the subs aren’t finished on this drama, I can’t watch it, but with the automatic translation (assuming that it would be used for Mandarin to English translation) I would be able to watch it, and the volunteers can work on subbing practice. (I probably wouldn’t be able to watch it anyways since the segging for the English subtitles probably isn’t finished).
This brings me to the question: does the automatic translation also make segments?
2 Likes
I just checked and none of them are available in my area … So no subbers from this part of the world.
But either way, I think Viki should at the very least ask whether any of us is interested in working on such dramas before even considering handing it over to an AI agent. And even then I wouldn’t watch those shows or maybe learn Chinese first, so that I could turn off the subs.
I should hope not! I think in theory it’s possible, but … leads to all kinds of horror. But the segmenters have more insight in that than I do.
1 Like
They aren’t available in my area either 
Some, I would like to watch, but because of the unavailability and the uncompleted subs, I wouldn’t be able to watch them either way.
Now that you mention it…yes, it can be a problem. I have run into this while translating in Spanish. Because the English sentence is split up and the sentence structure is different in Spanish, I’ve had to completely reverse the sentence in Spanish in order for the sentence to make sense. I wonder if the automatic translation would be able to 1). segment the sentence properly for the auto subs and 2). accurately construct the sentence instead of directly translating.
2 Likes
I don’t think so, as they said:
So the segments must already be in place. I do know that some of the dramas come pre-segmented and that we get asked to adjust and combine the segments. This used to be a horrible task in the past because the segments were horrible, but they have gotten better at it. Still requires a lot of time and effort to fix them.
True, but then the segmenters can’t practice 
2 Likes
We’ll need a little more filters for this if it gets implemented.
Some viewers also cheer up the teams and that can get blocked too!
Such as “Thanks XYZ Team for fast subs.”
I support @mirjam_465’s idea with a little bit more filters.
Or we can have one more Moderator from the volunteers or the office, to whom all the comments are sent which have “subtitles” in them and they can approve it or maybe mine is just a bad idea 🤷
3 Likes
Some time ago I was doing some Google Translation tests with a friends who started to learn another language (that I once learnt but aren’t fluent anymore in building more complex own sentences) and he wanted to know the sentence I said in my language for this other language… So the result was that when I used a natural sentence for my language (in which you can use different word order but still keep the meaning) Google T wasn’t able to catch that true meaning. The Google translation created a different meaning… only after rearranging the sentence in my language into a word order that is more ‘common’ and in the way how the word order for this other language is, Google was able to translate it correctly…
PS:
One could only say that without editing that might happen with human translators as well. I just saw it today while I was editing a scene but at least the meaning was mostly correct although the syntax wasn’t.
4 Likes
Yes, of course, we need to think it through thoroughly. Maybe a combination of certain words.
1 Like
True I didn’t see this far, thank you for your insight.
2 Likes
A human can make a mistake like that too, of course, but at least humans have the potential to handle these things in a correct way, and that’s more than I can say of Mr. AI.
3 Likes
Hello everyone,
Thank you so much for sharing your thoughts, feedback, and concerns about this experiment.
I’m late in responding as I wanted to give you all an opportunity to share your opinions openly and without pressure. I would like to address a few questions and comments raised during this discussion and hope that this can help explain what this is about a bit further.
Will Auto-generated subtitles be visible inside subtitling tools?
The answer is no. The additional subtitle track for the Auto-generated subtitles will not appear inside the tools, as it does not add any value and has the potential to be abused, as many of you raised with good reason. Because of that, the plan is to not display this on Subbing tools.
When will Auto-generated subtitles be available for Portuguese and Spanish?
For this experiment, the logic will make the Auto-generated subtitles appear once the episode reaches 100% in English. We’re only doing this with older library titles that have not been completed in either Spanish or Portuguese during this phase, so the hope is that English subtitles will already be fully edited.
If we validate this experiment and move on ahead with the project, the plan is to automate this function to enable edits to the English subtitles to reflect and update automatically in the Auto-generated subtitles track.
It’s better to use Machine translation with English subtitles rather than other languages.
We wanted to see if this was possible, but discovered early on that speech-to-text technology is not able to work well with content that has OST or any background noise, hence not the best option for entertainment media.
Lack of source language captions (ie. Korean, Chinese) will also make this very difficult, so we decided to instead use English as the source language for this project.
Better to use Auto-generated subtitles in missing segments or parts of the community translations rather than as another separate track.
The implementation of this will be very difficult and possibly create confusion as subtitles generated by AI will most likely use different vocabulary, which will be different than community created subtitles.
We also want to focus on differentiating what is Auto-generated vs. what is created by the community and something like this will make it very difficult to identify what is AI and what is human.
A big difference here is that we do not want to touch the community subtitles as we have done in the past with VikiBot.
This should be implemented in 1 channel only for the test.
The plan is to roll out this experiment to a limited number of library titles that haven’t been fully translated into Spanish or Portuguese.
Another important aspect of this experiment is to validate if our assumptions are correct or incorrect, which requires us to do this in more than one show to gather the most accurate feedback.
Volunteers will feel more pressured with the addition of AI leading to low-quality subtitles / Slow community subtitling as AI will lower volunteer’s satisfaction in completing a project / Subtitles by the community will remain unfinished as AI will be available already.
This is a valid concern and it is something we would like to research in-depth as well. We’re not planning on doing this in any on-air channel for the test, hence validating this during the experiment phase will be difficult.
Users do not re-watch shows and do not watch on-air shows that are older / Fewer people will see community subtitles as the assumption is that most users will watch with Auto-generated subtitles / Users will complain about the quality of the Auto-generated subtitles to Viki and volunteers.
Another valid concern and it’s certainly something we want to make sure we attend to with the following (non-exhaustive):
- Our suggested design will make it clear to any user that they’re watching Auto-generated subtitles at all times.
- Pop-up messages will also appear to indicate the subtitles are Auto-generated or to advise that they are an option.
- Down the line we want to work on the mechanisms that will notify users once Community generated subtitles are available for them to watch, giving them the flexibility to watch with Auto-generated subtitles or wait for Community subtitles. This feature is still in development and will not be ready for this experiment.
For the experiment, viewers need to manually opt-in for Auto-generated subtitles for every episode. Unlike subtitle language preferences where we make it convenient for users. This is to show our commitment that volunteer-created subtitles are still our preferred mode of showing subtitles as not everyone will want to watch Auto-generated subtitles.
We hope that these enhancements will make it easy for our viewers to understand that these Auto-generated subtitles are not made by the Community, and make it even more clear that waiting for community subtitles will mean better quality. We are trying to cater to the viewer that’s in-between or the viewer that is ok with lower quality subtitles.
Survey users to see how they like the idea of Auto-generated subtitles
We conducted a recent test + survey with more than 3,500 viewers where we showed them content with Auto-generated subtitles on various drama genres like medical, historical, romance, comedy, etc. and the results are the following:
User Acceptance towards the quality of Auto-generated subtitles
Portuguese Auto-translation
- More than 85% said they understood with content with Auto-translated subtitles
- More than 50% said they would turn on Auto-translated subtitles
Spanish Auto-translation
- More than 80% said they understood the content with Auto-translated subtitles
- Almost 50% said they would turn on Auto-translated subtitles
Need more English editors if you want faster subtitles
We’re aware of the importance of good editors in the community and that is why we’re currently working on a project to add various roles, including Editor and Chief Editor to the fold, in addition to better access controls across all roles.
This is not directly going to fix the problem of lack of editors, but we do hope to make the role much more visible while at making sure contributions for editing are properly applied. This is our first step towards increasing Editors across all languages on Viki and a long-requested feature by the community.
Vikibot = Auto-generated subtitles.
The big difference between VikiBot and the Auto-generated subtitles project is that the version of the AI is much more advanced than it was on VikiBot.
On various tests, the new AI version surpassed Google Translate as a translation tool by a significant amount, as it was built and trained specifically by Viki’s content, meaning Asian dramas only.
This might work for simple sentences not complex, poetic, metaphorical, or archaic translations.
While beating Google Translate is not a great victory and human translation will always surpass AI translation, we hope that it will be good enough and specific enough to cover most of the content that today is not fully subtitled yet.
There are, of course, limitations to the Auto-generated translations, but these will inevitably always be present and we hope that with the Community increasing their capacity little by little, some of these projects will also get completed and users will eventually have the quality they always expect from Viki.
Will Auto-generated subtitles replace the community-generated ones?
Another big difference between these tools is implementation. Since VikiBot was implemented to add and replace subtitles in the same language track, sometimes with terrible results, while also having a tool meant to help volunteers translate faster, which was also constantly abused, we can understand your hesitation and fear that this project will behave in the same way.
The Auto-generated subtitles in this project will not behave in the same manner as VikiBot. It is not going to be used to replace, remove, or add subtitles to Community generated subtitles, making the experience much smoother. There will also be clear indicators that the subtitles are not made by human and better ways to educate the audience about how the Community subtitles work.
Question - For interview is the time the user’s timezone?
This is specifically displaying your timezone. Feel free to select a time that is most convenient for you.
4 Likes
Final thoughts
It’s never been our wish to replace the community with AI subtitles. That would be the equivalent of shooting ourselves in the foot. This is intended as a test that we would like to validate it with real data. The reason why we selected Spanish and Portuguese as the experiment languages is because the adoption and watch time for both is higher than other languages.
Another thing is that we have no intention of implementing this on all channels in the future either. If we do reach a consensus and make this available for certain languages, on certain channels, there will be careful requirements and considerations to take to make this happen in the least obstructive way possible.
We want to work together with all of you to find that balance and see all the potential problems that could arise from this. Your careful and insightful comments here have been incredibly helpful in bringing that point home as well. The one thing that I would like you all to take away from this is to give it a try. Sign up for the research review, take a look at the prototype, give us your feedback, tells us your concerns, and let us work on this together. We are listening and want you to know that your input here is valid, but let us at least show you the prototype first.
I hope that I was able to answer some of your questions and concerns about this experiment. I would also like to take this opportunity to introduce you to our Product Manager, Valerie (@beefshi) and our Designer, Yu Siang (@clone0504) who will be available to answer any further questions you might have about the design and development of this project. And as always, I’ll also be happy to answer any further questions related to the Community.
With love and appreciation,
Mariliam
(Message divided as I reached the character limit set by the forum) 
9 Likes
Would this option be possible to the normal viewer too?
Since all other information doesn’t seem to have any effect on the so-called sub whiners, could a pop-up message be placed at the start of the video, to refrain from demanding subtitles?
Or like here at discussions like “you already posted 3 times …”?
I am no techie, so I don’t know the possibilities at all.
Thanks for going into detail, I wished I could help you analyze the AI’s work, but I think this should be more directed to the Spanish/Portuguese volunteer community. So good luck on this!
5 Likes
This does show that it’s not all about understanding the auto-generated subtitles. Of those 80/85% who says to understand them, only 50% plan to use them. So what would be the motivations of the other 30/35%?
2 Likes