Unsupervised speech-to-speech translation from monolingual data
blog.research.google
blog.research.google
Also the debate about whether all human speech need be piped through such a preemptive filter. (actually not looking forward to this one). Suddenly everything that anyone says will be couched with "it is important to consult a professional to ensure safety and compliance with local regulations".
Have you worked at a company with a PR, Comms, and HR department?
[Obama's] "Anger translator" (2012) https://www.youtube.com/results?sp=mAEA&search_query=anger+t...
A citizen ostensibly forfeits their right to sue for defamation when they become a public figure; but counter-non-fraud isn't fraud either then eh.
Say "that's not enhanced" just like the old one please.
In any case, text seems to stil form a part:
> During training, we use monolingual speech-text datasets
So there's still a way till machines learn language as humans do, i.e. with sounds as primary modality. But nowadays I won't bet as to how long any ml task for language will take to be solved
1) they use a single, shared embedding space for the two languages, forcing the model to learn "semantics" independently (or rather, interdependently) of language 2) using back-translation for training. I'm not sure that I got this right, but this seems to be round-trip translation? So the model can self-assess its performance by checking the spanish->english->spanish difference.
Sounds very promising and interesting! However, it seems they only tested on spanish and english. I wonder if the similarity of the languages at the lexical level made these results possible.
Aside from the lack of training data in many languages, I get the impression that tech companies like Google have been anglocentric in their approach, resulting in ok results only if at least one of the languages are “big”. That’s one thing that’s amazing about ChatGPT, it doesn’t discriminate between languages much, or, at least it seems like it’s able to transfer knowledge really well between languages. It seems it finds the higher level patterns of human knowledge to the point where language or even style is basically just a frontend.
Ironically, it seems the less you bother to teach computers about linguistics, the better they perform at language.
I really hope LLMs become mainstream in non-tech fields soon. I would love to see its impacts on the social sciences, linguistics, diplomacy (and yes, animal communications!), etc.
They’re obviously not perfect, but this is a task that’s impossible to do perfectly. To translate some speech accurately you first have to decipher what the speaker was trying to say, and with most speech coming from human beings, it’s not always clear exactly what it is they’re trying to express, and that’s before you even address trying to translate it.
This is also where the quality is varying a lot between languages.
I'm a native French speaker with an heavy non-standard accent and no speech recognition currently works for my normal voice.
I also speak Vietnamese to an okay level and while the speech recognition is usually good, the fact that the language is not based on words but syllables usually trips machine attempts at translation and the result is generally pretty bad.
These kind of live translations while impressive require the full pipeline to work, speech recognition and then proper translation. If any of them fail, the whole thing fails. On my case, it only works with my English for now then.
That's a good point. Machine translation of text input by LLMs is pretty good for the languages I speak--English and Japanese--but it can get much worse with spoken input if the speech recognition isn't very good.
That said, OpenAI's Whisper is able to transcribe my spoken English nearly perfectly, and GPT-4's translation of that transcription into Japanese is usually very good. The main failures are with proper names.
There was a thread today about LLMs' robustness to scrambled data; this seems heavily dependent on the quantity of training data.
That's for text to text though, not sure if voice to voice would be better or worse in this regard.
I cannot even imagine how this translation service might be capable of translating this. A human translator (translating it from speech or text to text) would likely add lots of footnotes with explanations so that the reader is capable of understanding the intended side-meanings that are hard to translate.