Hacker News Reader

Comment by AndrewKemendo | Hacker News Reader

This is much more impressive than I think it will get credit for.

I mean really, having an always on, immediate access massive NLP DNN system translating every piece of text from any language on the worlds largest platform is a staggering feat.

Facebook has the most impressive applications of ML right now in my opinion. They have Yann Lecun to thank for that (and Mark for recruiting him).

View on HN

> having an always on, immediate access massive NLP DNN system translating every piece of text from any language on the worlds largest platform is a staggering feat.

Too bad that they also have an always on system that's tracking my doings and whereabouts. Seriously, we need to stop applauding these companies. Sad to see that LeCun and colleagues don't care to find an employer with more noble goals.

Do u have a list of such noble companies to offer?

Gapminder, amnesty, msf, save the children... wait did you ask for ad companies like fb and google that sell clicks but had a noble agenda? I think the mercenary nature of ads precludes most moral stances in deference to the client.

I avoid FB as a social network, but I love their open source projects such as those in ML (PyTorch) and web design (React). Weirdly, I feel better working with FB products than the equivalent from Google (Angular and TensorFlow).

It seems that approval for the behavior of a company and it's open tech stack don't need to be correlated.

Is that a rational argument or an emotional argument?

??? Google's been doing this at scale for a while.

Google switched Google Translate over just last year, November 2016. Considering Facebook is developing all this internally and the scale they operate at and the presumable hardware disadvantages (does FB have any equivalent of Google's TPUs?), switching over in production so quickly is impressive.

Only for very few language pairs, though. But for those translation massively improved, especially for German the old solution was basically unusable.

Google was doing a statistical method of translation for a long time for Google Translate. Last year they switched over to Neural Machine Translation.

Google only supports Neural Machine Translation for a subset of languages. Whereas as Facebook is doing it for 2,000 translation directions.

45 languages * 44 other langues ~= 2000 translation directions. There are ~4,500 languages with more than 1,000 speakers ~4,500 * 4,499 ~= 20,254,500 language pairs so that's a very long way from every language.

Frankly though, I imagine the vast majority of those languages are unappealing targets in terms of return on investment.

and much smaller training data..

I guess they really don't need to cover all of them though as most of the people speaking a language with low number of speakers would be able to converse in other mainstream languages as well. I can speak four languages but I rarely converse in two of them as speakers of the two languages are usually comfortable with a mainstream language as well.

Even Swedish is barely in the top 100 with 8.7 million speakers.

The most obvious thing to me is to use an intermediate language - I assume this has already been discussed extensively, anyone have references?

edit: I should have scrolled down https://techcrunch.com/2016/11/22/googles-ai-translation-too...

Google may also have higher bar for translation quality..

Impressive indeed. Would you happen to know if Facebook is able to translate without using English as an intermediate language? For example, from Dutch directly to French, without needing to go from Dutch to English to French.

Reminds me reading about Google's new translator. It apparently uses a unique internal language it developed during training. All the languages it knows go through this internal language / neural pattern.

From what I recall reading, Google has people researching this internal language to see if they can discover any new interesting things about human thought.

https://techcrunch.com/2016/11/22/googles-ai-translation-too...

Interesting, thanks for sharing!

I don't know Facebook's specifics, but typically in research they will train a wide variety of examples, e.g.

English -> French French -> English English -> German French -> German

Isn't that what they are doing?

My impression of the pure DL approaches is that the preferred way to go about it is to map from each available language example, via a common latent representation, to every other available language example. The common currency is not English but a vector space representing something much more akin to the meaning underlying the statements.

Google created GNMT and put it into production first, then improve it so single model can tranlsate multiple languages.

Reply on news.ycombinator.com