Ukrainian Students Develop Gloves That Translate Sign Language Into Speech | Hacker News Reader

Ukrainian Students Develop Gloves That Translate Sign Language Into Speech

techcrunch.com

49 pointscasemorton17 comments

Previously: http://www.tecca.com/news/2012/01/09/sign-language-glove-and...

Hmm, I'm thinking... couldn't Kinect be used for something similar? It could observe the gestures, and the computer could translate them into spoken language.

Probably, but I'm guessing it's more convenient (and easier) to use gloves. Do you really want to haul a Kinect everywhere you might need someone to talk to you?

No, of course not :). I was thinking more along the lines of a deaf-mute person being able to give a presentation or hold a lecture for people that do not understand sign language.

Very cool. Unfortunately, it looks bulky.

Disney's Touche demonstrated how the capacitive profile across frequencies could detect hand gestures. Ideally a simple wristband, coupled with a strong predictive model could do the same thing.

It seems the hard part about this would be mapping kinematics to sign language, hopefully this is decoupled from actually capturing the positions and motions.

Even harder is translating from the sign language to a spoken language. Machine translation is a very much unsolved problem, and sign languages are no more similar grammatically to spoken languages than spoken languages are to spoken languages.

Even worse a deaf person can't hear a typo when they say fuck pudding instead of fig pudding.

This is a fun thing to think about.

Typos: various effects caused by the interface used to write down language (so, broadly comparable to ink blotches), genuis, betwween, [2]

Mis-spelling: caused by spelling a word different from a standard[0], whole → hole

Mis-speaking/hearings: caused by not producing or processing a word according to standard, or commonly due to losses in transmission, cogenital defect → <traffic noise>genital defect (not sure why that was the first thing I came up with, but I'm sticking with it)

Mis-understandings in ASL: I assume these happen when the movement for a certain sign is similar enough that it can be performed or read improperly. I guess ASL was designed/evolved in a way that minimizes this, e.g. by maximizing the contrast for for common neighbours (otherwise efficiency will drive in the other direction).

What's fun about this is that these three classes of errors are almost direct analogues in their respective domains, but they probably lead to very different kinds of errors. Typos are basically arbitrary, but "comprehensible" if you know QWERTY. Mis-spelling and mis-hearings both result from the deviation from a standard or from transmission losses[1], and are often easily comprehensible. I assume the same is true for mis-understandings in ASL for people who know ASL -- but the character of being comprehensible gets lost in translation.

YouTube sometimes offers the combination of speech recognition plus machine translation. That's a close equivalent to machine translating sign language: audio/video recognition followed by machine translation. I don't think I have ever seen it work on a YouTube video, not even close. And I'm guessing ASL translation is more difficult: transcribing audio is probably easier than recognizing signs, and for audio there is an obvious target domain for transcription (ie. written English/phonetic transcription); whereas I'm not sure if there is any standard way to transcribe ASL, is there a "written ASL"? Come to think of it, there has to be, at least for "meta" tasks like dicussing or teaching ASL in printed form.

... okay so what was I going to do half an hour ago?

[0] but often in line with intuitive phonetic rules; you might go as far as calling standard English spelling another unrelated scheme ;)

[1] arguably betwween and genuis are also transmission losses; a more extreme example for interface transmission losses (same source and target, different interface) are showcased on pages like damn you autocorrect

[2] http://bash.org/?5300

Well, we don't have machine translation that does anything resembling natural language. But what we have is a lot better than nothing!

As an ASL user I can tell you that there are so many similar signs and body language that are used, only a couple hundred (very distinct) signs will be available to the users of these gloves (just like those before them). The idea I typed in my other comment would allow mutes to have unlimited conversations with complete strangers.

These gloves are crap so stop making stupid excuses. ASL is only good for communication between ASL users (few and far between). Every person signs differently and some $25 gloves will never be able to interpret all the signs and body language an ASL user uses (Thus the limitation of 200-250 words). Get real. Sorry for the harsh tone, but how about some real innovation.

This sounds incredibly harsh. In a world filled with people building "facebook, but for dogs" and the like, it seems very harsh indeed to be slamming some students for trying to build something in this space. I think even just building something like this at the price-point that they have is pretty cool. Even if this doesn't solve all the world's problems for ASL users it might be an evolutionary stepping stone towards something more useful.

I am always a critic but seriously why??? This may be useful for young children I suppose (who cannot read/spell yet), but typing is way faster than signing. So why not just have a custom formed mini half keyboard attached on each hip? The expense would be miniscule compared to this, the accuracy would be much better and the thing could probably be on the market in 4 months.

These gloves will probably break or need maintenance every 2 months, They will need constant replacing for growing users or users who gain weight, users will not be able to work at any jobs in the food industry (or many other industries for that matter)

typing is way faster than signing

Native ASL signers appear to converse just as fast as speakers of English. Perhaps you mean that typing is way faster than fingerspelling.

Uh, because deaf people can't hear it.

Using keyboards to voice speak would essentially make all deaf people mute to each other. Using gloves mean people who speak can be integrated into a deaf community.

Given the choice would you rather be integrated into a deaf community or the rest of the world? Also, nothing will stop ASL users from signing to other ASL users. How would the keyboard idea stop this?

As an alternative, they could always have an e-ink display that the words can be projected onto.

There are only 500k ASL users in the entire world. Deaf people are not that common. They don't interact with each other that much in the real world.

This would never work anyway. As an ASL user I can tell you that there are so many similar signs and body language that are used, only a couple hundred words will be available to the users of these gloves (just like those before them). The keyboard thing would allow mutes to have unlimited conversations with complete strangers.

Would you rather be able to have unlimited conversation with anyone or only be able to speak a few words (just enough to get by) but be able to speak with an occasional mute. These gloves are crap so stop making stupid excuses. ASL is only good for communication between ASL users (few and far between). Every person signs differently and some $25 gloves will never be able to interpret all the signs and body language an ASL user uses (Thus the limitation of 200-250 words)

The question really is whether the signing deaf community want to be integrated with rest of the world.

An interesting but often forgotten part of the development of the Cochlear Implant was the stiff opposition by the signing deaf community that considered the implantation of deaf children as a form of 'genocide'.

It's actually still an active topic in the Bioethics field. http://www.neiu.edu/~gmoreno1/Special_Education__Courses_wit...

Reply on news.ycombinator.com