Show HN: Touchable augmented reality on your phone, 3 years in the making | Hacker News Reader

Show HN: Touchable augmented reality on your phone, 3 years in the making

portalble.cs.brown.edu

15 pointslazyjeff5 comments

Looks quite interesting. I remember when the Leap Motion was first being released but I haven't seen a great deal in the wild that use it.

I signed up to get an early dev kit of their technology when I was working at an old job. We were a non-profit agency that helped educate individuals with disabilities - often having difficulty with their motor skills (using a typical mouse/keyboard was impossible for most of them). The idea was we'd use Unity3D or a similar game engine to create minigames that used the leap motion as the input device. The games would educate the individuals on many things from normal daily hygiene to cooking, etc.

In the end, our tiny team (of 3-4 people) bit off more than we could chew and nothing substantial was ever materialized. The leader of the team moved on to another company and the project dissolved. It was definitely one of the more interesting projects I worked on at that company.

Cool. But you shouldn't need Leap for this these days, you can do hand tracking with ML: https://ai.googleblog.com/2019/08/on-device-real-time-hand-t...

We have directly tested the MediaPipe one and it's less accurate than the Leap. Like it's completely unusable in practice, especially when the palm is not facing the camera. Also there's an HCI problem, which is that people often go offscreen when interacting with objects (your brain assumes things can happen outside the field of view), but even a moment outside the field of view breaks the interaction.

Try checking out https://ai.googleblog.com/2019/08/on-device-real-time-hand-t... this might remove the need for any hardware apart from the phone & its vibration motor.

Yup we've looked into that. It's not accurate enough to feel good to use, and often you go off screen for a bit which breaks the interaction. The infrared sensor actually has a field of view wider than the camera and helps maintain the connection in that brief offscreen moment.

Reply on news.ycombinator.com