Hacker News Reader

Comment by Symmetry | Hacker News Reader

You're overstating how well wrist cams can compensate for a lack of touch. It's really hard to get the instant estimate of weight and rigidity we humans get when picking up an object with a wrist camera, especially for rigid end effectors. They can guide a grasp in ways that would require touch for a human and help detect that you've dropped an object but don't compensate for many aspects of grasp that prevent the drop in the first place.

View on HN

"Estimate of weight and rigidity" is exactly the kind of thing that's performed rather well by low-dimensional sensing of actuators themselves.

Backdrivable actuators know how much current are they sinking - and thus, how much force are they facing. So even the simplest two finger gripper can easily "know" that one object is squishy and the other is not. The response curves are different. Same can apply to arm actuators for weight estimation.

Now, what this doesn't actually get you is an estimate of how hot/cold an object is, and how moist an object is.

Where cameras come in is specifically the "guide a grasp" part. Force sensing alone can tell a robot "the grasp is failing". A camera pointed at the grasped object can tell "why".

In theory yes but in practice backlash and stiction make drive currents much less useful than the raw force at point of contact for any realistic gearing. And an important part of weight estimation is that force lateral to the point of contact. Now, your arm motors will be drawing more current as they move so again in theory you can distinguish that, you've still got the weight of the whole gripper added in.

I spent years of my professional life on these topics[1] and from my experience motor drive current usually isn't enough to narrow down the weight of an object in a typical picking task even to within a factor of 2 to detect when you've accidentally picked up two items from a bin instead of one.

[1]https://patents.google.com/patent/US12612266

Oh, it gets extremely annoying when you try to use those kinds of indirect, noisy, smeared, nonlinear, relative measurements for absolute sensing.

The power of modern AI robotics is that, much like the biological mess it's trying to replicate, it doesn't care that much about absolute sensing.

Imperfect sensors fused by a high performance control model with a good foundation of world dynamics and self dynamics? Gets you impressively close to "perfect sensors" in practice. Being able to take a big bundle of noisy nonlinear ambiguous measurements, and implicitly reference them off each other and over time to get to a usable control loop is powerful.

It's the kind of power human nervous system wields to force the horrid mess that is the human body to perform.

For "detect when you've accidentally picked up two items from a bin instead of one", a wrist camera is a far more natural sensor than a bundle of actuators - especially for lightweight items. Wrist cameras are awesome. But even a bundle of actuators, if fused well enough, can often do it.

The problem is that designing algorithms to sensor fuse to this level by hand is pretty much a no-go. The kind of sensor wrangling hell I would only wish upon an enemy. Which gets us right back to general purpose robotics being an AI problem at its core.

Reply on news.ycombinator.com