Comment by fault1 | Hacker News Reader

Comment by fault1 | Hacker News Reader

Parent Full thread

> I actually do mean a walking robot that can find the exact knife it needs to fillet a fish, regardless of whether it's in the sink, in its proper place on a stand, on the table, or dropped on the floor (and if it finds it on the floor it will wash it before usage first). I mean a robot that can open the fridge and find the tomatoes, regardless if your brother has moved them to the top shelf or if they are where they should be. Etc.

From a computer vision perspective, I think most of that is fairly easy, maybe not getting all of the edge cases right, but it's more or less what this generation of machine learning enables (over "classical" computer vision).

What's hard about the scenario you proposed is probably on the robotics side, you would need a revolution in soft robotics:

https://en.wikipedia.org/wiki/Soft_robotics

Soft robotics is significantly behind hard robotics, and there are hard materials science and economics challenges from what I understand.

> From a computer vision perspective, I think most of that is fairly easy

No, not at all. The hard part is to do computer vision to understand what you can do with objects, not just convert object representation of images into string texts. For example, can I move this object away to get an object under it? Can I place an object on this object without it toppling over? Is this object dirty and moving it will smear things all over? If I topple this object, will it break? How much pressure can I apply to this object without breaking?

Those things are necessary to have an intelligent agent act in a room, every animal can do it, and our software models are nowhere near good enough to solve them.

You need the computer vision program to also have a physics component so it can identify strain, viscosity, stability etc on objects, and not just a string lookup table it needs to understand that naturally like humans as the same object category can have vastly different properties identified by looks.

I agree with all that (though I don't really necessarily associate any of that with computer vision, perhaps it's my physics bias).

Having good state estimation, kinematic, and dynamics models of real world objects is something that is very mature in controlled environments, but not very mature in other environments.

I think you might over-estimate what the current computer vision models are capable of. They are very good at recognizing what class an object belongs to, but they aren't very good at extracting data about the object based on the image. Extracting data of objects from images is image recognition, and humans and most animals relies heavily on vision to get data about objects.

they aren't very good at extracting data about the object based on the image

I’d say they are pretty good: https://towardsdatascience.com/a-guide-to-image-captioning-e...

Hm? Do we have robots that can pick eggs without squishing them, for example? I vaguely remember reading something like this years ago and it was impressive.

Yes, "soft robotic grippers" can do it.

Good link, thanks.

My point is that the corporations seems to just want to get the lowest possible hanging fruit with which they can reap the maximum profit... and stop there. I am not seeing any efforts to make an acceptable actual home robot.

I think like you: that most of the problems (that the bigger problem is comprised of) are solvable, but it seems that nobody is even trying to put the effort to make the entire package.

Where's the iPhone equivalent of a home robot?

Reply on news.ycombinator.com