How to Train Your Robot | Hacker News Reader

How to Train Your Robot

construction-physics.com

141 pointscrescit_eundo47 comments

Sebastian Thrun's talk at Google about winning the DARPA Grand Challenge [0] has a great discussion of training "robots".

Summary:

- the LIDAR had error rates that were not published

- hitting potholes could cause the INS in the car to get confused

- they weren't sure how to fix the errors from the combination of the above

- they ended up just driving the car to generate training data

It worked out well because the computer was predicting if there was an obstacle or not and a split second later the driver of the car would either drive around the real obstacle or the computer would realize that there was no obstacle.

If you've never seen the talk, highly recommend as Thrun is very funny and it's also a time capsule back to when self driving was still a brand new concept.

0 - https://www.youtube.com/watch?v=j8zj5lBpFTY

Great video, thank you for sharing

This is an excellent write-up and has successfully changed my opinion on something I think will prove critical in the near future: I always laughed at humanoid robots, assuming they were being built for "comfort" reasons. I now realise this is almost certainly the most reliable way to solve the general robot problem - not just for the reasons the OP lays out, but because the whole world is already built for human-shaped agents anyway.

It's not just that the world is built around human-shape, but it's also that humans are very, very adaptable. Our hands are a marvel of evolution allowing us to do extremely precise tasks while still being adaptable to all kind of movement : Screwing in, out, grabbing weird shape in 6dof, rotating along several axes, ... And ours arms, our legs, etc, have similar benefit. As an example, we are not a fast specie, but we can walk on even ground, rough ground, steep inclined, we can climb rock and more. If you see a place, you can probably go there.

This is why human are still used for many, many tasks, including in industry where a lot of things are handle by robot. Yes, you can have an automatic belt feeder, or moving tray system, to move pcb from the pick and place machine, to the furnace, to the assembly, etc. But it means your assembly line is fairly rigid. Accommodating a new process, a different size of pcb or whatever become a costly endeavor. A human on minimum wage can cheaply move things from machine to machine, check if everything is alright in the process, adapt to new processes, new floor layout, very easily and cheaply.

Robot don't need to be exactly human shape, but having robot with the same adaptability has human have a huge advantage for companies where adaptability is a core part of the business.

> It's not just that the world is built around human-shape, but it's also that humans are very, very adaptable. Our hands are a marvel of evolution allowing us to do extremely precise tasks while still being adaptable to all kind of movement

Not just our hands but really our whole bodies.

I recently taught one of my kids to ride a bike and once they got it I thought to myself:

"Somehow millions of years of evolution have somehow allowed us to ride a machine that was invented <200 years ago"

“When man invented the bicycle he reached the peak of his attainments. Here was a machine of precision and balance for the convenience of man. And (unlike subsequent inventions for man's convenience) the more he used it, the fitter his body became. Here, for once, was a product of man's brain that was entirely beneficial to those who used it, and of no harm or irritation to others. Progress should have stopped when man invented the bicycle.” — Elizabeth Howard West

Well, it was invented by and for us, but your point still stands.

“Somehow millions of iterations of training has allowed us to one shot a task that was made up 0.1 s ago”, the agent quipped in CoT

The difference is whether or not a factory is designed with robotics as the driving principle or with human labor as the driving principle. They will be completely different, and as a result a lot of those things that would cause issues are simply engineered away.

This only works for high volume / low margin. Humans excel at rapid change - for now.

> Our hands are a marvel of evolution allowing us to do extremely precise tasks while still being adaptable to all kind of movement

It's easy to overlook, but to me it's amazing that humans are able to type! Subtle, precise movements by our fingers to express language were presumably not skills our ancestors had an evolutionary advantage for.

Yeah, for all we know there's a creature at the bottom of the ocean that's just as intelligent as humans, but it's hard to build a civilization with fins.

> but because the whole world is already built for human-shaped agents anyway

not really, there's several non-humanoid shapes that can easily adapt the human shape and to it but it's a little out of the box. you can imagine it without imagination, too, with a little thinking. it's just a few rational hops, really.

Most countries have laws like the US Americans with Disabilities Act meaning that there are enough ramps and elevators around that a wheeled robot can do pretty well. It actually is more about the training factor.

LLMs have shown a way to absorb large amounts of textual training data and be useful. For sensory processing you need a different architecture to absorb spatio-temporal streams. Bolting vision onto language models is stretching the tech a bit too thin.

The good news is the data is plentiful — it is all around (the robot). The real world — the robot’s umwelt sensed through its sensors and acted on by its actuators — will provide heaps of data.

Interestingly my colleagues propose that embodied learning will lead to grounding, alignment, and AI safety.

(Worked for 4 years at a robotics + machine learning startup, Director of R&D.)

Embodiment for general purpose goals like "grounding, alignment, and AI safety" just seems to be one of those things that some people want to be true. There's no actual reason for it to be true. There are no findings showing it to be true. People just want it to be true anyway.

I'm very skeptical of that. Wishful thinking is a treacherous thing.

LLMs provide some incredibly solid "general purpose commonsense reasoning" primitives. And the likes of Astra, Opus 5.5 and Gemini 4 have demonstrated generational leaps in "spatial problem solving" - including robotics tasks. Those don't even have dedicated "sensor input embedding" and "action output diffusion" ends rolled into them at model level - but you can give them control over robot arms, and they'll have a decent chance of, slowly, finding a way to complete new tasks you'd previously would have to tune a robotics model for.

Previously, I have considered "generative video models" to be a decent candidate for the foundation of future robotics models - they, by necessity, perceive video, learn the kind of world dynamics required to plan and execute actions in real world environments. And they can begin to learn those dynamics off embodiment-free general purpose web-scale datasets.

But non-robotics LLM generalists have come to show they're no slouches either. They're not at all optimal for things like rapid precise manipulation. But they find more and more use in robotics pipelines anyway. Makes me wonder what something like "Astra reasoning engine but with robotics architectural extensions" could accomplish.

Why would the tech be "stretched too thin"? We have made it work better by adding images, video, audio. Why would other sensory input be any different?

Is the issue also a lack of available sensor data?

Our own nerves produce a huge amount of data. Our eyes and balance also contribute to a very rich picture to enable movement and kinematics. This is all fused into usable feedback.

Arms on fixed axis with disconnected vision feel like we are still very far away from true robot training.

That is part of the problem. Yes, humans are absolutely studded with sensors, every hair you've got, every nerve ending. It's amazing that all that information gets processed as fast as it does even though the 'base clock' is rather low. There is a ton of pre-processing done on those signals though (for instance: the optical nerve is pretty much an extension of the brain, how is that for a high speed, high bandwidth interface). Then there are the reflexes, semi-autonomous parts of the nervous system that can react faster than the brain to deal with safety related concerns. For a very interesting biology angle have a look at octopi.

Human "sense of balance"? Human vestibular system is a sad joke if you compare it against any off the shelf smartphone IMU. Proprioreception is a joke when you compare it against current sensing in every motor and absolute encoders in every joint. Human hearing loses to a half-decent microphone array.

Even human vision, the one perception channel evolution has put an awful lot of weight on, is beat pretty resoundingly by the kinds of camera units that ship in smartphones nowadays.

The two senses robots don't have an edge in are: touch and smell. But I'd struggle to make the case that smell is critical at all, and touch? Hard to beat it directly. In practice, however, it's already partially compensated for.

Sure, a robot with a basic UMI-style gripper doesn't get rich tactile feedback - only a few channels of feedback from arm joints and finger motors. But you know what those actuated UMI gripper get that humans don't? Wrist cams. A robot can see the object it's manipulating - right near the point where the manipulation happens. That matters a lot for precise manipulation tasks.

Humans don't have wrist cameras not because they're undesirable, but because evolution sucks at wiring. We don't suck at wiring.

The fact that humans can pull off general purpose manipulation tasks with the bodies they have, and the best of robots can't? A testament to how much of a gap is there between modern robotics AIs and the evolved wiring of human brains. The intelligence edge is what makes the human body, flawed as it is, work as well as it does. Everything in the human body is half-assed and then salvaged by the brain in post. But that gap is closing as AI tech advances.

Humans have better dynamic range for hearing and vision compared to any sensor I'm familiar with.

Human vision is also extraordinary sensitive, with some people being able to detect individual photons with their rod cells.

Your reply to my comment downstairs got chewed up by HN. Maybe some sort of LLM detector thought you're speaking too much like an AI?

The reason why I'm not "team sensors" is that we're nowhere near to being able to as much as squeeze out even one mid-range camera sensor for all its worth. Things like AI assisted scene reconstruction show just how much data is there in even a small series of photos - when the AI is there to back the raw measurements with sensible assumptions.

The difference between "a plausible reconstruction" and "an accurate measurement" is academical rather than practical. You never get perfectly accurate measurements (because real world sucks), so a well designed "plausible reconstruction" will often have better error averages AND bounds than a naive measurements-first pipeline - simply by taking better advantage of spatial and temporal redundancies. Adding more assumptions to the pipe from there gives you even better performance.

I expect approximately none of the near-term gains in robotics performance to come from the sensors themselves. There's still room for improvements there, mind. There is some cool sensor tech that could result in a better performance ceiling. But we're nowhere near that "sensor ceiling".

Today's robotics sensors are so far ahead of what robotics AI can reliably utilize that trying to push for "we need better sensors" is a mistake in nearly every case.

I agree with everything you state about squeezing data out of sensors.

But, I also run into a LOT of cases where I need better sensors too. I just don't see them as mutually exclusive.

I did stick my comment into an LLM, annoying, I hand wrote the entire comment, and had it fact check my statements. Here is my original unmodified comment:

    Eh, there's too much here for me to give a proper reply.

    I agree with a lot of what you said, and I'm on team sensor, but I've been studying how the eye works and processes info.

    A *system* of many sensors can cover larger ground for sure... But I'm very interested in compact systems that can be better than the eye or ear, I know we can do it, I just haven't seen it yet.

     Also, AI and/or filtering can't give you data that wasn't picked up by the sensor in the first place.

     There are limits in the sensor, and there are also limits in getting the data off the sensor. That's true for biology and hardware.

    I'm pretty excited to get my hands of Quantum Dot cameras for example, but even they don't have the same Dynamic Range as the human eye, but could probably beat it with clever systems engineering.

Human "dynamic range" is a hack - the limits look impressive, but they're only reachable through a lengthy adaptation process. Modern camera sensors with HDR, backside illumination and analog gain controls get pretty close to that - and much, much faster.

In practice, if you want really good performance across an extreme dynamic range, it's usually easier to stack multiple specialized sensors within a single sensor unit. A filter-free reduced resolution sensor for high QE in low light conditions (or at high framerates), and a daylight-optimized sensor that gets high resolution, full Bayer, IR-cut and a polarized glare filter to get you as much data as it can in high light environments. This gives you straight up superhuman range with no adaptation time.

Do the raw low light images look nice? No. But human brain itself takes a lot of postprocessing for "looks nice" to happen. Salvaging good images from series of noisy measurements is an AI problem. We've had some pretty capable sensors for a while now, but it took AI-powered computational photography to truly make low light photography happen on a smartphone.

I'm aware of the "detection of individual photons at above-chance rates" experiments with human vision. But modern backside illuminated CMOS, if plugged into the same setup, would also be able to detect photons at above-chance rates. Not to mention the actual photon event cameras. Very niche items, but they can detect photons pretty reliably - not merely "more likely to be right than wrong".

There are also thermal imagers and image intensifier stages, if you want to take the "niche extreme sensors" route. Not commonly done. But goes to show just how much ass engineered sensors can kick.

The limits aren't in the sensors - it's in making sense of the data they give you.

You're overstating how well wrist cams can compensate for a lack of touch. It's really hard to get the instant estimate of weight and rigidity we humans get when picking up an object with a wrist camera, especially for rigid end effectors. They can guide a grasp in ways that would require touch for a human and help detect that you've dropped an object but don't compensate for many aspects of grasp that prevent the drop in the first place.

"Estimate of weight and rigidity" is exactly the kind of thing that's performed rather well by low-dimensional sensing of actuators themselves.

Backdrivable actuators know how much current are they sinking - and thus, how much force are they facing. So even the simplest two finger gripper can easily "know" that one object is squishy and the other is not. The response curves are different. Same can apply to arm actuators for weight estimation.

Now, what this doesn't actually get you is an estimate of how hot/cold an object is, and how moist an object is.

Where cameras come in is specifically the "guide a grasp" part. Force sensing alone can tell a robot "the grasp is failing". A camera pointed at the grasped object can tell "why".

In theory yes but in practice backlash and stiction make drive currents much less useful than the raw force at point of contact for any realistic gearing. And an important part of weight estimation is that force lateral to the point of contact. Now, your arm motors will be drawing more current as they move so again in theory you can distinguish that, you've still got the weight of the whole gripper added in.

I spent years of my professional life on these topics[1] and from my experience motor drive current usually isn't enough to narrow down the weight of an object in a typical picking task even to within a factor of 2 to detect when you've accidentally picked up two items from a bin instead of one.

[1]https://patents.google.com/patent/US12612266

Oh, it gets extremely annoying when you try to use those kinds of indirect, noisy, smeared, nonlinear, relative measurements for absolute sensing.

The power of modern AI robotics is that, much like the biological mess it's trying to replicate, it doesn't care that much about absolute sensing.

Imperfect sensors fused by a high performance control model with a good foundation of world dynamics and self dynamics? Gets you impressively close to "perfect sensors" in practice. Being able to take a big bundle of noisy nonlinear ambiguous measurements, and implicitly reference them off each other and over time to get to a usable control loop is powerful.

It's the kind of power human nervous system wields to force the horrid mess that is the human body to perform.

For "detect when you've accidentally picked up two items from a bin instead of one", a wrist camera is a far more natural sensor than a bundle of actuators - especially for lightweight items. Wrist cameras are awesome. But even a bundle of actuators, if fused well enough, can often do it.

The problem is that designing algorithms to sensor fuse to this level by hand is pretty much a no-go. The kind of sensor wrangling hell I would only wish upon an enemy. Which gets us right back to general purpose robotics being an AI problem at its core.

Animal olfactory sensing is more or less just detecting the presence of certain molecules in the air.

We already have devices that do that better than humans and even ones that detect things we can’t smell (carbon monoxide, for example).

Though I suppose you could say that “smelling” is the result of the synthesis of the accumulated signals from the olfactory system in the brain, and more than merely sensing their presence.

Sure, olfactory sensing sensing is "just" sensing of volatile molecules. And we can build plenty of different molecular sensors. Including for things humans can't detect naturally.

But, just like being able to build a load cell and a thermometer doesn't translate to "skin-like sensing", we can't go from "chemical sensors" directly to smell.

For one, things like sensor diversity, density, longevity and manufacturability matter a whole lot. Can you give a robot a weather station type "pressure + temperature + humidity + PM etc" air quality sensor? Yes. Can you give that robot a sensor with enough molecular coverage and fidelity to replace the human sense of smell? Almost certainly not. Can you even get close on a sensible budget? Not with the tech we have.

Getting a system to perform molecular sensing with discrimination against many molecule classes - the kind of operation human smell performs - is very much not trivial.

Now, whether that's worth pursuing at all - I'm still leaning towards a "no". It's a very niche capability.

Sensors on industrial robots aren't THAT great. Robots people seem to use torque sensors on both sides of a motor, meaning they need all of input, output, and commanded torque to know the actual torque value that the software needs. Things like that. I think GP's point stands.

Exactly. I suspect the problem isn't just the amount of training data, but how much of the relevant information is missing from it.

I read once analysing EEG (brain electrical activity) data is like standing outside a football stadium and trying to understand what the coach is saying from the roars of the crowd.

> Our own nerves produce a huge amount of data.

I wonder if collection of training data is going to be a purpose for Neurallink.

IMO current robots must be still fundamentally flawed in the architecture somewhere. We humans seem to operate on commanded force, but robots primarily work on commanded joint angles and speeds.

I realized this when I woke up in a odd posture and one of my arms was completely numb. I could still "throw" the arm like a lasso with squeezable fingers at the end and use visual feedback to achieve my goals, with only tiny amounts of actuator feedback or control. A robot working on joint angles won't be able to do that; therefore IMO the force angles should be more often looked into.

Another pet peeve of mine about robots is that extremely few robots even attempt to have an anatomically correct upper torso and especially shoulders! The upper torso do have limited snake-like mobilities, and the shoulder complex(as doctors call it) is an amazing thing and not just a ball joint or orthogonal 2-axis rotary joints.

Not exactly sure what that means. Your brain also has to figure out angles, but your cerebellum does that in the background and you don't notice. You mostly need to figure out where you want your hand to be, and the appropriate muscles will be recruited to move in just the exact amount.

But that's exactly what inverse kinematics is.

It just feels to myself that the way I energize my muscles to achieve desired effects to be more influenced by something like a total force budget distributed using a table of channels with multipliers or something using forward kinematics, helped by joint structures that has non-uniform force-angle relationships, than actual IK.

To demonstrate: put up an image of a protractor onscreen, and try moving your index finger in say 10 degrees increment from 0 to 90 degrees. That isn't going to be precise enough to hold an egg without cracking. It has to be trivial and precise to fractions of degrees if we were doing IK all the time. The IK theory just doesn't feel right(though IK as a technology is obviously extremely useful).

Also forgot to say in GP: our default coordination system must be screen space 2D, with 3D perception being an option. People swing arms outwards against annoyances. If it was default 3D, we should be constantly pushing forward in depth directions.

Here is a good video on the subject by nvidia, very fun prez:

https://www.youtube.com/watch?v=3Y8aq_ofEVs&pp=ygUKbnZpZGlhI...

PS: It's sad that robotic does not seems to get a lot of traction on HN :(

Feedback loops are too long with anything real worldly

The feed back loop is getting reduced by those new ways of training, simulation and video generation with realistic physics is kind of changing the game.

- https://www.nvidia.com/fr-fr/ai/cosmos/

- https://dreamzero0.github.io/

- https://generalistai.com/blog/gen-1.5

Can always skip the real world for training and evolve the controller in simulation.

Currently working on modernizing SIGEL (2000-era project at the University of Dortmund), which uses genetic programming to evolve walking programs for simulated robots in a physics engine.

Some of the 'evolved' walkers so far can be found at youtube.com/@virtualspecies

Differnent kind of beast entirely compared to what NVIDIA is doing, but then again it's fun to see what evolution comes up with - and it works on humble hardware too

robotic is waiting for its chatgpt moment

You do not think that the chatgpt moment is happening right now ?

Is it? Genuinely asking.

Like said by others, the sensor problem is a real one, and personally I don't see a robotic hand matching the dexterity of a human one, within a viable economic cost and sufficient robustness, before a long time (but I could be wrong ).

But for the part that concern physical understanding of the world, adaptability, and basic manipulation, it feels like the field is going through a break through.

The implication might be the apparition of generalist robots, that could perform (in the limit of their dexterity) on a wide range of tasks, commanded by natural language.

Sure, not on pair with human dexterity yet, but as a thought exercise, think of all the things you could do with simple claws, or heavy glove. What if a robot was able to do the same ?

Here is a set of links to feed your though:

- https://www.youtube.com/watch?v=3Y8aq_ofEVs

- https://generalistai.com/blog/gen-1.5

- https://dreamzero0.github.io/

"Matching the dexterity" is a false goal. It's focusing on the wrong things.

As teleoperation readily shows, even a very dumb primitive hand can be salvaged if you get a very advanced control policy to drive it.

Sure, it would still take a lot longer for a well tuned robot with just a pair of UMI-style grippers to tie shoe laces than it would a practiced human. But a robot doesn't have to work faster than a human. "It can do the same task for cheaper" is often good enough.

The right thing to focus on is: AI. We already have robot bodies that are good enough to do many economically valuable tasks. We don't have the AI that would make those bodies perform to their potential. Modern robotics is an AI problem above all.

There are such things as established robotics giants, supplying robotic arms to factories like car plants that put together giant metal panels together and junk food industries assembling muffins into plastic trays and boxing up.

Those companies seem to be rushing to get NVIDIA Isaac systems and some even buying DGX pod hardware to get those systems and processes integrated into existing factories to avoid getting disrupted by startups. I guess those + AI companies like Google applying LLM derived technologies to robotics is what GP is referring to.

It's a really good article, thanks!

Reply on news.ycombinator.com