Hacker News Reader

Humans have better dynamic range for hearing and vision compared to any sensor I'm familiar with.

Human vision is also extraordinary sensitive, with some people being able to detect individual photons with their rod cells.

Human "dynamic range" is a hack - the limits look impressive, but they're only reachable through a lengthy adaptation process. Modern camera sensors with HDR, backside illumination and analog gain controls get pretty close to that - and much, much faster.

In practice, if you want really good performance across an extreme dynamic range, it's usually easier to stack multiple specialized sensors within a single sensor unit. A filter-free reduced resolution sensor for high QE in low light conditions (or at high framerates), and a daylight-optimized sensor that gets high resolution, full Bayer, IR-cut and a polarized glare filter to get you as much data as it can in high light environments. This gives you straight up superhuman range with no adaptation time.

Do the raw low light images look nice? No. But human brain itself takes a lot of postprocessing for "looks nice" to happen. Salvaging good images from series of noisy measurements is an AI problem. We've had some pretty capable sensors for a while now, but it took AI-powered computational photography to truly make low light photography happen on a smartphone.

I'm aware of the "detection of individual photons at above-chance rates" experiments with human vision. But modern backside illuminated CMOS, if plugged into the same setup, would also be able to detect photons at above-chance rates. Not to mention the actual photon event cameras. Very niche items, but they can detect photons pretty reliably - not merely "more likely to be right than wrong".

There are also thermal imagers and image intensifier stages, if you want to take the "niche extreme sensors" route. Not commonly done. But goes to show just how much ass engineered sensors can kick.

The limits aren't in the sensors - it's in making sense of the data they give you.

Your reply to my comment downstairs got chewed up by HN. Maybe some sort of LLM detector thought you're speaking too much like an AI?

The reason why I'm not "team sensors" is that we're nowhere near to being able to as much as squeeze out even one mid-range camera sensor for all its worth. Things like AI assisted scene reconstruction show just how much data is there in even a small series of photos - when the AI is there to back the raw measurements with sensible assumptions.

The difference between "a plausible reconstruction" and "an accurate measurement" is academical rather than practical. You never get perfectly accurate measurements (because real world sucks), so a well designed "plausible reconstruction" will often have better error averages AND bounds than a naive measurements-first pipeline - simply by taking better advantage of spatial and temporal redundancies. Adding more assumptions to the pipe from there gives you even better performance.

I expect approximately none of the near-term gains in robotics performance to come from the sensors themselves. There's still room for improvements there, mind. There is some cool sensor tech that could result in a better performance ceiling. But we're nowhere near that "sensor ceiling".

Today's robotics sensors are so far ahead of what robotics AI can reliably utilize that trying to push for "we need better sensors" is a mistake in nearly every case.

I agree with everything you state about squeezing data out of sensors.

But, I also run into a LOT of cases where I need better sensors too. I just don't see them as mutually exclusive.

I did stick my comment into an LLM, annoying, I hand wrote the entire comment, and had it fact check my statements. Here is my original unmodified comment:

    Eh, there's too much here for me to give a proper reply.

    I agree with a lot of what you said, and I'm on team sensor, but I've been studying how the eye works and processes info.

    A *system* of many sensors can cover larger ground for sure... But I'm very interested in compact systems that can be better than the eye or ear, I know we can do it, I just haven't seen it yet.

     Also, AI and/or filtering can't give you data that wasn't picked up by the sensor in the first place.

     There are limits in the sensor, and there are also limits in getting the data off the sensor. That's true for biology and hardware.

    I'm pretty excited to get my hands of Quantum Dot cameras for example, but even they don't have the same Dynamic Range as the human eye, but could probably beat it with clever systems engineering.

You're overstating how well wrist cams can compensate for a lack of touch. It's really hard to get the instant estimate of weight and rigidity we humans get when picking up an object with a wrist camera, especially for rigid end effectors. They can guide a grasp in ways that would require touch for a human and help detect that you've dropped an object but don't compensate for many aspects of grasp that prevent the drop in the first place.

"Estimate of weight and rigidity" is exactly the kind of thing that's performed rather well by low-dimensional sensing of actuators themselves.

Backdrivable actuators know how much current are they sinking - and thus, how much force are they facing. So even the simplest two finger gripper can easily "know" that one object is squishy and the other is not. The response curves are different. Same can apply to arm actuators for weight estimation.

Now, what this doesn't actually get you is an estimate of how hot/cold an object is, and how moist an object is.

Where cameras come in is specifically the "guide a grasp" part. Force sensing alone can tell a robot "the grasp is failing". A camera pointed at the grasped object can tell "why".

In theory yes but in practice backlash and stiction make drive currents much less useful than the raw force at point of contact for any realistic gearing. And an important part of weight estimation is that force lateral to the point of contact. Now, your arm motors will be drawing more current as they move so again in theory you can distinguish that, you've still got the weight of the whole gripper added in.

I spent years of my professional life on these topics[1] and from my experience motor drive current usually isn't enough to narrow down the weight of an object in a typical picking task even to within a factor of 2 to detect when you've accidentally picked up two items from a bin instead of one.

[1]https://patents.google.com/patent/US12612266

Oh, it gets extremely annoying when you try to use those kinds of indirect, noisy, smeared, nonlinear, relative measurements for absolute sensing.

The power of modern AI robotics is that, much like the biological mess it's trying to replicate, it doesn't care that much about absolute sensing.

Imperfect sensors fused by a high performance control model with a good foundation of world dynamics and self dynamics? Gets you impressively close to "perfect sensors" in practice. Being able to take a big bundle of noisy nonlinear ambiguous measurements, and implicitly reference them off each other and over time to get to a usable control loop is powerful.

It's the kind of power human nervous system wields to force the horrid mess that is the human body to perform.

For "detect when you've accidentally picked up two items from a bin instead of one", a wrist camera is a far more natural sensor than a bundle of actuators - especially for lightweight items. Wrist cameras are awesome. But even a bundle of actuators, if fused well enough, can often do it.

The problem is that designing algorithms to sensor fuse to this level by hand is pretty much a no-go. The kind of sensor wrangling hell I would only wish upon an enemy. Which gets us right back to general purpose robotics being an AI problem at its core.

Animal olfactory sensing is more or less just detecting the presence of certain molecules in the air.

We already have devices that do that better than humans and even ones that detect things we can’t smell (carbon monoxide, for example).

Though I suppose you could say that “smelling” is the result of the synthesis of the accumulated signals from the olfactory system in the brain, and more than merely sensing their presence.

Sure, olfactory sensing sensing is "just" sensing of volatile molecules. And we can build plenty of different molecular sensors. Including for things humans can't detect naturally.

But, just like being able to build a load cell and a thermometer doesn't translate to "skin-like sensing", we can't go from "chemical sensors" directly to smell.

For one, things like sensor diversity, density, longevity and manufacturability matter a whole lot. Can you give a robot a weather station type "pressure + temperature + humidity + PM etc" air quality sensor? Yes. Can you give that robot a sensor with enough molecular coverage and fidelity to replace the human sense of smell? Almost certainly not. Can you even get close on a sensible budget? Not with the tech we have.

Getting a system to perform molecular sensing with discrimination against many molecule classes - the kind of operation human smell performs - is very much not trivial.

Now, whether that's worth pursuing at all - I'm still leaning towards a "no". It's a very niche capability.

Sensors on industrial robots aren't THAT great. Robots people seem to use torque sensors on both sides of a motor, meaning they need all of input, output, and commanded torque to know the actual torque value that the software needs. Things like that. I think GP's point stands.