Humans have better dynamic range for hearing and vision compared to any sensor I'm familiar with.
Human vision is also extraordinary sensitive, with some people being able to detect individual photons with their rod cells.
Human vision is also extraordinary sensitive, with some people being able to detect individual photons with their rod cells.
The reason why I'm not "team sensors" is that we're nowhere near to being able to as much as squeeze out even one mid-range camera sensor for all its worth. Things like AI assisted scene reconstruction show just how much data is there in even a small series of photos - when the AI is there to back the raw measurements with sensible assumptions.
The difference between "a plausible reconstruction" and "an accurate measurement" is academical rather than practical. You never get perfectly accurate measurements (because real world sucks), so a well designed "plausible reconstruction" will often have better error averages AND bounds than a naive measurements-first pipeline - simply by taking better advantage of spatial and temporal redundancies. Adding more assumptions to the pipe from there gives you even better performance.
I expect approximately none of the near-term gains in robotics performance to come from the sensors themselves. There's still room for improvements there, mind. There is some cool sensor tech that could result in a better performance ceiling. But we're nowhere near that "sensor ceiling".
Today's robotics sensors are so far ahead of what robotics AI can reliably utilize that trying to push for "we need better sensors" is a mistake in nearly every case.
But, I also run into a LOT of cases where I need better sensors too. I just don't see them as mutually exclusive.
I did stick my comment into an LLM, annoying, I hand wrote the entire comment, and had it fact check my statements. Here is my original unmodified comment:
Eh, there's too much here for me to give a proper reply.
I agree with a lot of what you said, and I'm on team sensor, but I've been studying how the eye works and processes info.
A *system* of many sensors can cover larger ground for sure... But I'm very interested in compact systems that can be better than the eye or ear, I know we can do it, I just haven't seen it yet.
Also, AI and/or filtering can't give you data that wasn't picked up by the sensor in the first place.
There are limits in the sensor, and there are also limits in getting the data off the sensor. That's true for biology and hardware.
I'm pretty excited to get my hands of Quantum Dot cameras for example, but even they don't have the same Dynamic Range as the human eye, but could probably beat it with clever systems engineering.In practice, if you want really good performance across an extreme dynamic range, it's usually easier to stack multiple specialized sensors within a single sensor unit. A filter-free reduced resolution sensor for high QE in low light conditions (or at high framerates), and a daylight-optimized sensor that gets high resolution, full Bayer, IR-cut and a polarized glare filter to get you as much data as it can in high light environments. This gives you straight up superhuman range with no adaptation time.
Do the raw low light images look nice? No. But human brain itself takes a lot of postprocessing for "looks nice" to happen. Salvaging good images from series of noisy measurements is an AI problem. We've had some pretty capable sensors for a while now, but it took AI-powered computational photography to truly make low light photography happen on a smartphone.
I'm aware of the "detection of individual photons at above-chance rates" experiments with human vision. But modern backside illuminated CMOS, if plugged into the same setup, would also be able to detect photons at above-chance rates. Not to mention the actual photon event cameras. Very niche items, but they can detect photons pretty reliably - not merely "more likely to be right than wrong".
There are also thermal imagers and image intensifier stages, if you want to take the "niche extreme sensors" route. Not commonly done. But goes to show just how much ass engineered sensors can kick.
The limits aren't in the sensors - it's in making sense of the data they give you.