Heat-assisted detection and ranging | Hacker News Reader

Heat-assisted detection and ranging

nature.com

48 pointsbookofjoe28 comments

Open articleView on HN

For some pictures

https://github.com/FanglinBao/HADAR

and videos

https://purdue0-my.sharepoint.com/personal/baof_purdue_edu/_...

Behind Microsoft paywall.

What? I’m not even logged in and can see it. Are you under the impression you need to pay for a GitHub account to view things hosted on it?

You don't think they're talking about the sharepoint.com URL?

Well, it's also same for me and it's asking for €24.99 for access!

Just because nobody made me sign an NDA for this: there are satellite companies using this for finding other satellites in eclipse. It's a pretty brilliant strategy if you want to dock with something and there's no usable visible light. You can make markings that appear clearly in LWIR by making dark parts shiny (see mostly empty space reflected) and light parts matte (see the thermal emissions of the spacecraft)

This is a common approach - besides being robust to lighting conditions, it's less sensitive to glinting.

Most IR cameras work with IR as heat emission. They are useful in the dark, without revealing the camera.

Most optical cameras work with reflected light, not direct emission. The source of light is usually away and at an angle; this allows for shadows, textures, etc.

I don't see why an IR camera can't work the same way as an optical camera, if reflected IR is used; IR photos under e.g. sunlight look fine enough.

Radars or lidars work with their own scanning sources of illumination: microwave, IR, or optical. I wonder if the HADAR uses its own source of IR; if it does, how different it from a LIDAR?

(All these mysteries can be solved for $29, but I'm not yet curious enough.)

All IR cameras catch also reflected IR. Thats why you have to know the emissivity to calculate the true temperature.

Yeah go play with an LWIR camera of you really want to understand the spectrum. For instance, everything becomes a mirror, you can see yourself standing in front of surfaces that have better than ~5um surface finish (quite rough sheet metal and such) and clouds appear hot-ish because while they are themselves quite cold, they're reflecting LWIR from the ground, hence why overcast nights are much warmer.

https://hal.cse.msu.edu/assets/pdfs/papers/2023-nature-heat-...

Thanks! This clarifies a lot.

- They use and combine both optical and LWIR data.

- They are able to split the IR signal into emissive and reflected, and thus measure the temperature and extract small variations (texture).

- They can get higher-res images using LWIR, and thus do (more) precise ranging using a stereo pair.

- Using the spectral information + some ML models, they are able to identify materials, and paint / augment pictures using this information.

That's actually impressive.

As a note, try to avoid using IR to describe LWIR, there are many IR sub bands (NIR, SWIR, MWIR, LWIR) and your phone camera can detect at least one of them.

I can't tell what they are doing. It seems like it's merging visible and thermal imaging to improve segmentation and identification, then looking up the identified objects for ranging. No?

The thermal images in their examples seem to be intentionally using a wide temperature gamut to reduce the contrast in the image (and make it look bad). Most thermal imaging solutions will automatically adjust and give you a much better result.

Regardless I can see lots of benefits of integrating the two as basically another color channel. There's a lot of information available.

The short of it is stereovision using hyperspectral thermal cameras, but the long answer is a bit more interesting.

The problem with doing stereoimaging with thermal radiation is that almost all objects are glowing due to radiation emission which blurs out the 3d texture of the object. The example they give is if you have a lighbulb with some texture, you can't see it when it's on(you might experiment with this in various 3d engines). This makes it difficult to do stereoimaging because everything is so blurry.

To do this they develop a way to deconvolute the temperature, emissivity, and light reflected by the object(they call this texture). They assume that the much of the reflected light comes from a couple objects in the scene with known properties such as the sky and ground which allows them to get an estimate of surface normal and do something like inverse ray tracing to estimate the light reflected by the object.

They also estimate the emissivity by comparing the spectra to a library of spectra for different materials. I think they use this as part a second step in their inverse ray tracing to work out reflected light and temperature of objects. Much of this is detailed in the supplementary information[0].

It's interesting because they have a passive sensor for depth which works at night, but also estimates what objects are made of. It may degrade when there are fires which may break some of the assumptions they make to estimate reflection or spills of IR absorbing substances. Gasoline, CO2, methane, and other gases can strongly absorb in the infrared spectrum. Which may interfere with temperature and material estimates.

[0]https://www.nature.com/articles/s41586-023-06174-6#Sec24

Sounds similar to IRST (Infrared Search & Track) as used in combat aircraft.

To me, "ranging" means measuring the range to the target directly, not estimating it from other measurement types (bearings, for example). Can someone explain if that's what the authors are doing here? Is it analogous to stadiametric ranging? I haven't read through this in depth, but I can't figure out what their method is.

I don't have access, what's the gist? Heat emission then timing combined with thermal imaging? AFAIK, most *AR methods are active and rely on timing, but the abstract mentions passive detection.

I am very disappointed that unlike radar, lidar, and sonar, this "HADAR" technology is not an active-sensing technology, but just relies on passive IR emissions. I was imagining a car driving around with a big open flame at the front like something out of Mad Max.

Somebody is prepping for Burning Man I take it

All I'm saying is that if we're going to have killer robot cars roaming the streets and interfering with firefighters [1], we might as well do it properly.

[1] https://www.theguardian.com/us-news/2023/jun/23/self-driving...

Interfering with firefighters is a bit different from driving around and starting fires though, init?

IR floodlights mounted to the front of trucks is a thing. Here's an example with photos of infrared lights used on a smuggling vehicle so the driver could turn the headlights off and drive by night vision: https://youtu.be/a8Q9Yibblbc?t=626

"Our work leads to a disruptive technology that can accelerate the Fourth Industrial Revolution (Industry 4.0) with HADAR-based autonomous navigation and human–robot social interactions."

Really, though?

These industrial revolutions are coming faster and faster, and somehow feel less transformative with each unveiling. I only know arguably two. I have no idea what the third, let alone the fourth was.

Industrial, chemical, and information.

First: powered tools, mechanised factories, basic but modern chemical processes.

Second: mass production, interchangeable parts, reliable steel, telegraph and other basic uses of electricity.

Third: computers and everything related to them.

Fourth: all the buzzwords and not much substantial at this time — though when the dust settles, I won't be surprised if at least a few currently popular things are still seen as relevant and not merely flash-in-the-pan cultural artefacts like the 18th Amendment, patent medicine, or Spiritualism.

Jeez... that reads like poorly written PR.

It doesn't belong in academic work, no matter how impressive the authors think it may be.

Reply on news.ycombinator.com