frizensami

Ah, I see. Thank you!

Yes, I agree. I believe military-grade optics detection systems use multiple wavelengths, and some even use short bursts of visible light, making it harder to block.

Interesting question. Mainly as mentioned above we needed Google Play Services for AR. Nothing else stands out at a high level.

However there is a clear bottleneck with the Google AR team for features that the community really wants. While it's understandable, perhaps the open source libraries aren't the priority or there's short staffing, there are issues like this one - https://github.com/google-ar/arcore-android-sdk/issues/153 - that have been open since Jan 2018. This one asks that we can use the smartphone flashlight simultaneously with ARCore, but there's no visibility on how close we are at the moment. This feature would have likely improved our work's performance greatly, but even in general, many other AR developers are asking for it.

Re: overcoming it, there's not much we can do in this case. We just didn't implement the feature.

Oh, I must have misunderstand, since the root comment was talking about detection methods. But yes, jamming is a good option, and there have been papers to make areas "unrecordable". I don't have my citation manager on hand now, but I think it was an early ACM Sensys / Mobisys research work.

I'm not sure that it's a conclusive yes here. We have a procedure to remove the effects of background reflection saturation by constantly analysing the maximum reflection size. So if there's a large reflection in the scene, we walk the user back til we have all reflections at the same size or smaller than the the larger expected hidden camera reflection. This works fairly well.

However, if the camera reflection is always within a larger reflection for all possible distances and angles, then I doubt this will work.

Funnily enough this is exactly how we started the project, just with a different Thinkpad model.

Separately, Lubos, the creator of ToF Viewer, is very helpful to the ToF community and constantly answers any questions about APIs or sample code. I'd recommend talking to him if anyone wants to get started with ToF-based applications.

Perhaps you could try this direct PDF link at https://dl.acm.org/doi/pdf/10.1145/3485730.3485941. Hopefully that works.

Agreed. I expect however that any easily available thermal sensing, especially on smartphones (see FLIR, etc), could be easily defeated with a strong adversary doing good thermal management.

As you say though, extra heat is a physical guarantee, so maybe a smarter technique exists to separate signal from noise in the thermal domain that I don't know of yet.

There's a lot of marketing involved in this naming, with Samsung calling it DepthVision and Apple calling it LIDAR. There may be a difference here, however. My understanding is that Apple LIDAR is doing what we call "direct" ToF, where the round trip travel time of laser pulses is actually measured (this can be in nanoseconds). This lines up with what self driving car (and other expensive) LIDARs do.

Most other ToF sensors use "indirect" ToF, where they measure the phase difference between incoming and outgoing signals to derive distance.

However, it gets murky as cheap 2D LIDARs on say, robot vacuum cleaners, use geometric techniques to find distance (basically return angle of a reflection). I explored this in a previous work.

TLDR: I would recommend not taking any naming at face value and reading the actual datasheet or more commonly, technical marketing materials, since few ToF manufacturers that I see have a public datasheet.

I don't think so. There's "laser-assisted autofocus" which might be slang for an actual ToF sensor, but sometimes that's just a single laser pulse (I.e., no 2D image that we need).

Agreed, and a very valid point that these off the shelf detectors are pretty cheap compared to ToF smartphones. There are a number of unfavourable things about the detectors though:

- No automated detection, so lots of human subjectiveness on whether a bright spot should be investigated

- Background lighting can be an issue, so the view through the detector can be filled with bright reflections

- When I tried this for longer than 30 seconds, there's really some eye strain involved

The idea is that the smartphone has more than this one purpose, and the detection system can improve with software updates, whereas you get what you see with the detector and nothing more.

It could also possibly work on iOS, but there's a catch we have not yet explored. We're relying on something called the "confidence" score in the ToF camera API, which has 8 possible values (3 bits) in Android, but from what I last saw, has only 3 possible values total in iOS. It's not clear how this lack of info exposed by the APi will affect accuracy.

This is something we'd have to separately test, and ultimately make a wholly different app for if it's feasible on iOS.

I don't think this works, mainly because an extremely bright external source of 850 nm light looks incredibly suspicious through the sensor (as we expect the ToF sensor to be the only light source). We could add that check in explicitly.

Thank you! Hoping for higher resolutions soon.

Thats very interesting, first time hearing of it. Not sure, really depends on what kind of reflection it generates when light hits it, and because of the unique design I'm not sure what that will be.

Yes, perhaps. At the very least, adding a variable shutter and extra logic will drive up the price of the camera, which will be some consolation.

Thank you! Actually, my understanding when I started this project was that I would get a 640 x 480 image (IMX516 sensor). However, I could only get a 320 x 240 image from the sensor through the Android API, so that was a bit of an oddity.

But at least for the larger camera case, we can just increase one of the filtering thresholds for reflection size, which is already doable on the UI

The free option: use your smartphone's flashlight and try to spot unnaturally bright / colored reflection.

If you want to spend some money and don't mind carrying an extra device: probably any of the "hidden camera detectors" on the market will be at least somewhat useful. K18, CC308+, etc.

We're trying with our work to get better or at least equivalent results without having to use external devices.

So far the biggest issue is if the camera is angled oddly away from the user (basically, outside the 20 degree observable FoV). Another issue would be if they manage to install a larger camera that returns larger reflections (which we filter out).

Regarding using the app to check, I guess that applies for the existing handheld detectors as well. It's definitely something that intelligent attackers can try to plan for, but we havent tested the adversarial robustness of the system right now. That would be a very interesting direction for us as well.

That's a great question. I'm not totally sure if this is possible. If it was, I think one-way mirrors would use this technology. For now, it seems like they only work because they only let 50% of the light through + there's a brightness differential between both sides. Perhaps someone more informed about this could chime in.

While the broader technique should work for those applications, the platform (smartphone ToF sensors) probably won't. Smartphone ToF sensors have a pretty limited range. We were only able to detect cameras within 1 metre of the smartphone because we're really using the hardware for something it's not intended for.

For the applications you suggest, there are some existing military-looking devices out there that use multiple lasers to find sniper scopes, for example. My basic searching shows at least https://www.ldsystems.us/product/sniper-optics-detector/#, though I'm sure there's more.

I would love to, but I'm not sure how yet without a major re-write. The augmented reality code we completely rely on is part of Google Play Services for AR. I definitely do understand the benefit of degoogling, so maybe it can be a community effort once I opensource it.

Also I appreciate the interest, thank you. I'm heading back from the conference over the next few days, so I should be more free to take a look at it soon.

It's definitely something I'm considering. The only hard blocker now is a hashmap that checks for our test phones' unique ID and applies a fixed transform to align both cameras. Right now, any other phones will cause an immediate crash. Minimally I think we need to disable this and maybe include a small UI to let users put in the alignment parameters (just a scale + offset).

That's an interesting idea: trying to create so many false positives that the user gives up. We're already removing around 100+ false positives per frame at the moment in difficult cases, so perhaps it's possible to overwhelm the filters with very maliciously designed environments.

I think the easier way would be to hide the cameras in much harder-to-reach places so that it's inconvenient for the user to get their smartphone near. This might reduce the kind of videos that can be taken, but maybe an attacker will find that a reasonable tradeoff.

High priced flagships have them but there are also a few midrange phones like the Huawei P30 Pro that have them too. The trend seems pretty positive towards more ToF sensors in phones, especially because Apple has had them for two iPhone Pros in a row now.

They're basically for augmented reality applications because they sense depth. Placing objects at the right size and scale in the augmented view is much easier and more accurate with the ToF sensors, for instance.

Pretty interesting idea. In fact, there are a number of military-related systems to spot sniper scopes (generally optics) with lasers, which I expect is similar to what they used here.

Yep, the combination is highly reflective. I cited another work (by a physics research group) in a comment that modelled and tested this effect comprehensively.

Thank you!

While this work operates on ToF sensors that are already present in smartphones (e.g., Samsung S20+/Ultra), a Microsoft/Azure Kinect should also be a valid option because it has a depth camera as well. It has a higher resolution and bit-depth as well.

We initially intended to compare against the Kinect, but it doesn't fit the use case (something that you can have on you at all times). However, it could be a cheap choice for a different kind of deployment (automated hidden camera detection with robots, perhaps?)