Interview: Apple’s Craig Federighi about Face ID
techcrunch.com
techcrunch.com
See I don't doubt that FaceID will be good. I don't doubt the quality Apple provides when they introduce features like this. I'm sure it will work amazingly well, because unlike almost all previous implementations Apple actually makes purposeful hardware for it.
The thing is, I don't want to look at my phone to unlock it. With a fingerprint reader - on my current Xperia and my current iPhone - my phone is unlocked before it faces me. I unlock it when I get it out of my pocket. I unlock it while lying on the table when it's facing the ceiling. I unlock it to peek at messages in meetings under the table. Needing to face your phone to unlock it seems to just be a really weird concession to me. As others point out, it also seems to be a minor concession security wise. It seems to be some typical form-over-function Apple thing again - UX and security concessions for a better looking phone. They should've slapped a fingerprint reader on the back once it turned out the under-the-screen fingerprint reader wasn't viable.
Having to go through some ritual to face your phone and slide it open to unlock feels like a hassle and a downgrade. Especially on a €1000+ phone. So yeah, the Apple distortion field is still a real thing it seems. Or maybe the way I use my phone is weird. But either way, I don't get it.
I really don't see what's so bad. To use my phone I have to look at it. So as long as it unlocks fast enough (reportedly it does) them it's a non-issue.
> I unlock it when I get it out of my pocket.
Once you look at it, it will be unlocked.
> I unlock it while lying on the table when it's facing the ceiling.
Depending on how you're positioned this maybe an issue.
> I unlock it to peek at messages in meetings under the table.
If you can look at the screen from a reasonable angle then it can see you and unlock.
I wish people would stop bashing this stuff 15m after it's announced in the presentation. Let's see what the security white paper says. Let's see what reviewers who have been allowed to use it say (they couldn't enroll at the event).
Until then this is all straw man arguments (on both sides, my comments are based on Apple reasonably delivering what they promise).
But this is much slower and are far less flexible than picking your phone out of your pocket, or from anywhere else, with your finger on the sensor. And nobody is mentioning that this is a downgrade.
You see, FaceID is not 100% new. Face unlock existed on the Galaxy Nexus. It also existed in much improved form on the Lumia 950 and Galaxy Note 7. I will 100% grant you that FaceID will be far more accurate still than these implementations. But accuracy is not the issue here: all these methods imply a two step face unlocking ritual, which again is a downgrade over no ritual with a fingerprint scanner.
In the end it seems pretty clear to me that face unlock - in any form - will be far less seamless than finger unlock. It makes your phone slightly less easy to use. Yet Apple seems to be successfully arguing that it's an upgrade or at least an equivalent. But it just can't be, no matter how good their implementation is. That's what I don't get.
That doesn’t dismiss the lock screen, to do that you either have to press the home button or swipe up.
There is no new third step.
2. FaceID is completely different to the Nexus which is based solely on 2D image recognition and is easily able to be forged and doesn't work in low light conditions.
3. From what Craig said in the Daring Fireball interview it will actually be much faster than TouchID since the authentication apparently happens in parallel to the unlock operation.
Let me clarify for the third time. My opinion is the following: no face unlock mechanism can be as good as a fingerprint unlock for the flexibility and speed of use as explained above. FaceID will likely be far and away the best face unlock method we've ever seen but it's still a face unlock method and therefore slower and more restrictive than a fingerprint unlock. What I don't get is that nobody else is talking about it, because Apple has successfully spun the discussion to be about the quality of FaceID as opposed to the (lack of) merits for face unlocking on smartphones.
I'd actually be excited to have face unlock on laptops and PCs (actually that exists already as Windows Hello, but dedicated hardware would be nice). But on phones I consider it to be inferior to fingerprints conceptually.
When did you last unlock your phone without subsequently looking at it?
Seems to me FaceID removes the need to first unlock, then use phone. New way is: just use phone. So I expect FaceID to be faster to use.
I unlock my phone and then not look at it when making phone payments (Apple Pay will definitely take a step back with this). I also switch between maps and driving apps in the car while keeping my eyes on the road. I often peek at something on my screen well outside of the FOV of any front camera. My use of my unlocked phone is not at all limited to things I need to take my face's positioning in account for.
I don't particularly appreciate being accused of a failure of imagination and then having to respond to an arbitrary limited assumption of my smartphone use. It's not my imagination that's the problem here.
But the point is moot. My entire issue with it that I don't want to wait to initiate unlocking once it's in front of my face. Even if FaceID is instant, and I would accept that all my use is in the FOV of the front camera, Apple still requires me to slide to unlock the device.
Touch ID lets you do the touch before you look at the phone. The phone unlocks while you take it out of your pocket, so by the time you look at it, it's already unlocked.
Face ID should also significantly improve the UX around all interactions that used Touch ID prior (Apple Pay, authentication for third party apps, etc.).
I don't see why it upsets you that someone doesn not like a product. Every comment does not need to be supported by a scientific study, or even be objective in any way. Your own comment is speculative.
My annoyance is that people have an opinion on something that no one has used.
It’s perfectly fine to have an opinion about the Galaxy S8 without owning one. There were reviews and now there are other owners who you can look to find out how well parts of it work.
With the iPhone X we don’t even have refused to tell us if the feature works or not so complaints about it are pure conjecture.
Those poor objects, won't someone think of the objects?
> If you can look at the screen from a reasonable angle then it can see you and unlock.
You don't need to be able to see the camera to respond to messages in the bottom half of the screen. And even so, larger iPhones already allow double tapping home to bring the top of the screen down for one-handed interaction, which means you can already do everything but camera stuff (right?) with the camera obscured.
But the camera _does_ need to see you in order to unlock in this case. You can't read or reply to a message without unlocking first.
Once you look at it, it will be unlocked.
It's a mistake to underestimate the impact of small delays. Developers ruthlessly cull their editor configurations to avoid little delays here and there. A delay every time you want to use your phone sounds annoying.
There are plenty of contexts I'd like to use my phone where face recognition isn't available: on a bus at night with no light, so as not to disturb others.
1. Indicates it will recognize you in the dark (https://youtu.be/mMV_LcrZtLg?t=21m29s)
2. An accompanying video indicates it works in the dark (https://youtu.be/K4wEI5zhHB0?t=1m47s)
3. The Apple website indicates that "Invisible infrared light helps identify your face even when it’s dark." (https://www.apple.com/iphone-x/#face-id)
4. The parent article indicates "It uses only infrared and existing light, which means it will work in darkness without any more light than is coming off of the phone’s screen."
..that seemingly knowledgable folks can be still confused.
Bringing new tech to market isn't easy, but convincing folks that it's not a rehash of existing tech is hard too.
It would be kinda stupid to introduce a feature that may not work 50% of the time.
> [...] on a bus at night with no light, so as not to disturb others.
That's why the phone has an IR light emitter, so it can work in total darkness. The images in the presentation where the phone lit the person's face up or projected dots on it were simulations to give you an idea what it's doing, but it's all done with IR.
I'd be impressed if this was the case. Looking at a phone <> a phone's camera being able to see your face straight-on.
Face-unlocking is not new, and it's not widely used. If they've managed some physics-defying scanners then maybe it'll be OK.
My guess is FaceID will be the same. You WILL be looking at the phone because you look at the phone to USE it. When you do, it will be unlocked fast enough that you don't think about unlocking (you see the screen transition, but your thoughts will be on using the phone not on the unlock that already happened).
Not saying I can't adjust my behavior. We all have our own way of using our devices.
How can you use your phone without looking at it ?
Because from what Apple has said if you can see the screen the camera will at most angles be able to authenticate you.
In those situations FaceID would be perfect.
But I'm still using a Galaxy S3 so I'm not in the target demographic for sexy new phones.
https://www.forbes.com/sites/josephsteinberg/2015/03/05/why-...
Is it? Before touch sensors became the norm people would put in their PIN every few minutes. It would get picked up by countless security cameras. You could talk to a stranger for 2 minutes and pick up their PIN if you didn't avert your eyes.
And most people didn't use a PIN at all, because it was too much bother.
If someone can shoulder surf while I unlock my phone with a PIN, then they can watch which buttons I press and now they can unlock my phone too. If I use the fingerprint reader to unlock, then they have no chance.
Even if you never use the fingerprint reader, an iPhone with TouchID is still more secure than your Galaxy because the PIN is stored in the Secure Enclave.
TouchID is more conventient than a PIN, and it's no less secure because the fingerprint hash is also stored in the SE chip.
This is another one of those things where Apple marketing fares much better than their competitors :)
But then i am getting old...
I also agree that FaceID might be easier to "socially engineer" or obtain by force in the sense that you could just point it at someone's face or trick them into looking at it. Less so with someone's thumb on the home button.
At the end of the day though, I think these concerns are overstated. No modern security method can guarantee 100% protection - you can watch someone input their PIN, steal their phone and force their face into it, or force someone to put their thumb on it. The last one is the most secure in this sense, for sure, but I'm of the opinion that if you've gotten yourself into one of these situations, this is far from your security bottleneck in the first place.
Someone will inevitably bring up the point of law enforcement / 4th amendment breaches, but there are safeguards with the new iPhone X that circumvent these issues.
And of course, you can always decide to just use a PIN and nothing else :)
Well, you can't actually have it lying flat on the table because the camera sticks out in the back :D
As for opening the home screen, I think your worries are misplaced. You have to look at your phone in order to use it, and FaceID is instantaneous and works at sharp angles, so there's no need to unlock it ahead of time. All it does is replace the button press with a swipe.
The early reports are that it doesn't feel like a ritual at all. It's more like it just makes the whole authentication step go away.[1]
[1] "you get used to not thinking about it very quickly, and when you go back to a Touch ID device, it feels broken that you have to touch the button to unlock the device." https://daringfireball.net/2017/09/iphone_x_event_thoughts_a...
In the end apple will lead people will follow. Clearly they see face id as the way of the future my money is on apple getting their way.
How about using it in a room with over 100F temperature. Hopefully nobody endures that sort of conditions on a regular basis anymore, use the fallback mechanisms for those situations. Except cars of course -- those without AC.
I've heard that people go outside occasionally. Don't know, why, either. :)
I imagine that authorities will catch on pretty quickly: "keep your hands up and away from your pockets while we retrieve your device." It would have to happen before any kind of duress.
The tech community's reaction to this stuff seems to boil down to "it does not solve every single problem therefore it is pointless." Which is silly, although to be fair it seems to be the tech community's reaction to a lot of things.
I keep seeing people mention locking the phone before going through customs. This accomplishes nothing. If they ask for your device, you are required to unlock it. If you refuse to unlock, best case scenario means you turn around and go home. Worst case scenario, you are detained as a criminal. If your plan is to fight back and not allow the agents access to your phone, you won't be making it across the border. Bringing the device with you means you must be willing to unlock without causing a fuss. The only way to "win" is to not bring the device.
If its something to do with security then yes, it is pointless. It doesn't matter how secure your door is if your window is open.
Said another way, the chain is only as strong as its weakest link.
How do they know you've disabled Face ID?
LOL. Now the police have the right to physically search random people? The police can do nothing if you put your hand in your pocket and disable the phone.
You are worried about a tiny percentage of cases. Most people are worried about border agents (who have authority as much as 100 miles inland) or cops in traffic stops asking to see their phones. 99% of the time you will be able to hand them the phone and disable it before they get it. They aren't allowed to ask you for the pin-code, so it's game-set-match at that point.
See: stop-and-frisk.
http://www.loopinsight.com/2017/09/14/craig-federighi-on-con...
http://blog.wilshipley.com/2011/03/celebrating-betrand-serle...
Interesting. I did not know that
That said, I find Face ID extraordinarily troubling, because it normalizes the idea that your phone actively scans your facial features during use. Just like carrying around an always-on pocket beacon became part of the 'new normal' with the introduction of the smartphone, a phone that looks back at you during use will become part of the new normal, too.
When you combine this with business models that rely not just on advertising, but on promises to investors around novelty in advertising, and machine learning that has proven extremely effective at provoking user engagement, what you end up with is a mobile sensor that can read second-by-second facial expressions and adjust what is being shown in real time with great sophistication. All that's required is for a company to close the loop between facial sensor and server.
Apple is unlikely to be this company. But Google, Facebook and Amazon are. What I anticipate is the next generation of Home and Echo to have cameras (Amazon is already moving in this direction), along with whatever piece of hardware Facebook produces. The idea of devices that look back at you will gain acceptance, just like always-listening voice assistants have gained acceptance. All of these will become input sources to learning algorithms.
What is already an incredibly potent toolchain for political manipulation will become even more powerful, with no oversight, accountability, or even much understanding by those who built it on the way it can be profitably used and misused. Its effects will be field-tested in democratic elections that affect the lives of billions.
This is what Zeynep Tufekci has called the architecture for networked fascism, and by manufacturing a mass-market device with active facial scanning, with the best of intentions, Apple has moved us a big step further along this dismal road.
So Facebook will add its own spin on animoji, requiring full face-tracking permissions - and then begin quietly mapping emotional response to every timeline entry.
The tech is out of bag and apple not doing it would just delay it by a few years. Managing a world where microphones and worse are everywhere and are being used by autocracy is how we should be acting going forward. Accepting that this is the world and dealing with the changes should be the focus, not trying to prevent tech that will be created whether or not some people like it.
edit: I do agree with your tweet about how it changes social norms although.
We've been able to bug houses for years, but only recently did Google and Amazon normalize it by selling an always-on network-connected microphone (and now camera) as a consumer product.
Apple built it with secure enclave, so they are actively protecting your privacy.
Now they've got to take their eyes off the road for a second, or fumble with holding the phone above the wheel.
https://www.macrumors.com/2017/06/30/ios-11-do-not-disturb-w...
Can law enforcement just hold the phone up to your face and unlock it? We all know Touch ID is not protected under the 5th Amendment so I am going to assume Face ID is not either.
EDIT: Apparently both Touch ID and Face ID disable after 5 failures — https://developer.apple.com/documentation/localauthenticatio...
Shouldn't one just disable both of them and use a long passlock code if they are serious about security?
This and TouchID are the most secure authorization features ever created, because 100% of people will use them,, and less than 1% will create long passcodes.
FaceID has to be at least as good as TouchID or it is a step back for the iPhone.
This has me very excited. ARKit already does an impressive job of tracking as you move around your environment, and the major thing that seems lacking is being able to map existing objects to know where they are in relation to virtual ones.
If they do this well, especially when it comes to automatically obfuscating virtual objects with real ones, it could allow ARKit to be used for creating much more immersive experiences.
Possibly, but it's also easy to pick up someone's fingerprint from anything they've touched after the fact. With faces, you have to actively record/3D scan someone. It's not clear whether e.g. security cam footage will be sufficient fidelity to reconstruct a 3D face in enough detail, but this will be interesting to find out.
In case anyone missed it, Apple claimed false positive rates for Touch ID were 1 in 50,000 and for Face ID were 1 in 1,000,000.
> Statistics about the false-positive rate are mostly meaningless in an adversarial case.
I imagine what those special effect masks were for, to test for the adversarial case. I also imagine that's why there's an IR camera too -- I'm not sure what wavelength of IR this is, but it seems distinct from the depth mapping, and would be more difficult to fool on a mask.
That would be NSA-grade security cameras, and based on the photorealistic masks they tested, even that shouldn't be good enough.
But it's not yet clear how much photorealism matters vs. 3D shape, which may be a lot lower fidelity.
edit: or even a fake iphone and I hand you mine and ask you to play a game that I'm building that steals your face data.
The basic idea (as I understand it) it to project a grid of even dots (or some other known pattern). Once they land on a 3D surface you can tell how close the surface is by how close together the dots are and how angled the surface is based on how the dots skew.
It requires the projector, a camera, and the distance between the two. But it's relatively simple (in theory, I imagine it's quite hard in practice).
If the identification requires just image, it could be printed on a black paper with IR ink (or just white?). Apparently they also look for life signals, so it might require video. In that case, it can't be produced with normal display, but maybe with some sort of old-school analog film and IR light.
This kind of attack (if it works) could be mitigated by adding random IR dots, which would be different each time.
It produces a 3D model of your head so paper definitely wouldn't work.
It would be a phenomenal oversight on their part if they failed to do this. As in, the only reason I wouldn't deem it criminal negligence would be because the legal bar for that WRT people's personal data is so low.
Par for the course if Samsung doesn't do it.
Right now, if you access your bank, Dropbox, 1Password etc. you open the app, wait for the Touch ID dialog to appear, then move your finger from the screen (where it was) and to the Touch ID to authorize.
With Face ID; just open the app, and you're authorized. This is a HUGE thing in regards of usability.
I love it.
https://en.wikipedia.org/wiki/Riley_v._California
Border agents operate under more permissive rules, although a recently filed lawsuit is seeking to change that as well.
https://www.eff.org/press/releases/border-agents-need-warran...
[0] https://www.macrumors.com/2017/04/12/apple-struggling-with-t...
Reading fingerprint with/through the touch screen [1] sounds a better alternative if they ever get it working well enough.
[1] https://www.google.fi/amp/s/www.theverge.com/platform/amp/20...
Well, on the demo onstage, he did not enter the passcode after being asked for it. He just said "Let me try that again". Is it just a cover up?
- If you haven’t used Face ID in 48 hours, or if you’ve just rebooted, it will ask for a passcode.
- If there are 5 failed attempts to Face ID, it will default back to passcode. (Federighi has confirmed that this is what happened in the demo onstage when he was asked for a passcode — it tried to read the people setting the phones up on the podium.)
- Developers do not have access to raw sensor data from the Face ID array. Instead, they’re given a depth map they can use for applications like the Snap face filters shown onstage. This can also be used in ARKit applications.
- You’ll also get a passcode request if you haven’t unlocked the phone using a passcode or at all in 6.5 days and if Face ID hasn’t unlocked it in 4 hours.
Lots of people in the threads here yesterday seemed to misunderstand how these things are implemented. Importantly, you cannot set up FaceID without first setting up a password (biometrics are a carrot for getting users to set up passwords, not intended to subsume passwords), and you'll still be prompted for a passcode around once a week so you won't go forgetting it.
This sounds annoying. With Touch ID, unlocking the phone is a deliberate action. With Face ID, it sounds less predictable.
Simply glancing at the phone sitting on your desk will not cause it to unlock, as far as I can tell.
There's speculation that the demo failure was because they had staff clean/handle the devices before the demo - i.e., they both looked at the device and touched the screen. Of course, lots of uncertainty around this particular botched demo.
This is what the article says, but it's also been reported to take just 2 failures with Face ID before requiring a passcode (as opposed to 5 with Touch ID). I can't remember now if that was said by Apple directly on Tuesday, but I've seen the "2" figure reported several places, so I'm not certain which is right.
EDIT: It's apparently been clarified to be 5 on both Touch ID and Face ID — https://developer.apple.com/documentation/localauthenticatio...
I think this is the case for how many times a passcode can be entered before the phone is wiped.
After 10 attempts, the phone is disabled until a restore, with the user-settable option of erasing the phone in this case.
This isn't Samsung dumping underbaked technologies on the world. Apple actually puts some thought into these things.
E.g., would Snapchat or Facebook now be able to pull a 3d model of the face of all their iPhone X users into their own servers?
> You’ll also get a passcode request if you haven’t unlocked the phone using a passcode or at all in 6.5 days and if Face ID hasn’t unlocked it in 4 hours.
You already need a password if you haven't used TouchID in 48 hours.
This rule seems to exist to make sure you have to enter your password after about a week if by then you go over 4 hours without using FaceID.
You don't need to use a password any time you go more than 4 hours. I agree that would be insane and would be much stricter than what they apply to TouchID (which they say is less secure).
EDIT: I notice the GP's comment seems to have been edited since the time you posted yours.
Sorry.
Assuming that Apple engineers aren't stupid, the correct interpretation is "we want people to enter their passwords at least once a week so they don't forget them, but we also don't want to bother them if they're in the middle of something". So yes, once a week, you will be forced to type your password upon waking up.
"...but that won't keep me from commenting." The answer you seek is in the article that you didn't read.
Similarly, every facial recognition technology I have used has either taken forever or was miserably unreliable. This time I'm not betting against Apple.
"It turns out an engineer hid a backdoor in the phone. You can unlock anyone's phone if you can just show it The Macguffin.
But now he's died and the world is about to be blown up and we need to figure out what was on his phone before it's too late.
We only have 36 hours left to find The Macguffin and unlock his phone before FaceID won't work; his 75 character password is totally unguessable."
Hollywood? Get in touch.
But yea, totally worth the negative 5 karma for some reason.
I could also unlock my phone and my tablet with my face (don't remember how Sony and Samsung call that method) but it means that I have to point their cameras to me. Maybe it's me but their cameras tend to point to the ceiling when I'm holding them. Fingerprints are faster. Even the swipe pattern on the tablet is faster.
FaceID is probably more about the current impossibility of adding the fingerprint sensor to the bezelless and thin iPhone X. I expect Apple to solve that at the next iteration. FaceID would stay and be useful for some purposes, but people will be back at unlocking iPhones with their fingers.
According to the leak from last week it should be able to unlock even if laying flat on a table.
You'll need:
1. 2 phones (at least 1 with an IR camera, such as another Iphone X)
2. a helper app
3. access to 10+ photos of the victim (Facebook typically)
4. a small mirror
With the helper app:
1. capture the suspect's phone's unique IR dot pattern by shining their phone at white piece of paper, recording it with the helper app (the helper phone needs an IR camera of course, such as another Iphone X)
2. makes 3d model of persons face from the FB pictures
3. generates 2 animated videos of their face, 1 just a normal color video and another with the dot pattern applied
4. now you need to show the 2 videos to FaceID, using the mirror to show the correct video the the corresponding camera
To the extent that it worked, we don't know. Since I am very familiar with the tech, I know that IR light does not have any special properties in detecting a mask vs human skin. But I assume the nueral network they trained looks for movement of some sort.
Since you're familiar with the tech, you'll be aware that humans, who are alive, emit IR light which can be picked up by an IR camera, and a mask, which is not, does not.
And even if it could, heating a mask to body temperature isn't all that difficult.
That said, they could use the color video feed to require subtle eye movements which would make this attack harder.
the ir dot video would spoof the 3d measurements. look into how primesense 3d cameras work.