Finance worker pays out $25M after video call call with deepfake CFO
edition.cnn.com
edition.cnn.com
Pretty much every single hotel gets a call from Mr. Patel at night asking to wire money due to an emergency. A lot of hotel employees fell for it and wire money. These employees even drill open the safe. Some even wire money from their personal account.
This scam is mostly social engineering without any AI/Deepfake. It's going to be a fun time ahead for everyone.
He explained the whole thing to me.
At any rate, a 5s Google of "patel hotel indian scam call site:www.reddit.com" reveals dozens of threads of people sharing essentially the same anecdote. Is that enough for you sir, or do you need something peer reviewed?
But the way he described the phenomenon made it seem pretty common, and indeed, a quick search for "patel cfo scam hotels" turns up a number of relevant results... Seems like it's a pretty well known, frequently occurring event in the hotel industry.
Not quite identical but close.
Other common scam is IT support.
My engineering manager fell for the CEO gift card scam.
Except the kicker is the CEO has asked him to actually do the same thing before for real hahaha
(Media reporting suggests this can also be true at some US hardware tech companies).
The daisy chaining prevents single responsibility stuff like this.
Also for what it's worth I've done verification callbacks to every single one of my bosses at some point during my career here and no-one's ever questioned it.
If there's a need then this will change. You might as well say that they'd never use a telephone because it's culturally alien. It was alien, but it was useful, so they adapted. Same with email and video calls. The boss has to log into their banking just like everyone else, because there's a need for it. If there's a need for this, the OP's suggestion seems like a pretty good one, as it augments the existing culture with a security step.
It's true, I don't know Japan, but I suspect that they might have it much easier to adapt than western pretend-buddy orgs.
In any case what I find strange is that usually HK finance companies (like much of the rest of the world) will have some kind of maker-checker system which prevents individual mistakes like this.
I can only imagine this being leveraged nefariously.
But having lived & worked in a few countries now, the way other cultures do their overrides is always more visible (e.g. Country A you might pay bribes to get out of tickets, country B might just not pull people over in nice cars)
Sure there might be cultural differences, but maybe this guy is just careless.
There was a case in the US where someone pretended to be a cop, called a fast food restaurant, and actually convinced the manager to strip search an employee.
I guess this is also a case of cultural power distance.
Power distance might matter, depending on nationality of participants.
Also if English is a second language, then perhaps the sound quality of the synthetic voices wouldn't need to be as good - we are surely better at recognising voices in our mother tongue.
"Silence, power and communication in the operating room" https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3001035/
Prior to checklists, nurses would feel hesitant to point out errors by surgeons.
Post checklists, people felt more empowered to say "Doctor, I believe you missed step 5".
(Didn't completely remove the hesitancy but this point was identified explicitly in Atul Gawande's book The Checklist Manifesto)
This could be totally real, but also could one employee saying 'the CFO was on a call' and claim deepfake to make it an excuse?
I guess it was a matter of time before this occurred. How long before scammers do bulk video calls to parents/grandparent pretending to be the kids saying they are in trouble and need $$$ ASAP.
The even better question, is how can this be stopped or reduced and is there a new business there?
Especially when a high percentage of people post their face and voice on social media. I find this especially crazy in the age of AI. I trained a Stable Diffusion LORA with photos of a friend and showed it to them (with permission) and they were completely shocked. Showed it to one of their friends and they were fooled for at least a minute and took some careful looks to find discrepancies
Keeping yourself anonymous isn't compatible with a lot of even moderately senior-level jobs out there.
More generally --the billions of hours and growing of audio video on YouTube, TikTok, and other platforms -- is literally someone in real life (most cases), likely some employee, that could be or become a middle manager somewhere.
1. Most companies don't do that.
2. AWS, for example, has what, 100k employees? What percentage of them are actually featured in those videos?
>More generally --the billions of hours and growing of audio video on YouTube, TikTok, and other platforms -- is literally someone in real life (most cases), likely some employee, that could be or become a middle manager somewhere.
A vanishingly small percentage of that content is generated as part of that middle management job. Yes, many people choose to place themselves on publicly accessible video, but it mostly isn't part of a mid level office job, so not doing so isn't incompatible with holding such a position.
5 minutes later, one of them came up with a pic: it was a group photo of the company staff, taken a few weeks earlier (with me skulking at the back; I never wanted to be in the photo). It was in an article on the company blog.
There has been little issue for most people having photos of themselves online on social media.
If people want a photo of you they will find one.
If you refuse and it's an actual emergency with the real CFO, it might be a career limiting move, if you don't get fired.
If you accept, it might be a deepfake CFO and you might get sued.
This is really the crux of it: senior management needs to take the lead setting up policies which are efficient enough not encourage people to try to bypass them and the culture that everyone in the company should feel comfortable telling the CEO “I’m not allowed to do that”. This is possible but it has to be actively cultivated.
Good luck with that.
I've had a CFO that didn't talk to tech people except through proxy have a "tell your mom to pass the potatoes" style meeting with his secretary as medium. Yes I stood there he talked to his secretary and repeated what each of us said 5 feet away from each other. This was a large bank.
I've had a general council yell with spittle at me because I suggested that it was probably a bad thing that the IT Dept was effectively acting as power of attorney for the company by doing digital signing for him and he should probably learn how to do it himself for legal reasons.
If they can throw you under a bus because you raise a valid issue, what are the chances they'll protect you when some fraud paperwork gets signed by the IT dept (so you).
Very few CEOs are going to make people feel comfortable telling them no.
My anecdotes were to illustrate its widespread if I've personally encountered it multiple times. Also just to entertain.
Let’s not ascribe too much power to those, either: NSLs can compel release of certain types of information but they can’t force you to do things like transfer money or even disclose the contents of private messages.
I would assume the matter of time for it occurring has elapsed a while ago, and now we are in the place where it's not only being detected, but further, actually revealed, regardless of how embarassing that is.
Unfortunately, this is why we need open access to some deepfake tech. The only way to convince people who are not immersed in tech how convincing deepfakes can be is to sit with them, and create their own deepfakes.
Then memorize and practice security protocols like verbal passwords.
https://abcnews.go.com/US/utah-missing-foreign-exchange-stud...
The generic type of vulnerability referenced in the latter part of the article has sprung up after fintech tried to emulate traditional offline auth and KYC with things like scanned images of ID documents, face recognition and liveness detection. Anyone in the know could see these attacks coming miles away.
You know, like we've been doing with our emails since PGP was developed in 1991. You can tell how simple the process is, by how ubiquitous it has become in a mere 30 years!
or as a Nostr note, for cool kids to share with other cool kids.
Defeatists get defeated!
Who is this "we"? I know personally exactly one person with a web-of-trust keypair.
"Can you buy $1000 worth of egift cards and text me back with the redemption codes? Our jobs depend on this. I'm in a very important meeting, otherwise of so it myself, left my private key at office and can't sign this message right now."
The human element remains the weakest link.
I work with people who all have hardware crypto, you are right that we do not have the organizational knowledge to verify everything with crypto. Even if the tech is 60% there.
We already have infrastructure for bus and rail tickets, for logging in to banks, tax authorities, health services, etc. in Norway and other countries that could easily be extended to cover this use case..
Cryptography without strong social rules is just cargo-cult religion.
Umm where have you been the last decade? The "Grandma help me I'm in a foreign prison and need you to buy iTunes gift cards" scam is extremely lucrative.
Opening with the line "Umm where have you been the last decade?" feels like throwing insults, and not conducive to a positive enviroment to learn from one another. You probably didnt mean it that way but though Id point out this style.
Regarding the last decade,, the pertinent part of the comment you responded to is "do bulk video calls to parents/grandparent pretending to be the kids" - more referring to when these existing scams hit a higher level.
A friend of mine in the US actually personally knows two people that this has already happened to, albeit with audio only. With video it's going to be nuts.
This sounds like it required quite a bit of preparation, i.e. collecting data for each deep-faked participant including image/voice samples.
If it's reaching this level of sophistication already then I suspect a new participant validation scheme is on its way for sensitive meetings.
It would easily be worth it spending $1m on the perfect setup.
That seems unlikely. I'm pretty sure there's actually a lot of economies of scale here, where the attackers' pipelines will become vastly more efficient and higher quality over time, with each attack requiring less manual work.
I have clients where anything over even quite a low set limit (say €10k) requires multi-party authorisation - and it's very common for the person entering payments to be unable to authorise payments. That's just good practice.
A payment should not be able to be queued without a PO number. If the payee is new, the bank details need to be verified by phone. Once approved as a destination account, that payee is set up in banking, and authorised by a finance clerk and someone more senior. At the point a payment is requested the PO and other details should be double checked against what is in the system. If there's a match, then the payment can be queued for authorisation. The person entering payments and the people approving payments should be entirely different - and it should be people, not a single person. When payments are entered, the payments should be reviewed by first authorisation - a finance manager, for example - and once that authorisation is conducted, depending on payment limits, another authorisation or authorisations will be carried out.
If you have 10 business units trading 50 world currencies, checking 500 transactions for FX every day is a total chore hence it would get automated, and only unusually large transactions would be flagged. Rules like <10m goes through automatically would be tuned over time so that the workload on operations team members would add actual value without being onerous on their time.
So, depending on the business we are talking about, a 25m transaction could basically be lost in the noise. Given the mention of the CFO being london based and the operations team being in HK, it sounds like a typical investment bank setup to me.
After having worked IT for various startups I cannot understate just how much executives and other higher ups detest policies that make them verify who they are. It short circuits something with their ego.
This is astounding levels of incompetence.
If processes can't be bypassed, then as soon as you implement a detrimental process, your company dies.
From what I understand of the literature, it’s often several interactions to gather enough information from several employees to learn to sound like you belong there, then using it all against someone with “keys” who escorts you the rest of the way.
I can imagine a scan where the fake CEO gets a phone or laptop outside of the process "because CEO". This however will still be limited to generic, low value stuff handled by single people in a company.
There is no way that a reasonably organized company can leak 40 MM USD.
Think I found your problem, boss.
Oh, please, HP lost some 40 million in inventory while contracted to Solectron Global for repairs, because their inventory systems are utter garbage compared to Dell or Toshiba.
Except these sort of transfers almost always happen with, at a minimum, dual approval where exceptions cannot be made because it's software defining the rule.
1 employee submits the transaction for review, and a 2nd (and sometimes a 3rd, 4th) person must approve it before the payment initiates. There isn't typically a bypass function.
Also, CFOs are typically responsible for setting up and enforcing these controls. A big part of a CFO's job is to manage risk. If you work under a CFO, you would be more likely to be rewarded for following the process than be punished.
Obviously there are exceptions to this, but by and large no CFO would punish a finance person for disobeying an order to bypass a process intended to prevent financial fraud.
CFO are often involved in fraud. it is part of finance industry training to be wary of dealing with other financial institutions.
https://tax.thomsonreuters.com/news/enron-former-cfo-i-am-on...
https://www.accountancydaily.co/ex-countrywide-cfo-charged-f...
https://www.businessinsider.com/bed-bath-and-beyond-cfo-foun...
https://www.nytimes.com/2024/02/01/nyregion/weisselberg-perj...
https://www.justice.gov/usao-ndtx/pr/cfo-controller-corporat...
https://www.fraud-magazine.com/article.aspx?id=4294976271
https://core.ac.uk/reader/231825040
https://www.nydailynews.com/2023/10/05/former-ftx-co-founder...
https://www.justice.gov/usao-wdwa/pr/former-company-chief-fi...
https://www.nydailynews.com/2023/10/05/former-ftx-co-founder...
https://www.investopedia.com/terms/w/worldcom.asp
whistleblowers get punished all the time.
https://www.pbs.org/wgbh/pages/frontline/warning/interviews/...
https://www.institutionalinvestor.com/article/2btg8yx4pcckb0...
https://www.marketswiki.com/wiki/Madelyn_Antoncic
https://www.wsj.com/articles/wells-fargo-fined-22-million-fo...
https://www.npr.org/2016/11/04/500728907/senators-investigat...
etc etc
Obviously the statement isn’t literally accurate. Hopefully it’s 99% accurate (otherwise none of us would have jobs if all we did all day was sabotage our employers). Likewise, not every CFO is to be trusted, nor are all software engineers… but most can be.
These people aren't stupid. I'd expect them to understand risk better than your average senior software engineer and if you tell them "Sorry boss, too risky to do that right now. I can't be 100% sure this message is genuine. Let's sync on this after your meeting", your chances of promotion at this company would likely rise, not fall.
install notepad++ from pre-packaged store? approval needed
change to mailing list you own? approval needed
1 line config change to production alerting system? 8 approvals needed
I can easily imagine people just clicking Approve sometimes without reading
And many more stories like that.
But yes, for small fish there is an approval process for everything, even to but a paper clip.
a common thread in these modern rogue trader scandals are that the perp worked on the controls or monitoring system in a role prior to becoming a trader
so they knew how to structure their trades in such a way to evade detection
Regardless, approvals for multi-million transfers require a higher level of process and approval.
> Regardless, approvals for multi-million transfers require a higher level of process and approval.
if you say so
Oh, my sweet summer child. The larger the organization, the more dysfunctional it becomes.
See How this scammer used phishing emails to steal over $100 million from Google and Facebook
https://www.cnbc.com/2019/03/27/phishing-email-scam-stole-10...
Couple years ago I thought that too...
All the checks you describe - multiple approvers, standing data, callbacks etc - the guys going after big payments like this know these checks are in place, how they work and have a game plan for it.
If you can deepfake one guy with the checkbook, can’t you deepfake the guy with the checkbook and the guy who enters the POs into the system? Lower odds, but far from zero.
I mean we still right now live in the world where just a very rough match for signature on a piece of crappy paper is enough to move millions if needed.
Maybe it’s because I’m in the EU. Banking here is very different to the US.
Deepfake was used in the 2023 MGM casino breach to convince tech support staff to do things that compromised their MFA
Now we're seeing a combination of these for significantly higher gains.
this is the real problem. why oh why, after suspecting an email as phishing, would you then go on to even click ANYTHING, let alone join a video call?
insanity. either stupidity or he's lying about suspecting the email. how many corporate security trainings does it take? this is just about 101. "if asked to do a secret task by a suspicious email, DONT do it"
It takes $CURRENT_NUMBER + 1.
People are still, to this day, racking up thousands of dollars in iTunes gift cards on corporate cards and mailing them out, because they got a text from "the CEO". It happened at my spouse's work just last year. It'll continue happening again, forever, because to paraphrase P.T. Barnum, a sucker is hired every minute - in the probability distribution of humanity along that particular axis, there's always going to be some percentage at the bottom who'll fall for the most obvious scams. Sometimes repeatedly.
This is not what they teach you in trainings, though. They teach you to get the requestor (or your boss or whoever might be authoritative) on the line and confirm that the email is authentic. I believe a video call qualifies as well.
Have you ever actually done corporate security training? It's very obviously 100% useless and not going to teach anyone anything.
A company I worked for actually started sending test phishing campaigns which is a lot more effective, but I thought they were still pretty obvious and also it led to stupid people reporting them on Slack endlessly.
Still, probably the best thing you can do.
I've seen some decent ones. e.g. One that was presented from adversaries PoV which I thought was innovative & got people thinking about it in novel ways (at least did for me).
Many people will open a suspected phishing link, report it, then open it later in the afternoon...
If someone claims to be a police officer and hands you a number to call to see if they are real... don't use that number. Figure out the non-emergency number of the station they claim to be coming from independently and ask them. If a "new agent" from your bank calls you and gives you a "new number" to call them, figure out an official number of your bank and call that.
Also, whenever paying new accounts, once you've independently reached the person you think you're talking to, always do a test transaction and make sure they get it before sending the rest.
Nobody would accuse me of great people skills and while I'd like to point to my technical acumen as the reason I can spot fakes like this easily, it's my primate brain that knows something is wrong.
Social engineering works because people think they could spot it.
That belief is a catch-22, though. By definition, each time one fooled you, you didn't note anything other than a run-of-the-mill normal video. A lot of tiktok accounts lately are dedicated to deepfaking celebrities. For example, if I hadn't already told you and you just casually scrolled by it, would you immediately suspect this isn't Jenna Ortega https://www.tiktok.com/@fake_ortegafan/video/732425793067973... ? I didn't look for the best example, that was just the very first that came up.
>Is this an already existing product
Usually cutting edge ML has to be done with a github repo last updated a few days ago using Tensorflow/Pytorch and installing a bazillion dependancies. And then months later you might see it packaged up as a polished product startup website. I've seen this repo a lot https://github.com/chervonij/DFL-Colab
I tried to find the link but my search-fu is not good today it seems..
I did find this, which seems related: https://blog.metaphysic.ai/the-emergence-of-full-body-gaussi...
There's also the fact from the article that this was an employee in Hong Kong on a video call with people supposedly in the UK, so it's also possible they took advantage of bad video quality to do this..
Get on video for the first minute or so, then, as we've all done, say "I'm going to turn off my video so my connection sounds better" etc...
The researchers really just raise awareness on where things are going, but ultimately the solution will be to improve process and verify anything that has to do with money through specific internal company channels that are hard to forge - and anybody in a call like this that would not use them needs to automatically raise an alarm by procedure.
Just the idea that the perps in this case had the ability to code this all up by themselves is ridiculous, 99.99% of the cyber crime out there is point-and-click from some downloaded tool and maybe 0.01% 'hackers' that use their own tools. Releasing all this junk in easy to use form is a very large factor in the rise of cybercrime. Imagine an outlet on every streetcorner where advanced weapons were given away freely and then to make the claim that since someone could theoretically come up with any of these there is no reason why we shouldn't be giving them out for free. That's roughly the level where we are at.
There is some middle ground between researching how things could be done and releasing those tools to every wannabe criminal on the planet, many of who are in places that you'll never be able to reach from a legal point of view. 1000's of businesses are hacked every day by tools released by 'researchers' to prove that they are oh-so-smart without a shred of consideration for the consequences.
I mean sure, you can nicely ask or try to shame people, but when did that ever do anything of note?
Talking about how something like this can happen in a big company is fun and all, but the scary thing is is that it is _so much easier_ to do these sorts of scams with deepfakes. Which means they will be deployed against "softer" targets, like you and me, and your parents and grandparents.
Imagine every C-level exec who's opened a top-urgent ticket with IT because their printer doesn't work (they forgot to plug it in/forgot it needs paper/it's not a printer, it's a paper shredder) trying to operate some form of key exchange software securely, while people capable of pulling off this sort of scam are targeting them.
I don't think this is a problem that can be solved with technology.
We already have facial verification systems in hundreds of millions of devices that are genuinely very difficult to spoof.
Oh, you mean magic? :P
These $25M... waves hands gone !
Doesn't the US military have DoD people plug in their ID badges to read/sign emails through outlook?
Such headlines are usually followed, a few weeks later by an headline reading not unlike this:
"Three indicted in scheme involving deep-fake to steal $25m".
Basically, it was a well thought and well executed scam that perfectly fit the employee's situation.
I fthe CFO gave a number on the call, it wouldn't also be much of a check.
I think the real improvement would be to have the CFO file a ticket, but obviously that company was used to play it loose and fast.
I'm guessing that someone who can authorize a $25M transaction is fairly high up in the corporate hierarchy, not that many levels away from the CFO.
I have no idea, but I suppose moving funds from one subsidiary to another for instance wouldn't be for a few thousands only, and he's seeing money fly around day in day out. Would it feel the same as an infra engineer rebalancing a few millions of access from a cluster to another ?
That is, the scammer manages to get ahold of the SIM card / phone number of the CFO, and be on the receiving end if/when a worker calls the CFO up.
Weakest link would probably be to compromise some telecom worker, so that this can be orchestrated.
This problem isn't a technical one..it's a process issue. One person shouldn't be able to transfer $25m without multiple people authenticating and authorising.
It wasn't just a fake call, and he had a paper trail of the order...at this point it's pretty hard to prevent this from happening, short of having every order double checked by some other independent entity.
If an employee routinely receives email or zoom instructions to transfer $25m without any sort of sign off then the company is completely at fault for terrible process.
Most non-enterprise companies have fairly loose wire protocols. That said, outgoing phone calls to two separate signers is a good, simple best practice.
Sam deal for the call as well. I'd expect the video client to warn that some members of the call are external to the organization (Google Meet does that). Or the CFO is expected to be outside (from another org) from the get go.
> Initially, the worker suspected it was a phishing email, as it talked of the need for a secret transaction to be carried out.
That's how I almost lost £100k. I got an email from my lawyer instructing me to pay an amount that I was expecting to have to pay, but to the wrong account. The email "from:" was definitely my lawyser's email address. It satisfied Gmail's spoofing checks. But it was not my lawyer who sent it.
The funny thing is, I ask them to say say "I don't know" rather than the above, but they still do it...
You can work around it by picking a difficult practical problem from your domain and talking through choices and their different tradeoffs.
This is an obvious and natural evolution of the kinds of attacks that have existed for years. It was bound to happen eventually. I think it's just sooner than people expected.
1) a multiperson zoom of deep fakes fooled the worker 2) the worker was in on it as an inside man and the deep fake story is cover
Don’t write a check unless you hear me mention aardvark or Mad King Ludwig.
That is the actual title of the linked article.
Finance worker pays out $25M after vid call with deepfake CFO
Edit: maybe not zoom
Some of those words are even in the dictionary…