Supermarket freezers across Europe hit by second suspected cyber-attack
theguardian.com
theguardian.com
One thing I see repeatedly brought up here in this discussion is "Why can't we just set the refrigerator temperature with a knob and walk away?"
The problem is that these refrigerator cases are not as simple as your refrigerator at home. As a simple example, many systems allow you have control over the deadband: the exact low temperature at which the compressor turns off and the high temperature at which the compressor turns back on. You also have control over defrost cycles, compressor pressure and many other things (I worked with this a long while ago, and I don't remember all the details).
Take a look at some of these manuals, for example: https://www.hillphoenix.com/resource/installation-manuals-2/... As far as I can tell, those manuals just cover the refrigerator cases, and not the outside compressors and heat exchangers, which are another layer to deal with.
This is not something that a non-expert supermarket employee would be able to handle well. The purpose of all these settings is optimize around energy usage, maintenance costs, customer experience, etc. Even if you only save $500/store/year, you can see that it really adds up over a chain of 1000 stores.
That's why this equipment is remotely accessible: it's not like your fridge at home and requires a lot more knowledge to get the most out of it. Of course, that does not excuse poor security implementations.
One particular example: store managers want a very cold, very narrow temperature deadband on their frozen items. This gives the customers a nice experience. They're less likely to complain about melty ice cream. But this type of setup leads to more energy expense, and failures on the compressors due to excessive cycling.
I still don't see why the expert has to be remote, or why an integrated computer can't do the optimization. Or why it needs to be fiddled with at any significant frequency.
These fridges may need more complex control for a variety of reasons, but that doesn't mean they have to be connected to _the internet_. To a local network for control, sure.
Put your security measures into the network itself, but I really do feel fridges are another thing which do not need to be IoT devices.
> I still don't see why the expert has to be remote
Yup, they're done with it, all right. "It" being all the food being cooled for sale this week. Done.
They do not need engineers at every location to turn every knob. They need to define procedures and limits, and the ability to call the engineers when things go out of those limits. Even putting only remote MONITORING online, but requiring the local person to check and turn the knobs/press the buttons. That also adds an extra set of real eyes on the situation — when the engineer calls the local dept manager to start the defrost because the hacked line says it is now time to defrost, the local manager —with eyes on the freezer— can tell him "NO!" because this was not on their schedule and the freezers are all stocked, not emptied for the defrost.
Instead, the defrost cycle just turned on in the middle of the night, and neither the now-remote engineers nor the local managers knew until it was too late. ——Again.
Security is always a trade-off against convenience. And the idiot human managers almost always all pick convenience because their managers incentivize profit, and security costs and convenience appears to save up front. They nevermind the later costs because they'll already be promoted up or job-hopped out.
Of course this is not enough time to entirely change one's strategy, but it is a lot of time to install hardening measures, such as stronger passwords, limiting access to only MAC addresses of specific machines, taking access offline wherever possible, disabling the defrost cycle, etc., etc., etc., and that is just the obvious.
One store loosing its frozen food is an inconvenience and insurance claim, and one electric substation failing is an inconvenience for a few days.
But whenever any one of these attacks could be scaled up, as is obviously possible with online access enabled, and the consequences become a regionally or national security threat, enabling such access must come with the seriousness of implementing national security.
Unfortunately, it is not sexy for politicians to take national security seriously, and they will get inundated with lobbying decrying the costs. Until the losses to the companies exceed the costs of security, when they come screaming to the govt for help, and demanding why they didn't do something.
It is the same with almost every systemic problem that seems small now but is cumulative and/or exponential.
And for sure, if it is a few stores, it is just an expensive prank. But scale it to the size of a nation or region in a country with a largely just-in-time food supply, and you're a week away from food riots, particularly if it is combined with other attacks, e.g., on the electrical and distribution system. Not a shot fired, and the country is in chaos in a week. Because no one bothered to harden the systems that seem trivial but are critical at scale.
It took no time at all for this to become deadly.
And social engineering 1000 managers is FAR more work than scaling a software hack across 1000 machines, and also far less likely to succeed. At least one of those managers is going to get suspicious and call corporate first. Moreover, it will be pretty hard to get an on-site manager looking right at the full freezers to set them to defrost....
- "machine doesn't work!" It was set in the wrong mode. Who did it? No answer
- "machine doesn't work!" The HMI shows a huge message with "CHECK SENSOR AA-123" or "VALVE BB-321 CLOSED" and they are puzzled when you point it to them
See also the "PLC program changed itself!" in-joke in industrial automation circles.
The reality is that what you envisage "does not scale", to borrow from software engineering.
So, how can this be solved? Ideally for the long-term, but also for the current interim state — we have a known attack vector costing millions, known exposed, and need to do something.
Short term, seems like disable defrost or other critical vectors? IDK the architecture, so IDK how difficult it would be to add a local interlock or override for critical instructions that would damage the whole contents, so the local manager must also confirm "READY TO DEFROST"?
Long term, seems the only solution is to rearchitect a truly secure solution with secure access controls, locked-down to a limited set of IP and MAC addresses, all traffic encrypted, etc. Plus a deep evaluation of the sets of instructions that could be catastrophic, and adding requirement for local confirmation/override to execute those functions.
It also seems there is a lot of available improvement in the UI so regular Joe-managers could use it. Even if they see "CHECK SENSOR AA-123", their eyes glaze over because they just see it as the gateway to a hellish maze they don't want to enter. Adding layers of actually informative error messages and walk-them-thru steps to fix it seems well within our technology. It'd probably require an additional local mini-PC on each site, but could save millions.
If this gets scaled to a whole country, when most of our food supply is Just In Time, it is no longer a prank.
That's too generic. How much exactly do these "connected fridges" contribute to that?
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=supe...
How much does these savings contribute to their overall costs?
How much difference in that over dumb fridges?
How much difference in that over what can be replicated by a localized control system, at most sending reports through an "output only" line?
And what exactly would remote repair fix if some fridge hardware fails (that's not just switching to a failover hardware, that could just as well be decided by a local controller)? Since the answer is close to nothing, how much of the "remote repair" need isn't cyclical, addressing issues coming from complicating their fridges to the degree that remote repair is required?
Right. Operating efficiency is just the pretext. Congratulations on uncovering the real story about networked supermarket fridges.
In all honesty, I can't prove that these systems help supermarkets reduce their operating costs. It just seems the overwhelmingly obvious and likely explanation for their adoption. If you want to argue otherwise, then I'd say you owe us more than just a list of skeptical questions.
Because then he can get paid more.
Weather changes, equipment performs differently as it wears, changes in customer behavior, different items being stocked, electricity cost updates...
Well, it doesn't. The retailer has a team of techs (or an outsourced vendor) rotating between stores, who are cleaning and replacing air filers, checking lubricants, and so on. They could certainly apply occasional changes to settings.
But to the product manufacturer - the remote management feature literally requires less than $2 of hardware. Hell, it might even save money, by removing screens and buttons.
And the person buying the refrigerator wants something that logs and monitors temperatures, for food safety. And they don't know anything about cybersecurity.
So when the manufacturer says that for just a little more they have a remote monitoring option, which not only provides the monitoring needed for food safety but also reduces maintenance costs - it sounds good to them. They don't realise they're signing up for a multi-million-dollar loss when they get hacked and have to throw out a load of products.
And this is the key fact. It’s no longer acceptable to sell food and not know anything about cybersecurity the same way it hasn’t been legal to drive a truck without a license in decades (because a lot of people died). There was ample warning since the incident at a couple US military bases earlier this year. Market owners need to face it.
The manufacturer/operator could, at least, not come with the omnipresent admin/admin credentials combo.
People always say stuff like this, but it never tracks for me. If you’re a chain with 1,000 stores, $500,000 is barely a blip on your budget sheet in exactly the same way that $500 is barely a blip on the budget of a single store.
Plus there’s no way this remote management stuff doesn’t cost at least $500/store in operating costs.
Going by the number of companies that still can't grasp having to pay for ongoing maintenance for and IoT system, even with CRA and NIS2, some still selling stuff running on 12+ year old software, I wouldn't expect them to pay much for this service or to even properly maintain it.
At $500/store/year times 1,000 stores, you can pay an entire team of experts for a year to come up with a way to save that money, and come out ahead. That team's manager can then go to their boss and say, look, I saved the company half a million dollars a year, and they'll get a nice bonus from it.
Famous last words.
Also, does all this remote management whatever stuff cost more or less than slightly better insulation, or an upgraded compressor?
Setting up and changing parameters would involve, at the very least, a technician walking a store employee through the process. Best case scenario the technician would dial in using a modem.
Complicated things don’t have to be connected to the internet. Most environment control systems in small buildings (typically with hundreds of sensors and actuators) aren’t. There is smart, deterministic, software involved but most of the time, no cloud.
You can have remote access through secure means. You might even have a web page open to the public if you secure some kind of safe login mechanism.
If your story is true, then these stores and chains better take the financial impact they hit out of their CEO's paycut, because they absolutely chose to risk it all for some minor cost savings. Maybe try to sue the manufacturer (though I doubt the judge will side with you if you're hit with the same attack just a week or so after the hack appeared on the news, without having fixed the problem).
And the real answer comes out eventually: it's necessary now because we consolidated and cut costs and jobs. The people left who we pay next to nothing could never even handle them at the level we had then, because we laid off everyone who knew that old stuff.
> Even if you only save $500/store/year, you can see that it really adds up over a chain of 1000 stores.
And even if an event like this only happens once in 20 years, all of those savings evaporate. Thank god nobody will get fired for this. Or at least the first wave - it will be insane if nobody at the chain hit by this current wave gets fired.
They managed to do it when none of their competitors was doing it either. It's a low margin business.
> And even if an event like this only happens once in 20 years, all of those savings evaporate.
I don't think the numbers on that actually track, if you look at the estimated damages that are available. At once every 20 years, you'd still come out ahead.
Maybe they $500/store/year is not worth the risk? Maybe having read-only remote access will still save $300/store/year and $200/store/year is not worth the risk? Maybe having a physical button that someone at the store needs to push to temporarily authorize write access is worth the hassle? etc.
The only remaining thing is the defrost cycle stuff, and even that can be 90% of the way to optimal just by having a clock that knows when peak power is. The classic thermostat we replaced the mercury switch had that. I imagine they're like $10 retail, $1 for the microcontroller.
Assuming the extra complexity does actually save another $500/store/year, how much of that $500 is eaten by maintenance contracts, software patches, wifi debugging, etc, etc? Also, what happens if the internet goes down?
I'm not convinced it makes any sense for these things to be online.
It would be a more sensible tradeoff bo matter the scale of the store, thab jeopardizing the entire fresh supply of a nation.
And yet they existed, and didn't need a connection to the net in the 80s and 90s. Most supermarkets not even in the 00s.
People find it more convenient to network everything, but it isn't Necessary we're seeing the very predictable downside.
At that point it should be firing alarms on every grocery chain everywhere. While you may not be in a US military base (they are impopular with a lot of people) the fact is that someone figured out how to do it and others will figure it our too.
Much of the coverage in Swedish media drives the point that only 4G-uplinked stores have been hit so far. I think that's unfortunate, because really it wouldn't have differed if these stores were connected by 5G.
I'm guessing here, but my money would be on freezer ICS systems being made available to the world by accident by way of NAT-PMP on a consumer router. In Scandinavia you can bypass CG-NAT on mobile connections by setting a particular APN, so my guess is that these control systems were simply findable on Shodan.
This also explains why this wasn't discovered after the recent French breach. Supposedly all the stores with "proper" uplinks were in compliance, and no one had thought of a way in for those non-compliant stores, because they "weren't even connected, so how would an attacker get in?"
Why would there be accidental port mappings on their router though? I could maybe see owners setting it up so they could monitor from home, but even that is a bit of a stretch for rural supermarket owners
That's the point of NAT-PMP / UPnP, it just works (for game servers, torrent clients, industrial control systems etc.)
These refrigeration providers work with local contractors and HVAC companies to set things up. Their installation techs do what they think is required to remotely operate these systems. If you bring up the question keeping these systems offline or at least on airgapped networks you will most likely get a shrug as reply.
This is not a technology problem. This is problem of responsibility. One can hope things will improve when litigation starts happening, but it doesn't look good considering these people had literally months of notice since the well-publicized attacks on the American fridges, then the French fridges, and now these.
Litigation does of course occur but it's not the main mechanism.
Do you folks coordinate in the way described, and show success with it? (Would most people agree with you?)
Sweden doesn't have a state created e-id system, instead a consortium of banks jointly funded development of a solution in the early 2000 and this is basically the default way you prove your identity online in Sweden.
Fucking Saltsjöbadsavtalet.
1. many countries have higher regulatory burden than the US
2. that does not mean regulatory gaps don't exist
The person above was simply pointing out that litigation is not the only way to solve a problem. Which is plainly true, you can simply require compliance via other legal means. For example, a government could create a regulatory standard for security and require vendors to follow it. Whereas in the US the political appetite is to be more hands-off and allow people make vulnerable products if the market allows.
someone manually poking a hole in their firewall is way more likely
It has to do with refrigerators reachable on the Internet.
If the refrigerators were not reachable on the Internet, none of this fragility would exist.
The fragility of refrigerators reachable on the Internet was demonstrated just a few weeks ago. These refrigerators were left connected to the Internet.
This lack of discernment and misunderstanding of risks and fragility will not be solved with a better router or a different uplink.
And it would be (mostly) fine if they were on the net, with read only access
And most importantly, these fridges always have someone close by. Make it a physical button instead of a remote function
I mean, what happens when a compressor or other physical part goes out? You can't fix that remotely anyways. You might still want remote monitoring for that kind of situation.
Details, kind sir
> It will recalibrate your refrigerator's coolness setting so all your ice cream goes melty.
https://open.spotify.com/track/1mvK5h91yWsm97sAugKy6K?si=wXw...
With the following interesting quote from the supplier of the system:
After that attack, Danfoss advised its Dutch clients on October 13 to disconnect their systems from the internet and ensure they only reconnect via a secure network. “Most stores took action at that time, but a small number did not,” the spokesperson said. “They are now the ones suffering the consequences.” The spokesperson added that the systems hit last night were not on a secure network and were still using default passwords. The attack could have been prevented if measures had been taken, he said.
https://news.ycombinator.com/item?id=50055304 (~1000 comments)
Hackers activate automatic defrosting in supermarket fridges across Europe - https://news.ycombinator.com/item?id=50055304 - Oct 2026 (1007 comments)
I think the military commissary's freezers were hacked - https://news.ycombinator.com/item?id=49508506 - Aug 2026 (239 comments)
The comments in the Aug 26 thread aged quite badly.
Was it a password break or an open admin console or SNMP or what?
Glass half full: The scale of this is rather small overall, so most supermarkets etc. appear to have at least some form of basic security.
> the hackers pirated controllers mostly supplied by the Danish company Danfoss, which said subsequently that it had identified some security flaws last year and had “long recommended” customers install updates.
The real issue is geniuses in food stores connecting to the open internet systems with 100K+ EUR of products exposed.
Remember, the 'S' in 'IoT' stands for Security.
For your typical home user that bought a VPN because a YouTuber promised them extra security and privacy, no. They're gaining next to no security unless they have a malicious ISP that injects content into HTTP connections.
Was a bit surprised (well, no, not really) to see that the linked XWEB bulletin in that alert mentions a backdoor which can be used to help customers regain access to breached systems. Wondering how likely it is that this backdoor algo has also been leaked or reverse-engineered too:
(translated from French to English with Google Translate)
> If your password has been changed and you are unable to log in, please contact Copeland Technical Support, available 24/7, at [...] and ask for instructions on how to obtain a temporary password. Copeland technical support will need the current date of the XWEB EVO and the MAC address displayed on the XWEB login page.
"In the French attack, the hackers pirated controllers mostly supplied by the Danish company Danfoss, which said subsequently that it had identified some security flaws last year and had “long recommended” customers install updates."
I wish commenters (in general) would do a bit more "steelmanning" before asking questions about why something is (in this case) connected to the internet.
People won't steelman because the answer to "why are the fridges slapdashedly networked?" is too often "because people of my ideological bent and demographics if not literally me previously demanded paper trails, audit-ability and regulatory compliance that was impossible for a low margin industry to pull off any other way."
Requirements in the food industry come not just from government rules and suggested best practices (that are more than suggestions in practice) but also indirectly as a result of requirements imposed by insurers, financiers and trade groups. In response to this companies that run supermarket chains or manufacture major brands of food have not quite the same verbosity but certainly comparable degrees of retention in their record keeping as medical and aviation.
This monitoring and control is increasingly centralized in response to all these requirements because it's most efficient that way. If you're gonna have one guy in an office deciding things you might as well automate away the middle man and have him set them too and let the people on the ground spend more time doing their core work.
If fifteen of the twenty thousand dozen systems (pulling that number out of my ass FYI) in an industry are being centrally controlled through computer systems it's inevitable that some of them wind up off the VPN and this happens. It's just a natural tradeoff or risk you run when doing that and it's completely foreseeable.
But the internet peanut gallery will not generally have any sort of adult discussion about these tradeoffs because "we saved X (maybe X is money, maybe energy, who knows, who cares) by operating the fridges more centrally" is a nice easy win for them and rather than have a discussion about subjective and hard to pin down long tail risks it's easier to just screech about "we should fine the people who did it wrong" or whatever when one of those long tail risks rears its ugly heads.
I’m just a software person, not a supermarket expert, so maybe I’m being a bit simpleminded here, but for the entirety of my career I’ve participated in a constant trend toward centralizing and automating operations. That’s been the case regardless of how highly regulated the industry was. The motivating force has always been making things easier, cheaper and less error prone.
I suppose we could throw a rant about regulation into the mix, but, in the words of Laplace, I have no need of that hypothesis.
I was talking yesterday to my AI imaginary friend about creating an MCP for one of my apps, and the first thing she told me was to make everything read only. No matter how many security layers you have in place, passwords, auth libs, whatever, it will be leaked/hacked so you better be prepared for minimum damage
Monitor your freezers temperature? yes, but never allow their thermostat to be set over the internet. I hope the lesson is learned. Applies to all appliances
I'm not that familiar with applications/current state of this type of hardware, but it seems like it will become more relevant with all the new vulns dropping with higher frequency.
https://www.svt.se/nyheter/lokalt/skane/icas-kylar-och-frysa...
At the human being level, this attack is mind boggling. What do you get out of this, other than making good food go to waste?
Edit: Also keep in mind that there's franchising in supermarkets as well. So the guy might literally be looking at losing a chunk of money as a result of something out of his control. Doesn't change the situation, but not having done something doesn't make them deserve this...
Snark aside, supermarkets are not in the tech business. The extent of their involvement is ringing their IT vendor to complain about it.
I don't know what "cold chain people" are, at the scale of supermarkets you permanently have refridgeration for different groups of consumables.
I'm not aware of a scenario where e.g. fish is being shipped, and the originating logistics remotely sets the temperature of a fridge at the destination.
And for anything infosec, unfortunately people who know a lot about infosec have a habit of 'crying wolf' and talking about infosec as some kind of absolute rule that everything has to bow down to with no willingness to balance against other business risks, so probably leadership is used to ignoring the people who knew that this was a big deal and were calling for action.
Commercial cold chains are expensive, high value collections of equipment that even small errors at the edges can cause huge monetary losses. And grocery stores are in an extremely low margin business.
None of this requires the customer to be technical, the ORM could ship the equipment with the network setup, and installation would just be connecting the DMZ device to the Internet.
Of course there are ways to setup these things securely, same with smart tvs and doorbell cameras and whatever else is getting hacked. People just have a bad time figuring out the cost benefit of all of this. And frankly we don’t know if that cost benefit has even been crossed yet, perhaps this doesn’t rate.
You yourself are engaging in cost prioritizing without adequate information. Sending a technician on site may be extremely expensive (opportunity costs and inventory risk on cold chains are wild).
It’s not obvious to me that the secure approach is measurably more work than the insecure approach. Someone had to go out of their way to add support for remote temperature control.
> Of course there are ways to setup these things securely, same with smart tvs and doorbell cameras and whatever else is getting hacked.
No one expects a $20 IoT light switch to ship with its own network equipment, but a $600,000 to $1.5M fleet of freezers for a single grocery store can ship with its own network equipment without increasing the cost a single percentage point.
> You yourself are engaging in cost prioritizing without adequate information
I had hoped that was obvious from the question mark in my original post :) I’m asking what information reconciles this “it feels like they went out of their way to make this insecure” security posture. Responding with “they must have information you don’t have or else they wouldn’t have done it that way” is neither helpful nor logical.
I'm not happy about the attacks, but it sure seems like someone needed a wakeup call.
If you charge money for something that is critical national infrastructure - and the food delivery pipeline is absolutely CNI - and you connect it to the Internet, you should also be responsible for keeping it secure - or at the very minimum at least trying to do so.
The same goes for other CNI such as telecoms (including home routers and phones), broadcasting (including "smart" TVs at the eyeball end), banking and payment processing infrastructure, cloud computing, electricity, gas and water supply, traffic signalling... all modern economies are now almost totally dependent on so many different interlocking internet-connected systems. I've got a stash of food and water at home, a battery powered radio, candles, blankets to keep warm and so on, but that's only really enough to deal with a short-term emergency, not the collapse of technological civilization.
Failure to understand this is not an excuse. Sadly, the only way to deal with this is legislation.
I now read some interviews with local chains in Sweden and Finland, and they were still just investigating what to do at this point.
Those freezers seem to be having lots of fun :-)
If I put a Zigbee thermometer in the freezer they can hack it all they want, but I am pretty sure they will not be able to set the temperature to 50C.
Hackers can still tamper with the telemetry and create false reports of problems, but actually changing the thermostat settings would require local access, so there's opportunity for somebody to verify the problem really exists.
- Remote alerting for the store manager/head office to quickly respond to a refrigerator failure after hours, providing options for saving valuable stock (e.g. move stock to another refrigerator).
- Maintaining an audit trail for cold chain management (e.g. regulator says you must keep an audit trail proving that a frozen fish has been kept below -30oC since it was processed on a ship all the way up to point of sale in a supermarket).
- Refrigerator manufacturer says a condition of warranty is that the refrigerator must provide remote telemetry proving some warranty condition has been met (e.g. regular cleaning cycle operated).
Note: all these speculated reasons are unidirectional data flows. So even if a valid reason exists for Internet connectivity, why can't it be limited to unidirectional data flows?
> there is really only the one obvious choice of internetwork
Hey did you notice how when the freezer connects to the on-prem VPN tunnel, it never actually touches the internet directly and has zero attack surface, and yet can still send the necessary data anywhere in the world? And that could even be bi-directional, sending commands back to the freezer, if you really really think you need that ability for some reason (but you don't).
It's this, primarily, or most importantly.
I work in the restaurant industry. Health departments require you to assure them that your hot things are kept hot and your cold things are kept cold. The old way was staff walking around the store reading thermometers and writing temperatures down on paper; the newer way was your staff walking around the store inputting temperatures on an iPad connected to your business-management software. The trouble with that is that staff forget to do it (despite threats of discipline, and push notifications to their phones), or even blatantly fake it.
Failing to keep an accurate temperature log means fines, and other consequences, up to and including loss of your health license. I am 100% in favor of this legislation, and solid enforcement of it. The main improvement I would make is to harmonize codes and enforcement across municipalities. I'm in the USA, every county has their own requirements - and their own "portal" for submitting documentation. It's a massive pain in the ass to write Standard Operating Procedures for staff across multiple stores in multiple enforcement districts when there simply aren't any SOPs.
The majority of the fines our (responsible, and genuinely trying-to-do-it-right) company received were things like a staff member moving from a store in County A to a store in County B - just for a few days, to cover someone's PTO, or something - and inadvertently followed County A's logging requirements rather than County B's. Two genuine cases: logging every two hours, instead of every ninety minutes; using whole degrees, instead of tenths. We could appeal, but that takes a lawyer's time, so even if successful it wasn't always worth it.
Real-time data, from connected sensors, is unambiguously better. Staff get loads of time back, to make food, or help guests; all the logging gets done "right", for whatever jurisdiction; data's held by a third-party, so nothing can be faked.
That said, we're not connecting our infrastructure to the internet! We have a separate IoT sensor service that tracks the temps for us. I'm sure their security is also terrible, but that doesn't matter nearly as much. A year of their service is ~80% of the "routine", and totally innocent, fines and other costs that we were previously paying. That's the way all of these stores should go, in my opinion.
Compliance tends to warp the underlying goal quite a bit.
My org has lots of freezers that need to be at precise temperatures. Sometimes they drift so you have to check them a few times per day, log the check, and adjust as necessary. Doing this for a thousand freezers is tedious and error prone. Networking helps. “Cloud” lets it all be done remotely without getting IT very involved for vpns to be set up with specific vendors. It also helps with patching and auditing security so seems odd that these supermarket ones were out of date.
(Assuming they didn't also patch the telemetry software, but that is a much harder attack than just raising the temperature).
We've been telling people for years that naive threat modeling was going to bite everybody one day. And finally, here we are.
The big IT firms syphon off all the talent with compensation these fridge and supermarket companies can't match. You won't find much talent there. Who is brilliant with computers and would want to work there?
But look also at how underfunded the basic blocks of IT are, how open-source is left to fend for itself. It is the whole culture of modern IT to hack things together and not make them well-defined and reliable.
That may have been true 3+ years ago, but now there is a glut of tech people begging for jobs.
I believe gigantic fines are appropriate. They need to be of a size where the entire management team has significant financial issues for many years to come, even after having sold their houses, big cars, etc.
Nothing will improve until there are personal consequences.
Why the heck would this be necessary? We have VPNs holy crap, the entire genesis of the technology was to solve this fucking problem. If it's not literally intended for the general random public to be able to randomly visit and use themselves, it shouldn't be exposed on the public internet it should be (heavily restricted) LAN-only with remote access done via VPN. While it's even better if every single last thing is well secured at the client-level too, it goes a long way with ancient (or new, but with ancient-equivalent software design expenditure/decisions) kit to protect it at the network level.
Creating a false binary of "take it offline!" when businesses have very good reasons for networking many classes of industrial hardware seems self-defeating vs "just get the cheap bare minimum basic network protection done". I mean shit, even if somehow they've gotten suckered into freezers which demand access to a non-self-hostable manufacturer server (hey EU! here's something useful to regulate! if nothing else order manufacturers to have WireGuard end points for connection) they could at least have the firewall allow !RFC1918 solely and exclusively to that address on those ports.
However, we're talking about companies that put unprotected systems on the public network and did nothing about it even after getting three weeks advance warning.
You can lead a horse to water, but ...
I don't know the companies in question here, I'm not European and my EU family lives in France not Belgium or Sweden, but fwiw here in the states a lot of supermarkets are independent grocers or in a sort of formal or quasi-franchise arrangement where it's part of a "chain" but each store is still somewhat or quite decentralized in management when it comes to the physical stuff at the local store. Which is probably a strength in a lot of cases, makes it easier to adapt to local environment faster, but I suspect there's just plain a knowledge/skill/priority mismatch when it comes to IT. That'd also line up with some of the reports on this I read where there was some official central management system, except certain grocery stores just weren't using it?
The components of solutions for this exist and could even be really cheap, we're not talking much bandwidth here. Not hard to imagine even in a independent grocer association or loose franchise system they could tape together an SBC with preconfigured OPNsense or VyOS or whatever on it and a switch and be able to just tell stores "here, plug your freezers into this, access the portal here" as a default and it'd just route all their traffic to the franchise/IGA's management system. Someone still has to secure that of course, but at least it wouldn't be on the open internet.
Places that wanted to could of course do this other ways, I'm just saying that even with the very tight margins grocery stores tend to run on I think it's very doable to do better going forward, just will take some coordination and knowledge building. "The future is already here, it's just not evenly distributed" and all that.
For example, where does the VPN connect to? The vendor? Do you trust the vendor more than the internet. Sure. Do you trust the vendor, that does not manage the firmware of their devices and uses inbound connections in the first place, enough to put them on a VPN?
How many devices and vendors do you have? How many VPNs can you establish with the existing networking hardware, and how many can you manage without IP clashes etc. IP clashes work both ways, btw. The vendor has to manage this themselves also if all their clients want VPNs.
Retailers are enterprises with 100s of vendors that run on a shoestring. On top, they use a lot of legacy architecture and hardware. They have teams that have a hard time making an architectural decision in the best of times, but even less so under stress. It is probably the same on the vendors side.
In my mind, the correct solution is indeed to deny any inbound connection to the network, now, to prevent further damage.
Incident response is, in order:
1. preparation (failed)
2. identification (done)
3. containment <--- we are here (take offline, put manual rota in place to manage fridge defrost cycles)
4. eradication (VPN, etc)
5. recovery (take it back online, relieve manual rota)
6. postmortem etcThat's a really strong argument for why systems like this need layered security and should not accept incoming connections from the internet at large.
Each to their own… but why do fridges need to be connected to the internet in the first place?
The issue is this needs to be handled centrally by the parent org also. They have IT ofc but they usually do POS and desktops/servers. The OT/SCADA systems are usually totally separate and not touched by the white collar IT systems.
The question is whether this is attack has cost them more than that would have.
It is incompetence to not have this capabilty.
I don't care whether it's the Russians, Chinese, Norks or script kiddies. This should never have happened, especially since a similar incident happened in France three weeks ago so they got ample time to prepare themselves. They just don't care.
Came into the thread thinking it would be a new report about the same incident not a new one, yikes.
To be fair, wouldn't the chains probably be reliant on their vendors for an update? and turning off all the fridges for weeks is probably not a sustainable solution... :/
Meanwhile, you have stores like CircleK that, at least in most of Europe, operate a large fleet of externally connected monitors that keep track of temperatures, but staff themselves adjust the fridges and freezers. All you get by breaching those would be a bunch of temperature readings. Great in a pinch if you're looking to buy an industrial fridge or freezer, not so much for anything else.
These freezers are manually stocked and cleaned. If they need to be powered down it's a simple matter of a hardware switch that you flip when the unit is empty.
Hej! You are wrong, here in the middle of Stockholm got hit.
https://www.mitti.se/nyheter/icabutikens-frysar-hackades--fi...
ICA Baronen on Odengatan is one of several stores in Sweden that have suffered a major hacker attack. On Monday, their freezers were cancelled at 50 degrees.
This is in Stockholm - https://maps.app.goo.gl/BwbDST4dPVkjcdgA8
So no lack of IT workers/tech workers... Nor is it very rural.
>They likely order their system from a supplier, that should have alerted them and helped them, but apparently did nothing.
The first thing to do when you get access to these as a hacker? You disable the alarms...!
Now, the supplier should have set them up correctly over VPN, good passwords, Zero Trust etc but these an OT/SCADA are usually just open to the internet due to the simplicity of use.
(I mean, you need also a bunch of other things but yes)
I have some unfortunate news for you.
There's a lot of scary shit online.
The issue is this needs to be handled centrally by the parent org also. They have IT of course but they usually do POS and desktops/servers.
The OT/SCADA systems are usually totally separate and not touched by the white collar IT systems.
Want something simple? Don't connect it.
The true problem doesn't lie with Internet access, it lies with low quality software.
The fact was that it was never safe, it was never secure, but that taking advantage of those problems was hard, and generally not monetarily worth it, meant we ignored the issues. That's no longer the case.
As for low quality software, the floor for the quality that internet connected software needs to have is now dramatically higher than the quality that non-connected software needs. I think that in many applications, it may no longer be economically worth it, or maybe even possible at all, to hit the necessary quality.
First, I don't think people will give up the convenience of having instant access to the information.
Second, hackers are way ahead in terms of using AI. But it doesn't mean that it's going to stay that way. AI is going to be used for cyber security more. The implication is that the operating costs will be higher. In this case, it will make your groceries more expensive.
The auditors should have their own devices separate from the temperature control to monitor them. All it needs is a few sensors, temp, humidity, etc.
No need for complicated cloud this-and-that nor air-gapping a grocery chain. Keep it simple.
When I first started credit cards were not taken as online payments, the system just stored your card details and the amount. That was sent to server in the back office, which collected all the data, then every night it would make a dial up or ISDN connection to the head office and send all the data there.
At least with CC's this would get your account cancelled now.
Nearly all software is low quality software. The programmers at the fridge company are likely 4 underpaid engineers in Pennsylvania or some other equivalent.
Saying the problem here is bad code is like saying the problem with gun violence is humans are too squishy to deflect bullets.
The right answer isn't forever patching freezer software. The answer is either to not have them on the internet, or to have something in front of them, and keep that thing very up-to-date. (Or maybe several layers of "something" in front of them.)
And the simplest solution is to air gap these systems, then figure out how everything else should work.
In NL we use paper and pencils for elections because it makes fraud incredibly hard. It makes the process harder as well but the baseline is that it breaks nation-scale attacks. Why this level of thinking isn't applied to *our foodchain* is beyond me.
Ironically, were the monitoring system competently made, not only would it have been unable to cause this problem, but it would have also been able to alert about it in time to prevent significant loss.
> There is no reason for them to ever have an internet connection.
You can't know this.
How do you remote control the temperature without having any remote connection to them?
Have you worked in a small rural grocery store ever? You speak confidently about what they need/don't need, but you fail to account for a very basic use case, making me unsure if you actually know what you're talking about or not.
I'm not even sure of a scenario where remote control from off site is faster or more convenient. I could see a managers office wanting a connection to them maybe.
How often do you think the knobs would need to be turned? You think someone or something is constantly fiddling with the temperature knobs of each individual freezer?
I would bet that if it will ever be resolved, it will turn out to be just some bored kids[] who read the recent report from France and had access to Shodan and a competent AI model.
[] I'm using the term "kids" loosely here, not refering to a certain age group but to a certain mindset, similar to the original term "hacker"
This is a cyber-attack made possible through severe negligence on the victims’ part. It’s like a home burglary where the victim left the door wide open for months without ever checking on their home.
Therefore, the news should be about that negligence. That's the interesting part.
We keep blaming store owners for not being IT wizards.
I actively worked to make some specific pieces of software better.
I am not going to take responsibility over software that I am not designing and not maintaining a not paying for.
> Even in this thread I see varying advice like airgap, VLAN, or VPN. Well which is it?!
one problem may have more than one solution and this is normal
> We keep blaming store owners for not being IT wizards.
no, for ignoring security
Too often it takes repeated failures to clock in as a persistent problem at the necessary levels in an org.
If neither law nor insurance nor a CEO require it, it often doesn't get done.
Say in 2015 they installed those smart refrigerators, they could go from 20 people monitoring temperatures to 1. That was some profits being made by those companies, and they got to enjoy those benefits until now.
With AI finding exploits for so many things people wouldn’t spend time looking, we might need to go back to dumb fridges out of security concerns.
Now extrapolate to many other industries, the economic gain from AI might be hampered.
Will it work, or will it harden their resolve to fight?
Russia knows that if they want to keep annex countries, they'll have to take them piece by piece, with little resistance.
Your explanation and mine equally have zero credible evidence.
Given the fact that public awareness will quickly push companies to scan all software for vulnerabilities using AI, attackers will be making the most of this period to find daily flaws in hundreds of software packages that we previously saw as either "robust" or "robust enough" given the attack time/cost vs. incentive, before these defensive tools are deployed and operationally effective.
Nation state security services and rogue actors must surely all be scrambling to decide what to do with the millions/billions of dollars worth of exploit development time that forms their tool-set, including potentially using up zero day exploits etc.
The other factor is in the midst of such hysteria as "rouge agents" and other crap piped by the media, along with the actual risks of individuals with a few hours now having as much power as teams with weeks of time to spend did just a few years ago, coherent reports of what's actually going on is drowned in the noise. Attackers take full advantage of this.
Why you would assume that Russian strategy is optimal?
And the required investment is minimal.
Not saying we know anything about who did it, but senseless vandalism is usually a hallmark of kids.
It is staggering to me the difference in people that come to HN.
This is one of the thousands of ways a hybrid war is fought!
Basically making everything more shitty, and having people walk on eggshells - so that they'll vote for isolationist parties that are either neutral, or will outright appease Russia.
The idea is that no-one in Europe actually wants to go to war with Russia, and small remainders like these is enough to increase that fear.
Except that everything that is shitty in the EU right now has been the consequence of the last ~20 years of EU leadership and individual EU member policies.
Russia didn't increase the EURO supply during covid devaluing our wages and purchasing power. Russia didn't connect our fridges on the internet without a VPN with credentials admin:admin that any script kiddie could pwn. Russia didn't open our borders to unvetted criminals. Russia didn't offshore our industry leading us to fall behind on everything manufacturing related from cars to leading edge semiconductor fabs. Russia didn't create our NIMBYistic housing policies that made our current housing shortage. etc, etc, etc.
Blaming Russia (and China and Trump) for every single shortcoming of the EU right now seems the default scapegoat for the incompetence of everyone in the EU from politicians to business leaders. Have those people ever taken any accountability for their mistakes?
at risk of being autistic: no, not everything
plenty of shitty things is happening for reasons utterly unrelated to that
Some (or even all) EU policies being dumb does not make impossible to have problems caused by something else.
For example Russia was directly involved in making migrant crisis worse.
Yes, obviously Russia will weaponize every single weakness of the EU (just how Ukraine, EU, US, NATO and China will weaponzie every weakness of Russia right now), but ultimately it's still a self inflicted weakness, that the EU could close at the snap of the finger if it actually cared about stopping it, and Russia would not be able to weaponize any longer.
The part I referred to is in fact almost entirely solved, especially compared to the peak.
The https://en.wikipedia.org/wiki/Belarus%E2%80%93Poland_border_... is one of them more symbolic part, though that is only one piece of puzzle.
It was built by Poland in very non-EU fashion (see below, "shall not apply to the investment. In particular, the following are not required"), but EU at least has not sabotaged it and in fact silently ignored some topics.
https://www.sejm.gov.pl/Sejm9.nsf/PrzebiegProc.xsp?nr=1657 Art. 6. 1. - English translation below:
> Separate regulations—including the Construction Law, the Water Law, environmental protection laws, regulations on the disclosure of environmental information, the Geodesy and Cartography Law, regulations on spatial planning and land development, regulations on the protection of agricultural and forest land, railway transport regulations, and regulations concerning special procedures for the preparation and implementation of public road investments—shall not apply to the investment. In particular, the following are not required:
>1) a building permit or a formal notification;
>2) a decision determining the location of an investment of public purpose;
>3) the preparation of a construction design;
>4) the obtaining of any other decisions, permits, opinions, or clearances, or the submission of any notifications.
Again, EUs problems, are EU self inflicted and they could stop them if they chose to.
at least part of that is consequence of not only that
1. "isolationist parties" are inherently aligned with Russia and that it's good for Russia, in a zero-sum way, for countries that are not Russia to implement those policies.
2. Attacks like these are definitely done by Russia and we don't need to wait for evidence to connect them.
But there are problems with that:
1. "Isolationist parties" are nationalist, so if they knew that Russia was causing a problem for their own country, why wouldn't they oppose Russia? More generally, the claim that "small remainders like these is enough to increase that fear" is basically arguing that when a country engages in terrorism that somehow makes everyone else less inclined to take a stand against them. If that were true, I'd expect to see far more terrorism going on in the world than actually occurs.
2. The main "isolationist" policy is restricting legal immigration and/or enforcing immigration law. But if it's good for Russia that other countries do these things, it should also be good for Russia for them to do the opposite (because again, the idea that the way other countries govern themselves should impact on Russia, requires zero-sum thinking). If the powers that be in Russia believe this, we should expect them to pursue expanded legal immigration for themselves aggressively. But in fact their rate of legal immigration (percentage of population per year) is roughly on par with the USA, and it mostly comes from former USSR republics.
3. The truth of this narrative would be far too politically convenient for the people who have been trying to tie Trump specifically to Russia for over a decade despite many of their stories completely falling apart.
4. The proposal supposes that Russia assumes far more direct control over foreign populations than is plausible. It makes more sense to suppose that "desinformatsiya" is about chaos rather than complacency.
The hell makes you think food supply chain refrigeration is non-critical? How do you think safe food supplies work? This type of attack should be treated as an overt act of war.
Or another ai has broken containment and has a fixation about freezers...
Insurance risk and premiums might be the main drag on the AI future.
We can’t get food in the supermarket. People get scared. Better not poke the bear. This government sucks, let’s get the populist one that will fix all this overnight somehow.
Many of the previous cyberattacks were actually coming from the AI labs (RubyGems, HuggingFace, Australian medicare, PyPI, gym class software, code repositories, etc).
Imagine if somehow all the geeky/nerdy/difficult people would be removed from society and then something evil descends upon humanity, that needs out of the box thinking, but they killed all the strange ones, because they were no sheeps.
In the age of personal super-empowerment, there doesn't have to be a rational reason anymore.
For large companies, infosec will never allow some POS IOT device on their network without a 50 page form and large amounts of funding. The business however needs this to happen for whatever goals they have.
So the project manager goes back to the supplier, who recommend sticking it on a 4g simcard with a public IP address, and "install this app" to manage it
The project manger finishes the project, gives the app to the store manager, and rides off into the sunset, and nobody actually owns the equipment or even knows its there.
For small companies they likely don't have a technology provider at all, and just use the ISP's router.
Unsure what you mean by this. Somebody definitely owns the equipment.
Like, make their underinvestment in decent IT so expensive via fines that it is cheaper for them to just provision a sufficiently effective division within their company.
I think this is the only way to explain supermarkets how IT security works. What a shame about the lovely food :(
This is the bad part. Frequent password resets suck.
https://en.wikipedia.org/wiki/2026_French_supermarket_refrig...
but what about the deranged (and very prediction market–brained) version of this idea: the government develops ransomware, in a manner functionally similar to towing cars or confiscating illegal items, and uses it against companies in proportion to their revenue and obviously puts the money towards improving the ransomware
"the r in rsi stands for…"
i'm not sure it's something i want to see, but i don't know if it's impossible for it to actually come to pass or merely unlikely
(of course, cross-government versions of this are much more likely to come about but not as ransomware)
Who ever thought it was OK to control freezers over the internet should be hung.
I've got really bad news for you about how many critical systems are connected to the Internet.
Attacking power plants, grocery stores, sending drones into multiple countries over and over, setting up gun/dead-drops in German forests, sabotaging warehouses and factories, etc wasn't enough?
The hybrid war goes DEEP and has been happening for a long time. https://en.wikipedia.org/wiki/Russian_sabotage_operations_in...
(... in hacker forums)
but you just know someone will eventually if not already
because some vendor thought it would be easier for monitoring or firmware update
Then at least the hacker couldn't just switch off 1000 freezers that are full of food at once.
1) I hope they learn some hard lessons about security and what should be exposed to the internet vs LAN.
2) These are some evil ass people doing this shit.
A VPN per store would be too complex for the vendor so they just expose these to the open.
Same story as with all the hacked OT/SCADA systems in the past. Like those hacks where some SCADA system is open as a website with pump controls.
Imagine they where internet exposed due to simplicity. Imagine you are a store owner with 0 IT knowledge.
The issue is this needs to be handled centrally by the parent org also. They have IT ofc but they usually do POS and desktops/servers. The OT/SCADA systems are usually totally separate and not touched by the white collar IT systems.
2) Yes, obvious it is Russia doing this. Very handy to be able to disrupt food supply and they are also checking our response and time to restore.
I love how we're back to good ol' times that there is only one "evil empire", and it's completely impossible that anyone else from dozen of others possible bad actors is involved
/sarcasm
how is that possible? we're all dead from the plague.
This little gimmick cost Ahold in the Netherlands an estimated 300 M€ a few weeks back. Enough money for any firm to set the engineers to work? Where they waiting it out? Having read some more… another part of Ahold (the Belgian part) was hit and has (only now…) taken all fridges down from the internet. What is IT risk doing in a firm like that? Internet of Shit is a decade old. I am not amazed non tech sectors are perhaps a bit slower in IT risk management. I am amazed that an ongoing attack situation can still cause trouble 4 months after publication and months after board level headline grabbing numbers are widely publicized.
Me and my crew (yet to form ;) will visit any fridge for a measly 100 K€ a pop to take them offline. Book early, because once the script kiddies hit, we are on the road.
That does lead to harder things to think about. Multiple different attacks on EU supermarktes? Resilience? And then the questions I point to above: how hard is it for non tech firms to fix an emerging IT risk situation.
What sort of electronic shitfight is about to go down between major powers?
Have been for a few years now.
If I were a country with this kind of attack sitting around and I wanted to use it for war stuff, I’d probably wait until a hot war had started. Destroy the freezers when everyone is busy making weapons or manning trenches, might actually cause a food situation.
"Western elites have been accustomed for centuries to fill their bellies with human flesh and their pockets with money. But they must understand that their vampire ball is ending."
1. Connecting them was probably considered best practice when they were installed in 2005 2. Their support contract required them to stay connected
A VPN per store would be too complex for the vendor so they just expose these to the open.
Same story as with all the hacked OT/SCADA systems in the past.
Brexit was the first to burst and despite their support of Ukraine, can at any moment capitulate to the ghost of nationalism and delusional escapism that is "white power". Americas on the verge of sinking further into the same. With those two entirely preoccupied, of course europe is next and already has the same underlying current.
I'm quite sure your average American company is just as willing to plug random shit into the internet.
Hell, there are probably hundreds of reports out there of American companies failing to secure their AWS S3 buckets, so you don't even have to plug anything into the internet.
[1] https://signalandsilence.substack.com/p/i-think-someone-hack...
[2] https://www.foxnews.com/us/mysterious-freezer-failures-us-mi...
Therefore even the US Army has not been better prepared than supermarket chains.
Hoax?