Hacker News Reader

Comment by renyicircle | Hacker News Reader

> Connecting your case controller to the internet offers further opportunities to generate savings. If you can diagnose issues and control the case remotely, service engineers needn’t be called out to simple situations like starting a defrost. And being able to compare performance and efficiency across an entire estate of stores makes it easy to spot patterns and make incremental efficiency gains. (https://www.danfoss.com/en/service-and-support/articles/dcs/...)

I wonder if the incremental efficiency gains from this outweigh the damages. That doesn't even sound like a bad idea, unless of course nobody stops to think "but what if someone hacks this".

View on HN

People underestimate just how much effort goes into minimizing food waste and shortages in these types of industrial systems. The entire supply chain coordinates using these embedded sensors and control systems to make sure you always have safe food available at the lowest possible cost.

Connectivity in these environments is often challenging, it isn't like connecting a consumer fridge to wifi. Sometimes you are bridging weird networks that multiple parties need access to. It should be secure but that isn't as straightforward as one might assume. Many of these systems have very long lives, so there are many limitations that reflect the technology that existed when they were built.

All of this is invisible to the average person but it is what makes the world work.

And yet, supermarkets existed in the year 2000, selling products at comparable or lower inflation-adjusted prices, without any of this connectivity.

Yes, a lot of it exists to squeeze a bit more of a margin out of the business, but come on. The alternative isn't going back to the Middle Ages, it's shaving 0.1% off the profit margin. It's OK to wonder if the risk is worth it.

How would you stay in business if your competitor was offering 20% off your prices because they had better logistics and tracking? Ask consumers to eat the cost due to their latent demand for, uh, inefficient delivery and food spoilage?

There are numbers between 0.1% and 20%

This is already the case in the grocery market. There are various leaner and more efficient operators offering the same goods for lower prices, sometimes a lot lower than just 20%. However most people don't think to shop at the various Grocery Outlet's or Food4Less's. Working class is wise to it but not really further up the class ladder. They don't think much at all and reflexively go to their ralphs, which plays the opposite game where they have higher prices such that they might occasionally mark them lower and make you think you are getting a deal. Some people are even going to erewhon.

Are you saying that stores in the 2000s were throwing away 20% more of their total revenue than they are today? That sounds crazy, especially given that the prices - again - seem pretty much the same or higher if you correct for inflation.

But more to the point, there's plenty of grocery stores that ask 20% more than their competitors and are doing just fine; in fact, constant price changes and promotions make it impossible for consumers to really compare if Safeway is cheaper than Whole Foods, so people usually go to the place that's physically closer, has better selection, or better vibes. I can't even know if a 12-pack of Coca-Cola is going to be $6.99 or $9.99 today.

Choose your own number, but it has to be large enough that supermarkets decided the CapEx was worth it, and also large enough that once one did it, the rest followed suit.

It's not even significant cost. The main cost is still the hardware, not electronics driving it, and especially not a tiny module that talks with electronics to send/receive commands/metrics

Coca cola is a bad example as it does not require refrigeration during transport and does not spoil fast (at all?).

Except that in this case, the possibility of something like this probably is worth it, because system vulnerabilities are solvable problems, while ignoring the ability to improve system efficiency is a static stance. A lot of things have driven up the price of groceries since that time. You mention "comparable or lower inflation adjusted prices", but this hides the fact that the inflation adjusted basket of groceries in the US is near inflation (105% vs a baseline of 94%), while the input costs to that supply chain (like labor, and diesel) have gone up faster. The efficiency isn't doing nothing, grocery prices would almost definitely be higher without it.

> the inflation adjusted basket of groceries in the US is near inflation

You must not mean what you have said here. If so, you are double-dipping.

> system vulnerabilities are solvable problems

Which apparently why after >2000yrs of effort from both sides we now, finally!, have locks that no lockpicker can compromise.

That's not what "vulnerabilities are solvable problems" means, except maybe for cybersecurity maximalists who believe that, "bro, add passkeys, and just one more thing, attestation, and W^X, and ..., and there will never be vulnerabilities".

Vulnerabilities will always exist. Systems will always be hackable. You solve the problem by patching what you can, and cleaning up after what gets exploited, if and when it gets exploited.

> because system vulnerabilities are solvable problems

If only…

That's why I spoke of them in the plural. Obviously the possibility of a system hack (presumably) won't ever completely go away, but individual vulnerabilities as they appear in specific systems are usually tractable problems, especially if you narrow it to things that one wants solved rather than all possible vulnerabilities regardless of severity or accessibility.

Grocery stores infamously have some of the lowest margins in all of retail and typically hover in the 1-3% range. As a sector they have about zero room for inefficiency and have to be this aggressive about chasing operating cost reductions.

The increased profits that have occurred in recent decades have landed with the big food companies, not with the retail chains selling their products at very little margin.

Wonder the margins on those $9 boxes of Cheerios at major California grocers.

They did promise to increase the cost of breakfast a couple years ago.

Well, for whatever it's worth, I think the closest manufacturing facility for those would be in Iowa. They had a plant in California but it closed in 2015.

AKA, the decade where everyone bought way too much into the 'Just do JIT supply chain for the manufacturing location to warehouse tail and pretend we are Amazon' bullshit.

I put it that way because it is short-sighted to not consider that transportation costs will always increase over time (i.e. fuel prices, wages of drivers) and shipping a final product over that long of a distance (i.e. at least half of the country, and instead of shipping grain to process it is finished boxes which is less efficient for volume) and yet they made that choice.

Where it got nasty is that when the average consumer base decided it was fine to pay dealer markups on cars because of the parts issues that industry saw from their JIT shortsights, everyone saw dollar signs instead of a problem to fix.

Wonder the max price a company would be charged to ship a trailer of the largest boxes of Cheerios from say Maine to San Diego. Maybe fifty cents a box? You got me curious about the breakdown of the $9.

I'll admit I'm curious about the cost breakdown myself.

I did go ahead and do a query against gemini, and the estimate is 25 cents a box after considering long tail expenses (i.e. fuel, CARB premiums, etc.)

So, Less than you thought and way less than even I initially expected.

Mea culpa on my end, I wonder where the other parts of the price are coming from.

I do have other thoughts, but don't know California labor laws and the like enough to guess.

e.x. I know one big factor in the 'slowing' of inflation of grocery prices in my state over the last 20 years, has been the erosion of pay/benefits for workers. If California is still guaranteeing that for the workers (decent health care, at least a couple weeks of vacation time, etc.) and it's built into the cost... makes sense and TBH I wouldn't hate paying extra for that. Grocers here have with only a few exceptions gotten terrible.

A lot of supermarkets here own their own "budget" food brands and for many items they are not that far off from the competition in price.

That's the work that goes into hiding the "other half" of the inflation (i.e. the 2x diff between reported inflation and what you can actually see in your life).

Yeah, I worked at one during that time. The amount of food we threw away on a regular basis was incredible.

Seems like these days stores just keep it on the shelves instead of throwing it away. The produce quality in even the more expensive stores here is abysmal.

I think it's an interesting question.

I would be interested to know what the damage caused by this attack is compared to savings over a system where thermostats need to be adjusted on-site.

No, this “user story” does not mean the customer actually wants or uses the feature. You do not need remote command and control to tell if your truck chiller is working.

You don't need remote command and control to tell you if your single truck chiller is working, you can just go look at the thermostat. But a company needs it to confirm that the chillers are working and at a safe temperature across all 1,000 trucks in their fleet.

That's remote monitoring, though, not remote operation. Very different threat vector.

Along with remote monitoring comes automatic remediation. "Your reefer is set to 40F degrees, but your cargo manifest says that you're hauling frozen chicken, we set it to 0F."

Or "Your reefer is set to 35 degrees but you're hauling chilled beer, we set it to 40 degrees to save fuel"

What am I missing? Of course the user cares if their food has been stored at the proper temperature from production to transit to the shelf. They don't care that this specific implementation clearly had a security vulnerability, they just want to check out and get back to their lives (and not get sick), but there's no way this whole setup was completely unnecessary to produce that desired outcome, flawed or not.

Maybe we throw it all away and go to 1990s-style grocery chain management, hire a bunch of people, and costs go up while reliability and monitoring shifts to humans. But that doesn't mesh with consumer expectations on price, which is probably how the whole system originated.

But you’re going to adopt a platform that meets the needs across your enterprise, so the capability is often there even if one situation may only call for monitoring.

Further, once you have hundreds or thousands of devices, “just call someone to turn it up/down/off” doesn’t scale.

Further, once you have hundreds or thousands of devices, “just call someone to turn it up/down/off” doesn’t scale.

And yet, somehow it did scale just fine for a hundred years before we hooked everything up to the internet.

And yet, progress marches forward and people want great efficiency and flexibility.

If you measure progess by reports of equipment that was needlessly connected to the internet getting hackedn then sure, those are way up.

Yeah, and people sewed clothes per hand for hundreds of years until someone automated it with a machine.

You need to save labor somewhere to free it up for other jobs that give you technological progress. If everyone would still be occupied plowing fields with oxen, we wouldn’t have cars and computers now.

And you know what the customer needs? Or is this just a personal feeling. There’s multiple people from the industry chiming in on these threads as to why this is needed.

It looks more like they set a target level for shortages and then try to minimize waste without exceeding that level. Almost every time I go to Safeway, they have run out of some items I intended to buy. Trader Joe's is more reliable, probably because they have fewer items in stock.

> The entire supply chain coordinates using these embedded sensors and control systems to make sure you always have safe food available at the lowest possible cost.

I think the entire supply chain is making sure it has maximum profits extractable by law and any other emergent phenomena you observe is serendipitous and probably temporary, damaged or dropped as soon as the regulatory environment or marketplace changes to allow another dime of profit.

and the law says food must be safe, and to attract buyers costs must be similar or lower than the competition.

Of course in a cynical view companies only try to "generate profit" but usually generating profit is tied to providing some sort of service or product, and people like to buy good products

There’s a whole massive industry out there of companies looking to make 2% efficiency savings on everything, and VCs looking for this. The glossy presentation might have even said 2FA somewhere so yes it’s definitely safe.

The perspective of resiliency and how much will you lose if things go wrong seems kinda… new?

Funny that it should happen in France which has been at the forefront of European self resilience.

Good find.

The biggest issue is that many of these IoT devices are badly secured. I spent years trying to convince people to build more secure IoT systems, but they always preferred cheap and easy solutions.

I would have said let them be doomed. But due to late stage capitalism so much has been centralized that everybody is doomed in many cases...

I look forward to the safe diversity provided by early stage socialism!

good on you

> I spent years trying to convince people to build more secure IoT systems, but they always preferred cheap and easy solutions.

For me this has always been true of systems in general all across the decades that I've worked with technology. It's like pulling teeth to convince anyone in charge that backups or security are even a little bit important until after there's been a costly disaster, then it's all about "Why didn't you warn me this could happen?" - Umm... You mean like every single day for many months until you told me to shut up about it because my paranoia doesn't justify the tiny expense of ensuring you don't lose many tens of thousands of dollars or years of irreplaceable work or important records?

I went through a manual you mentioned in another comment. It listed "High level of security against undesired user access" as one of the features but it wasn't expanded upon. So I did more searching and found cybersecurity best practices from Danfoss.

https://www.danfoss.com/en/service-and-support/fix-and-troub...

They highlight connection through a firewall, mention secure remote access methods, monitoring for data breaches, strong access controls, etc. Reasonable, but these are just recommendations, so the stores could have followed all of them or none at all.

An article about Danfoss vulnerabilities mentioned in a different comment (https://claroty.com/team82/research/freeze-the-controller-de...) says

> we searched publicly available internet-wide scanning platforms, including Shodan and Censys, for signatures associated with its management interface. It didn't take long to find what we were looking for. Numerous controllers [2765] were directly accessible from the public Internet.

So none at all is a possibility.

I get that having networked devices is a very nice to have feature. However, putting them directly to the open internet is bewildering. Any person worth their salt in IT would know that's a bad idea. Were these people never brought into the conversation? Do they connect to the store's WiFi so that IT isn't actually needed to install once the WiFi creds are obtained?

I imagine a very conscientious vendor could easily set up a system that detects if the device is trivially exposed to the internet, causing warning messages and beeps and general pestering.

... And then the customer demands it be turned off, and the customer is always right...

Yes, there is little demand for this sort of thing when the potential loss, should it even come, is already insured. Your customer might actually not be users themselves but their insurance companies. It may be a better model to say pitch to them and then have them offer discounts to their clients who are using your tech that mitigates whatever % risk. Someone pitched that car probe to Progressive after all.

IIoT (Industrial IoT) doesn't have time to deal with the load of bullcrap that cybersecurity is. Half of it is not even about security, but setting up moats and keeping control over users, which is very much not what you want to prioritize when setting up industrial infrastructure, and is rightfully rejected by engineers working there, who are concerned with safety and reliability first and foremost.

Though I'm afraid things are getting worse, or at least it looked so a few years ago, back when I worked in this space. The hot thing in IIoT / Industry 4.0 was basically taking all webshit technologies and deploying them on industrial hardware, to "leverage mature innovation and OSS ecosystem" or such nonsense. The problem (one of many) being, once the engineers were done trimming excessive security measures that negatively impacted safety, they were still left with node.js running on industrial controllers, now even less secure in general than it was originally.

Safety and reliability without security? Tell me more.

Emergency overrides.

Did you know that buildings have doors that cannot be locked to prevent them from being opened from inside, because otherwise in case of a fire (or in the US, active shooter), people will die if they cannot exit the building?

Did you know that many industrial plants intentionally disable certificate revocation checks, because having a certificate expire on you for whatever reason is a sure-fire way to break the plant (and in some case possibly kill people), and overall a nuisance when trying to keep the plant running, while the threat it protects against can be handled elsewhere in the stack?

Security and safety are very often at odds - not just in emergency, but during regular operations.

They're on Shodan and there's an admin account with a password that apparently *cannot* be changed.

So you can find one, log in, and have a look for yourself.

You shouldn't.

But you can.

But you shouldn't. Okay? Don't.

Thankfully these kind of attacks make pressure to finally have liability in software like in other industries.

The people who should be held liable never will be. Their money is like an impenetrable shield to any sort of accountability. They'll just scapegoat the guy that warned them about the risk repeatedly for not doing more to prevent it.

It has to start somewhere, even if imperfect.

I have been in meetings were the Data Security Officer refused to sign off on specific projects, without having a proper pentesting report in place, for example.

Because legally it would be their responsibility, if something would happen.

I wonder if, following the US army story in August, someone might have bet on a prediction market “Danfoss refrigeration systems will defrost after suffering an already-disclosed vulnerability”.

Why are technicians (service engineers!) coming out for a simple defrost in the first place? The normal defrost cycle should be automated. For anything else you need someone to come out anyway because something is wrong - there is no point in starting a defrost cycle if the root cause is leaking refrigerant - fix the leak and refill the system both of which need to be done in person.

Because in some ticket tracking system, some customer with the most tickets was nearly always calling for a tech to physically start a defrost. Maybe solving that accounts for 2% of support overhead. It’s best to lose that customer. Second best is telling them defrost is remotely-triggered, and then not doing it. Third best is actually triggering it.

And I would bet dollars to donuts that someone at that company would respond "who's going to hack a refrigerator? Heh"

Send those asking that question the scene from Silicon Valley

Seems like two totally different issues. One is cost and one is efficiency gains.

For the latter I would imagine you could have a short (maybe a month) logging period where you connect the fridges and collect data and then comeup with some offline system to manage them generally for better efficiency. You could even have a test group or single fridge that you keep connected to the internet for future data points (like in case of a giant heatwave or something)

The social media image for that page: https://www.danfoss.com/media/11988/nbu8570_clean.jpg

So we spill cold air on the floor and heat the place with AC, then talk about low margins and having to save energy costs. Does anyone see the issue?

The issue is that you conveniently leave out parts of the equation like how usable the design is and how it affects customer purchase decisions.

I thought about it and considered those "considerations" to be worthless. You didn't do any better, just name dropping stuff is like dropping two empty boxes with a word written on them in marker, "just look it up". Nah, if you think you have a point, you gotta make it.

> how it affects customer purchase decisions.

If nobody wastes energy in a stupid way like this, what changes? Are they no longer buying cold things? So what, then you sell more of the stuff that doesn't need refrigeration.

If there were two identical supermarkets, and one had running taps, and the other turned off their taps after using them, and paid their employes 5cents more a month, I would shop in the second one. But I can't, I don't get the choice, because a bunch of bunch of marketing people convinced someone to get an "edge" over the competition, and then told everybody else they have to, too, or they'll FaLL BeHiND.

Being able to remotely trigger a defrost cycle makes sense. Being able to remotely trigger a defrost cycle to run for so long that it wastes products does not.

Did everyone forget about Therac-25?

We all keep a shotgun next to the printer in case it makes an unexpected noise here because the average HN reader knows a fridge manufacturer knows nothing about infosec and the lowest-bid contractor they farm the connection does the minimum to check the boxes. So yeah, we've been rightfully complaining about why does my icemaker need an internet connection for years.

Lets combine some testy topics: will LLMs make this any better? On the one hand, if it's easy for fridge manufacturers to churn out code, will the slop create enough cracks in the icemaker to sail the titanic through? Or will the lower barrier to pretty good security agents get more security into the IoT field (assuming it's taken advantage of?) Or will it not matter because the store manager will just set the web portal password to 123456?

The combination will be based on the proposition that AI must consume all and nothing shall get in its way (even if it AI), whilst maintaining the existing devotion to capitalism above all:

- Cheapest possible test-passing AI generated code with minimal check-box style security audit.

- Looser and broader definitions and greater penalties for hacking.

That could be done by the servicemen or managers on-site. There is no reason at all to have remote control fridges. All this really does is make some higher up feel more importantl because they have a centralised controller, while providing very little real benefit.

Anything you might want to change with a freezer already requires someone physically there. It's like making a socket ratchet have a remote controlled direction toggle. Or a remote control button to light the inside of your money box on fire.

The benefit is the same as making websites remote controllable. A person then doesn't have to drive in to flip a switch, which means fewer people are needed to operate more systems, saving the company money.

Presumably the grocery stores are already staffed by at least one employee capable of flipping a switch, no?

Stores are now installing e-ink price tags to save labor updating price labels. They're not going to turn back from centralized HVAC/fridge monitoring and control.

Unlike fridge remote control (note, not remote monitoring but remote control) e-ink price tags actually make a good deal of sense. The prices are already managed centrally so it reduces the labor to needing to accurately document which piece of hardware is at which shelf location as opposed to needing to manually check that each tag has the up to date price. (TBF OCR is now good enough and mobile SoCs now powerful enough that you could probably automate most of the traditional task but that wasn't historically the case.)

No they are doing it so that they can change prices more quickly and thereby extract more profit. Next (or already current) step is adjusting prices dynamically based on nearby customers.

Switch to do what? If food is in it you can't change anything. If food isn't in it then why is it running or why wasn't it defrosted when it was emptied?

If you home refrigerator could be defrosted remotely, in what scenario would you ever do that?

Agreed, logging temperatures without the ability to actively control the fridge seems a much better approach.

Our company makes IIoT gateway devices that log parameters from the system and push them to a read-only location. I've personally reverse engineered a dozen or so management protocols to get the data out because pretty much no vendor supports pushing temperature data to a centralized place. Those that have monitoring cloud software only support their own devices.

Every now and then our customers ask if we can add remote control for specific parameters but we've always turned them down, citing liability reasons.

Re: legacy Danfoss systems such as AKM, 720 and 850 there is no way to have read-only access (for different reasons in each case). I haven't been able to find weaknesses in the new 800A yet, but that's only because I have not had to try yet.

The vulnerability making the rounds in the comments is specifically in 800A: https://www.claroty.com/team82/research/freeze-the-controlle...

This sounds like someone (or someone's AI agent) saw the CVE and did a one-off drive-by attack instead of systematically targeting refrigreration systems and trying to find vulnerabilities.

The article is dated August 2026. Synacktiv (a different company) reported the very same vulnerability for older system managers/controllers already in 2023. Their report[1] says Danfoss told the products are EOL and should be upgraded. At no point did Danfoss fix the same issue in 800A. Typical.

Back in 2016 when we were integrating their 850 series system managers we encountered an issue with the HTTP API returning 400 series errors over slow links. We sent them PoC code written in Python 3. One month later they sent us an email, complaining that they first had to backport the PoC code to Python 2.7 as that was the newest version they could install. They also attached a "fixed" PoC code snipped where they had removed the artificial delays that did not trigger the issue. Danfoss never did anything to actively fix the issue.

One year later I browsed changelogs for the commercial RTOS that Danfoss used and realized the issue was fixed upstream in 2010 or 2011. That means their AK-SM 850 had at least six years worth of unfixed bugs.

[1] https://www.synacktiv.com/sites/default/files/2023-10/synack... (despite saying SM800, the firmware in question is for AK-SM 850)

Edit: The parent commenter seems to work at Danfoss on their Leanheat platform.

Alas, not anymore. Still got friends there though.

I worked with district heating, no idea what the cooling guys are/were doing. It's a big company.

In the recent heat wave being able to remotely tell which systems are about to fail helped quite a bit because you could take parts of the freezers offline to save the rest. There is no technician permanently on site at any of large retailer stores, you have mobile teams.

Reply on news.ycombinator.com