Hacker News Reader

Comment by trio8453 | Hacker News Reader

Kind of insane how much of the front page is about security compromises today.

For 50 years, computer security has largely been built on the theory that most targets aren't worth the relatively high effort to compromise, so it's not worth it to spend the money on security. We're now reaching a point where the effort to compromise is dropping close to 0, which means we've got 50 years of insecure software to reckon for.

The advice today is the same as it has ever been, though. Always assume that any networked computer is or may become compromised. Do not do anything critical on a networked computer, because it is or may become compromised. You can still use computers, but assume that a bad actor may take control at any time, and plan for that eventuality.

For example, if you are hooking up a networked computer to your town's water supply system, ask yourself "what would happen if Iran got complete control of this system?" If you're putting your school system's students' mental health data into a networked computer, ask yourself "what would happen if all of this data became public?"

A better framing is that security has largely been built on theory that most targets aren't worth the effort to target, so security spending is focused on preventing/mitigating untargeted spray-and-pray attacks.

Computer security has largely been built on the “not enough bears” theory.

Every target is worth attacking, but there are not enough bears to eat everybody. Sure, the bears are breeding, but exponential growth does not look threatening until it suddenly is.

> Sure, the bears are breeding

I don't think the "breeding rate" was the limiting factor.

It reminds me of a bit from the book Lying For Money [0], where the author notes that you can often find multiple layers of scams attached to other scams, just like meta-parasite in biology.

[0] https://www.simonandschuster.com/books/Lying-for-Money/Dan-D...

Lotka-Volterra predator prey dynamics will emerge.

My new security through obscurity is to collect esoteric modern LLM written software, like this sweet gem of a terminal editor written in fortran: https://shithub.sh/fortrangoingonforty/facsimile

> what would happen if Iran got complete control of this system?

What would happen if America got complete control of this system?

Oh wait. Oops.

> The advice today is the same as it has ever been, though. Always assume that any networked computer is or may become compromised. Do not do anything critical on a networked computer, because it is or may become compromised.

This is completely unrealistic advice that 0% of people follow.

It might need to become a realistic view that >0% of people follow

I agree it's rarely followed. I don't think it's unrealistic. No one needed to connect water infrastructure or student data into the Internet, but we did, and the obvious thing happened.

https://www.bbc.com/news/articles/c934dq95zpgo

https://duckduckgo.com/?t=fpas&q=public+schools+ransomware&i...

Your on /hn/ and ¡¿don't?! have an entirely-local subnet, offline, for your NAS?

...with a hint of sarcasm, but the percentage is definitely a non-zero number.

In this scenario, is the Network-Attached Storage attached to the network (and so theoretically vulnerable)?

It seems like the only “safe” option is a PC (+storage) that never attaches to a network, but that extremely limits its usefulness.

This is correct; two of my three ZFS implementations are entirely local... the third (public-facing) covers the other twos' "limited usefulness."

Perhaps a vulnerability still exists (for one offline NAS, only): I use dedicated NICs (on each end) to direct-link my online NAS (which is otherwise local network, only, on a separate NIC), for routine backup.

Perhaps this leaves a bridge-LAN vulnerability?

But one is entirely offline, as are all its clients (dumb TVs, mostly).

Your NAS is not critical. The plan if your NAS gets compromised is to say "dang" and wipe it and re-download all your movies or whatever. You can connect that to the internet.

Stuff like your town's water, transportation, and medical infrastructure and employee and student data are critical. Your car's drivetrain and safety features are critical. Your home's HVAC can be critical in certain times of year. These things should not be connected to the Internet.

Is any of this actually an LLM-related compromise?

How do you transfer data then ? People walking with usb sticks ?? lol

It's very impractical, agreed, and even USB sticks, etc. have carried viruses.

I think we are about to enter a world where security compromises occur at least an order of magnitude more often due to AI. The fact that AI is sometimes wrong benefits the attacker because they can just automate attacking with agents until they breach a system and the failed attempts don't matter.

Nicholas Carlini has a good talk about the risks on his homepage https://nicholas.carlini.com/ (can't link to the talk directly for some reason)

The corollary is that service access will become increasingly penalized, authenticated, and finicky as the most general first line defense against automated "informed brute force" attackers will be to slow or ban access that looks like them.

If you've got a lot of roving bandits with fancy lockpicks, you're village needs to harden the walls, reduce the number of entry points, identify and document entrants more universally, check under more carriages and inside more trunks for stowaways, etc. And with all that comes delays, imprivacy, accusations, false positives.

The low trust world that we're headed towards isn't going to be nearly as fun.

Building on your analogy: Currently we spend a lot of time and effort making sure our walls appear nice and inviting, with lots of convenient entry points where advertisements can be placed. In the future we will build ugly walls with a few well guarded entrances.

What I mean is, look at how much time and effort we spend on making software services beautiful and branded, and so little time spent on security. In the future we'll probably have fewer webpages and more hardened text-based APIs that end users interact with via their AI agents.

This is just the beginning, once kimi k3.1 is out with cyber curated dataset post training we will likely see far more advanced attacks.

For now most of these appear to be related to supply chain attacks which can be avoided with basic discipline and using features that browsers had for years: https://developer.mozilla.org/en-US/docs/Web/Security/Defens...

SRI only really works if the 3rd-party script never changes, in which case why is it a third-party script?

If it changes and the hash no longer matches perhaps due customer changing config then the site is faced with the challenge of either the browser refusing to load the script (with potential breakage) or reporting the change which then is probably ignored

It's not really a simple fix for the way the web is built today

usually scripts have versions, this protects you from compromised cdns as well where you publish assets in a different location compared to where your frontend is served.

If a library serves a script and your site manages user funds you should probably avoid using it or use a cdn that does.

I am not quite sure, but there's also other integrity mechanisms that gurantee that a file was signed by the library developer via a served header with the signature of the file.

Why would you even load a library script from a third-party when hosting it yourself is more performant and more secure?

There's a lot of very weird websites. You still benefit from checksumming your own assets to prevent cache poisoning and so on.

Letting any vendor arbitrarily update code running on your customer's computers without even your knowledge is insane.

Welcome to the world of SaaS services on the web!

Integrity reporting can detect when the script changes

At least people aren't going to complain today that it's all AI related submissions.

Hey now, that's for tomorrow when someone writes an article about LLM contribution to these security events.

Well, it's called Hacker News after all :-)

People complained about excessive AI posts. Be careful what you wish for!

P(doom) on the rise

Reply on news.ycombinator.com