kchr

The limitations of old game platforms didn't vanish, they are still used (and being re-discovered by new generations). One of my favorite games on this side of the new millennium is Celeste, for example.

Some indie studios are even producing new games for GBA, GB, NES and other platforms from the 90s, sometimes including booklet and packaging!

Yeah, WinPE media tools have been around for years. Here is an article from 2021 (although it has been a thing long before then):

https://learn.microsoft.com/en-us/windows-hardware/manufactu...

Still, customizing the toolchain to fit this particular scenario and making sure it works, in two days, is commendable effort.

Yes, but it's not included in the upstream Ubuntu security repository. In fact, it's not available via any repository AFAIK. It updates itself via fetching new versions from CrowdStrike backend according to your update policy for the host in question. However, as we've learned the past days, that policy does not apply to the "update channel" files...

Ubuntu LTS has support for 5 years, can be extended to 10 years of maintenance/security support with ESM (which is a paid service).

Same with Rocky Linux, but the extra 5 years of maintenance/security support is provided for free.

Some comments theorise that the NUL-file was deployed to fix the issue with the corrupt file that caused the crashes.

CrowdStrike uses eBPF on Linux and System Extensions on macOS. Neither if which need kernel level presence. Microsoft should move towards offering these kind of solutions to make AV and EDR more resistent on Windows devices, without jeopardising system integrity and availability.

> Detecting system crashes would be hard.

Store something like an `attemptingUpdate` flag before updating, and remove it if the update was successful. Upon system startup, if the flag is present, revert to the previous config and mark the new config bad.

Absolutely, training is key. Alas, managers don't seem to want their employees spending time on anything other than delivering profit and so the training courses are zipped through just to mark them as completed.

Personally, I don't know how to solve that problem.

I totally agree. In my current work environment, we do deploy EDR but it is primarily for assets critical for delivering our main service to customers. Ironically, this incident caused them all to be unavailable and there is for sure a lesson to be learned here!

It is not considered a silver bullet by the security team, rather a last-resort detection mechanism for suspicious behavior (for example if the network segmentation or access control fails, or someone managed to get foothold by other means). It also helps them identify which employees need more training as they keep downloading random executables from the web.

I'm curious, what did your deployment plan look like? Phased/staggered, if so how?

A good reminder of the fact that your Thursday might be someone else's Friday.

The file extension is `sys` by convention, it's nothing magical to it and it's not handled in any special way by the OS. In the case of CrowdStrike, there seems to be some confusion as to why they use this file extension since it's only supposed to be a config/data file to be used by the real kernel driver.

Which also explains why they, only if needed to cover their back legally, confirm or deny details being shared on social and mass media.

> Many of the companies have Crowdstrike enabled and automatic updates turned on to check some audit box. They have to keep the updates going out regularly.

While many companies probably do that, it's usually not required if you can argue for an alternative approach and how it fits your risk appetite better (e.g. progressive updates on a routine schedule).

My experience is that in these workplaces where EDR is enforced on all devices used for work, your hypothetical is true (i.e. you are not expected to work on devices not provided by your employer - on the contrary, that is most likely forbidden).

Most AV and EDR solutions support exceptions, either on specific assets or fleets of assets. You can make exceptions for some employees (for example developers or IT) while keeping (sane) defaults for everybody else. Exceptions are usually applied on file paths, executable image names, file hashes, signature certificates or the complete asset. It sounds like people are applying these solutions wrong, which of course has a negative outcome for everybody and builds distrust.

This. From all the comments I've seen in the multiple posts and threads about the incident, this simple fact seems to be the least discussed. How else to protect a complex IT environment with thousands of assets in form of servers and workstations, without some kind of endpoint protection? Sure, these solutions like CrowdStrike et al are box-checking and risk transferring exercises in one sense, but they actually work as intended when it comes to protecting endpoints from novel malware and TTP:s. As long as they don't botch their own software, that is :D

Highly unlikely anyone except governments or top-paying corporations with custom-negotiated T&Cs will see a detailed post-portem, unless someone blows the whistle. Would love to read an AmA.

It is untrusted data in the sense of files being read from disk that are not part of the signed kernel driver code.

I suspect that regardless of which country CrowdStrike is from, the question would still arise: "should we really outsource information security protection of our critical infrastructure to country X?"

Naturally, the question of malicious intent would most likely be more or less prevalent depending on whether country X is considered an adversary.

> Some sort of sand box for all these kernel access

Yeah, like eBPF in Linux or System Extensions in macOS.

Instead of calling out AV/EDR solutions as malware and spyware, I have a better solution: Stop using your workstations for private stuff. They belong to the company, and they are a liability since you use them to access the company environment and could cause damage if actual malware would find its way onto it. Use a separate device for private stuff if you want privacy.

Last time I checked, CS primarily runs in kernel mode on Linux and only fall back to eBPF if the kernel version is not supported. When in eBPF mode, they call it "Reduced Functionality Mode (RFM)".

Has this changed?

CS doesn't force you to auto-upgrade the sensor software – there is quite some FUD thrown around at this moment. It's a policy you can adjust and apply to different sets of hosts if needed. Additionally, you can choose if you want the latest version or a number of versions behind the latest version.

What you cannot choose, however - at least to my knowledge - is whether or not to auto-update the release channel feed and IOC/signature files. The crashes that occured seems to have been caused by the kernel driver not properly handling invalid data in these auxilliary files, but I guess we have to wait on/hope for a post-mortem report for a detailed explanation. Obviously, only the top-paying customers will get those details...

Using it and loving it. Security aspects aside, it's also the best tool I've tried for proper asset management in multi-cloud scenarios. With the graph feature you can write queries for basically anything, across all accounts if you wish.

You debug it with shellcheck[0] and `set -ex` (which will trace each statement on STDOUT).

[0] https://www.shellcheck.net/

NameISP does this. Auto-renewal and payment by invoice.

What you are describing is a forwarding DNS query resolver, which can be configured to directly answer queries without forwarding them upstream in the DNS hierarchy. This is sometimes used for ad-blocking, but the archetypical example is the DNS resolvers your ISP usually provides. Their purpose is to forward your queries to the nameserver responsible for the zone you ask about.

Domain registrars, however, usually do not operate forwarding DNS resolvers. They typically only answer queries for zones that they are responsible for, and give you an NXDOMAIN response for anything else. For this reason, nobody would use a non-forwarding DNS as a resolver for their computer since it would make it near impossible to reach hosts on the Internet (unless you happen to query a record for one of the zones they provide service for).

Not to mention All-Seeing Eye!