2,493 karma · joined February 24, 2017
This could be really interesting if it wasn't due to trivial f-up (e.g. difference in inference speed).
[0] Assuming the paper isn't flawed, haven't read it thoroughly yet.
> is not modifying those code to use STL containers much cheaper
That's right. However, I'd add that most exploited bugs these days (in high-profile targets) are temporal memory safety (i.e. lifetime) bugs. The remaining spatial (out of bound) bugs are mostly in long forgotten dependencies.
> Are you referring to the idea of shadow stacks in general or a particular implementation of them?
The idea. Shadow stack (assuming perfect hardware assisted implementation) is a good backward-edge control flow integrity idea, and ruins one of the common ways to take over $rip (write a ROP chain to stack), but that's it. Besides making exploitation harder, both forward-edge and backward-edge CFI also kill some bugs. However, IMO we are long past non-linear stack buffer overflow days, once in a while there may still be news about one, but it could be news because it is an outlier. Hence, compared to CFI, the bugs shadow stack kills are pretty irrelevant now.
> JIT bugs leading to arbitrary script execution in JS engines
Not necessarily JIT bugs. Could also be an UAF and people went a bloody path to convert it to an `ArrayBuffer` with base address = 0 and size = 0x7FFFFFFFFFFFFFFF accessible from JavaScript. Chrome killed this specific primitive. But there's more, I'm not going to talk about them here.
You may have a slight confusion here. In case of browser renderer, people starts with arbitrary JavaScript execution, the goal here is to do what JavaScript (on this page!) can't do, via memory corruption - including, but not limited to executing arbitrary native code. For example, for a few years, being able to access Chrome-specific JS APIs to send arbitrary IPC message to browser process (out of renderer sandbox), is one `bool` flag on .bss away from JavaScript. If we managed to get arbitrary R/W (that is, can read / write all memory within the renderer process, within JavaScript, see my ArrayBuffer example above), we just change it and run our sandbox escape against browser process in JavaScript, who needs that native code execution?
Or, if you do want native code execution. For a few years in V8 the native code WASM gets compiled to, is RWX in memory, so you just use your arb R/W to write that. You can kill that too, but then people starts coming up with bizarre tricks like overwriting your WASM code cache when you load it from disk and before making it R/X, and there're enough fishes in the pool that you likely can't patch em'all.
> I mean, if you are after RCEs, don't CFI and shadow stacks halt the program instead of letting the CPU jumping to the injected code?
Yeah. But as I said, nowadays people usually use temporal memory safety bugs, and they want arbitrary R/W before they attempt to take over $rip. Don't get me wrong, this is because of the success of CFI and similar mitigations! So they did work, they just can't stop people from popping your phones.
> can you name one widespread C++ exploit that:
I just google'd "Chrome in the wild UAF" and casually found this in the first page: https://securelist.com/the-zero-day-exploits-of-operation-wi...
I assume "in the wild exploited" fits your "widespread" requirement.
Granted, it's five years old, but if you are okay with non-ITW bugs I can come up with a lot of more recent ones (in my mind).
This is an UAF. So it would not have happened in a memory safe language. While back then the exploited chrome.exe may not have enabled CFG (it was enabled late 2021 IIRC), I don't see how the exploit path could be blocked by CFI.
> Enable STL bounds checking using appropriate flags
This rarely helps. Most of the nice-to-exploit bugs were in older codes, which weren't using STL containers. Or they are even just write in C. However, if enabling these flags do not hurt you, please still do as it does make non-zero contribution.
> Enable mitigations like CFI and shadow stacks.
Shadow stack is meh. CFI helps a bit more, however there's some caveats depending on which CFI implementation you are talking about, i.e. how strong is it, for example, is it typed or not? But in best case it still just makes the bug chain one bug longer and maybe completely kills some bugs, which isn't enough to make your exploits impossible. It just raises the bar (that's important too though). It also depends on what the specific scenario. For example, for browser renderer without sandbox / site-isolation etc, CFI alone makes almost no impact, as in this case achieving arbitrary R/W is usually easier than taking over $rip, and it's obvious you can do data-only attack to have UXSS, which is a serious enough threat. On the other hand, if it's a server and you are mainly dealing with remote attackers and there's inherently no good leak primitive etc, various mitigations soup could make real difference.
So, all in all, it's hard to tell without your project details.
> How much of the "70% CVE" statistic is relevant to such a C++ codebase?
Uh, I'd guess, half or more of that. But still, it just raises the bar.
If you are talking about DeepSeek's own hosted API service. It's because they deliberately decided to run the service in heavily overloaded conditions and have very aggressive batching policy to extract more out of their (limited) H800s.
Yes, for some reason (the reason I heard is "our boss don't want to run such a business" which sounds absurd but /shrug) they refuse to scale up serving their own models.
When? I vaguely remember that it used to be like that, but I haven't seen nearly as many failures on Aliyun for the last few years.
They didn't.
OP misunderstood what gVisor is, and thought gVisor's uname() return [1] was from the actual kernel. It's not. That's the whole point of gVisor. You don't get to talk to the real kernel.
[1] https://github.com/google/gvisor/blob/c68fb3199281d6f8fe02c7...
This sounds like... a good feature? There are indeed some scenarios where doing so poses a security risk. But most of the time I do want to be able to run arbitrary code on e.g. my WiFi dongle when I'm in control. I know FCC is not a fan of this idea though.
Anything in between suffers.
H100 has 3.3TB/s HBM bandwidth on paper, and ~1000TFLOPS bf16 compute on paper. That's 1:300. 0.6GB vs ~2GFLOPS is 1:3. Tell me how is this compute bound?
(also, your number, even after accounting for GQA, is still off. You usually can't store kvcache in fp8.)
So they are red teamers :p
Building a system for serving read-only data at NVMe SSD speed (as in IOPS) took surprisingly few effort, and is mostly enough for training data. Kudos to DeepSeek who decided to spend extra effort to build a full PFS and share it.
Imagine that you have a sequence of numbers. You want to randomly select a window of, say, 1024 consecutive numbers, a sequence, as input to your model. Now, say, you have n items in this sequence, you want to sample n/c (c is a constant and << 1024) sequences in total. How to do fixed shuffle?
The key is, we have overlap in data we want to read. If we brute force fixed shuffle and expand, we need to save 1024/c times more than original data.
This isn't useful for LLMs, but hey, wonder how it started?
I'll add that latency also doesn't matter that much. You are doing batched data loading for batch n+1 on CPU when GPUs are churning batch n-1 and copying batch n from host memory at the same time.
So as long as your "load next batch" doesn't run for like >1s it would be fine. But one single "load next batch" on one worker means thousands (if not more) random read.
Does it really matter whether it's illegal or not, if there is no enforcement? Pinduoduo (in other name, Temu) has been doing 70 hours week since they started. Yes, they are still doing it right now.
I agree that a lot of "modern" storage stack is way too slow though, tried to find a replication-first object storage for crazy-fast random read in small number of objects last year and found none.