Hacker News Reader

Comment by cryptonector | Hacker News Reader

> Safety measures make memory-unsafe languages slow > > Mobile devices have limited resources and we’re always trying to make better use of them to provide users with a better experience (for example, by optimizing performance, improving battery life, and reducing lag). Using memory unsafe code often means that we have to make tradeoffs between security and performance, such as adding additional sandboxing, sanitizers, runtime mitigations, and hardware protections. Unfortunately, these all negatively impact code size, memory, and performance.

Even more evidence that the negative performance impact of bounds checking is minimal, nay, it can even be positive.

View on HN

No it isn't. It's just evidence that it's a trade-off you might want to make in order to achieve some other goal, specifically security.

But if "security" isn't remotely a concern for a given project (like almost anything graphics / gaming related), this is not at all evidence for changing anything. It could be that Rust's optimizer eliminates the bounds checking so regularly as to be a moot point, but this isn't saying anything of the sort. It's saying that the cost, whatever it was, was judged to be worth paying for the improved security for these projects

I take it the opposite: C programmers out of abundance of caution of putting in bounds checks for code that will never be called with out of bound data. As such rust is eliminating code that is being manually written. If you don't write the bounds check in C, and rust for the equivalent determines that the bounds check isn't needed the code should be the same (to the assembly level). However if you write a bounds check in C the optimizer might not eliminate it.

Can you point to any such bounds check in C that an optimizer cannot eliminate but it can eliminate the equivalent one in Rust?

I'm sure it's possible to construct such a thing, but I cannot imagine it ever being common enough to show up on any sort of head to head comparison.

there's almost no bounds checking in rust code before the optimizer even looks at it because we use iterators and not goofy manually indexed for loops that are begging you to make a typo that crashes your code :)

Yeah but idiomatic modern C++ is also using iterators and even before that there's no bounds checking to eliminate in the first place since operator[] is unchecked so the optimizer can't be struggling to eliminate it since it's not there.

The question isn't "does Rust have bad bounds checking optimizations" but rather "what is this mythical heavily-bounds-checked C code that the compiler can't optimize away?"

No the claim is always that Rust "must" be slower than C/C++ because it has pervasive bounds checking for array indexing.

Then people insist on wanting to replace every x[i] in prod with x.get_unchecked(i) only to learn that, not only was that indexing not slowing the code down (the branch is perfectly predictable in a correct program!), but actually any difference is so in the noise that the random perturbation is worse (or that the asserts were actually adding extra facts for more profitable optimizations in llvm).

There is definitely specific hot loops with weird access patterns where it can be high impact but those are the exception, not the rule, as the Android team demonstrated.

Trivially, anything using the Iterator trait.

I don't know that I've ever actually manually indexed an array over years of using Rust.

In the HPC / offline graphics / simulation world, there's lots of things like sparse arrays / grids, where you index into compacted grids and iterators wouldn't be that practical in that scenario (i.e. one single item, although for things like filtering with surrounding cells they can still be useful sometimes), and bounds checks do make a bit of a difference there (it's definitely measureable above the noise threshold), and due to the random nature of the data, the branch predictors don't help avoid the overhead.

Sure, I know there's paradigms where manual indexing is important. GP asked for an example where bounds-checking could be eliminated in Rust, so I gave the first one I thought of.

I don't think your answer is relevant to the question:

> Can you point to any such bounds check in C

I would guess all the aliasing stuff will get you. In C it's very difficult for the compiler to know whether two pointers are aliased, if we change X maybe Y changes too (because actually X and Y were the same). In Rust if we can write to it then it isn't aliased, and if we can't write to it then nobody can change it, thus changing X definitely can't change Y and the emitted machine code is sometimes simpler as a result, doing what you naively expected rather than what the C needs to do just in case you're crazy and there is an alias.

Now, in modern C you can say you don't have aliasing, but you're probably wrong and so there's a high risk when you do that you now get "impossible" bugs because you swore to the optimiser that if X changes, Y is unaffected, then created a situation where that wasn't true and now your program has no defined meaning, which is going to be tricky to debug. So, on the whole C programmers do not use this, indeed in places like the Linux kernel they even turn off the C standard's very minimal aliasing rules (which forbid aliasing objects of different types), they just don't trust themselves.

But the compiler doesn't need to care in that case because it's not bounds checking those pointers in the first place in C. So that's not going to give you slow C code from bounds checking that the optimizer failed to eliminate.

Like yeah there's aliasing changes, but in "idiomatic" C/C++ how is that getting you bounds checking that's not being optimized away fairly consistently?

Wait, previously you were talking about bounds checks which can't be optimised out, now you seem to be saying in C you wouldn't bother writing any bounds checks, which is a quite different claim.

I think what they're saying is that C compilers don't care about aliasing here because it's not actually bounds checking the pointers, it's just a numeric comparison between two random arguments. It's much easier for an optimizing compiler to eliminate a duplicated boolean check between two numbers this way because passing numbers from one function to the next has no aliasing concerns.

This is a misunderstanding of what's useful about aliasing information. A lot of C code can't be autovectorized because it can't tell that accesses to two arrays don't alias, for example. Similarly it often can't reorder / eliminate redundant updates to arrays because it can't tell they're not aliased. These aren't related to bounds checking specifically but they are things that can improve performance in Rust over C (theoretically, anyway; in practice LLVM is still very much tuned for C so it doesn't take advantage of a lot of this stuff yet, but it likely will in the future).

> But if "security" isn't remotely a concern for a given project (like almost anything graphics / gaming related)

Gaming platforms have gotten a lot less lenient over time, and with pretty much every game these days having online components, "security isn't remotely a concern" has become a lot less true.

Sure, but then there's things like HPC / offline graphics / simulation (VFX/CG), where performance is the end-all concern (or memory efficiency sometimes at the expense of CPU time), and security isn't a concern at all there, with lots of things like random index lookups into sparse arrays / grids, etc. I know for a fact that bound checks do make a bit of a difference there, as the data's random, so the branch predictors are close to useless in that situation...

Everybody who works with Maya, Flash (now Adobe Animate) etc. knows that they crash all the time, and often corrupt files so you back them up every hour or so. Carmack insists, in a gaming context, to run heavyweight, high-false-positive-rate static analyzers because users don't like crashes.

When C++ dies (which in 20 years it will, and I wasn't that hopeful 20 years ago), people will look in the same bewilderment at excuses made for its insane behavior as they look now at mid-20th-century arguments against high-level languages (and assemblers before them - like Mel said, "you never know where it will put things so you'd have to use separate constants.")

> which in 20 years it will, and I wasn't that hopeful 20 years ago

That's too optimistic. C++ would easily die in 20 years if it didn't already have 30+ years of still-active legacy that can't easily be converted or rewritten.

I've recently even had to start new projects in C++ because platforms I depend on demand it or because I have to interface with existing code and libraries that still only exist as C++. I'm not a fan of the language by any means, but I'll eat my shoe if it's "dead" in 20 years for anything except maybe greenfield development.

Dead - no, dying COBOL-style - quite possibly.

How much C and C++ do we have now? How much COBOL did we have at it's peak? I'm not sure the analogy holds for that reason alone.

What if the better analogy is updating building codes in Manhattan?

At least in VFX, at the high-end Maya's only really used for modelling/UVing/layout now, other apps have taken over the rendering/lighting side of things...

But anyway, in my experience a lot of the crashes are often due to quickly hacked together plugins for the various DCCs written for artists, that don't have good error checking or testing, and it's not completely clear to me how that situation's going to improve that much with something like Rust, if the same programmer time constraints are going to exist in writing them: i.e. I think it's very likely people will just unwrap() their way to getting things to compile instead of correctly handling errors, so it will be the same situation from the artists' perspective: technically it may be a panic rather than a segfault, but from the artists' perspective, it will likely be identical and take the DCC down.

panics can be caught and the presence of those unwraps come with enough data that filing a bug report upstream is helpful enough to fix the bug.

Sure, but that (we do it) happens currently with C/C++ and signal handler traps which gather the callstack and collate them: the issue isn't usually that we don't know the crashes aren't happening or having the call stacks - the issue is hacky code that was written for one purpose is now being used for other things it wasn't designed for (because it is useful to artists, despite its limitations), and there isn't the time to go back and write it properly for the new expanded use-case. That's my point: a new safer language isn't going to improve much in this area without more development time provided to write better code, given the time constraints are going to be the same as they are currently.

It does change the situation if the plugin throws an exception on errors instead of causing a segfault that brings down the parent application.

I kinda hate .unwrap() even exists, it leads to excessively shitty error messages with no good context

Security is absolutely a concern in these areas (except maybe offline graphics).

In my experiences with university HPC clusters, security is very important because you have a lot of young students with no Unix experience accessing the resources. We've had real compromises of individual research machines because of this.

This happens all the time at research universities, but it's not always public. In one public example from my uni, hackers from China compromised a research machine, which was used to attack IT infrastructure, which lead to PII including SSNs being compromised.

That's security of the generic infrastructure the code's running under though is it not? It's not security of say CUDA kernel code being executed on a GPU?

I'm talking about the actual HPC algorithm code heavily priortising performance (or in some cases memory efficiency), at the expense of pretty much everything else (other than correctness, obviously).

If your high performance code running on a sensitive cluster is vulnerable, then it opens up the rest of the system to exploitation also. How is it a problem of the infrastructure around the code, and not the code itself?

Ah, I think I understand what you mean. Let me rephrase:

Students writing code are not prioritizing security or performance. (I've seen FEM analysis written in Matlab, large neural-networks written in nearly-pure Python, etc.) The real 'performance' priority is human time, at the cost of everything else. To this extent, extra security "for free" from memory safety is nice.

There are exceptions, of course. The 2012 AlexNet breakthrough was a result of performance-engineering, for example. But generally speaking, publish-or-perish rewards neither optimizing performance nor optimizing security.

So, students will be installing Docker images (which have super user privileges), sudo running bash scripts, sudo installing pip or npm packages. I've seen students replace libraries (including CUDA) with modded binary blobs from researchers from other universities. All to save time in pursuit of ~~interesting~~ publishable results.

These are horrible things I've seen during my time in academia. We (should) do virtualization, jails, firewalls, etc. to insulate the rest of us from these horrible things. (I'd add "keep machines offline", but that's rare, and even rarer because of the pandemic.) This insulation is imperfect, and many of those imperfections are due to memory safety flaws.

"random index lookups into sparse arrays" is almost always an anti-pattern in HPC. Successful data structures are designed for streaming access and fine-grained parallelism, even when the problem domain seems irregular. Bounds checks sometimes matter (less in the logic than in inhibiting vectorization), but can sometimes be safely eliminated using existential lifetimes/branding or different control flow.

Rust is starting to make inroads in HPC/scientific computing. The libraries have a ways to go for widespread end-to-end adoption, but to give a concrete example, a current project has drastically beaten OpenBLAS across a suite of matrix factorizations. It was developed over a few months by one person with much less arch-specific or unsafe code. (The library is on GitHub/crates.io, but the author isn't ready for a public announcement so I won't link it yet.) Expect to see lots more Rust in HPC over the next few years.

If you're talking about the library I think you are, from what I could see there weren't any tests of the numerics (nor comparisons of the output from competing libraries), which is quite concerning?

I do think Rust will make inroads, but more because of better WASM toolchain,so loading data into the browser is significantly easier than with JS (e.g. https://crates.io/crates/moc).

That’s why default bound safety with optional unsafe access is a thing. Remove bound safe access after measuring its performance impact. But one should start with the safe thing, as for most part of even HPC, it doesn’t really matter (not everything will be the hot loop).

I work in HPC, and while security isn't an issue for your typical simulation code, correctness certainly is. Spending a million CPU hours on a supercomputer computing junk because memory unsafely caused the simulation to corrupt itself, and then writing a paper publishing those results isn't good.

Many times when I've helped some researcher make their code run on a cluster I have discovered that the code crashes at runtime if bounds checking is enabled. The usual response is that "this can't be a problem because we've (or someone else) published papers with results computed with this program". Sorry sunshine, this isn't how it works. Maybe the corruption is entirely benign, but how can you tell?

It's not just about bounds checks. I am way more agressive about borrowing fields of other types, particularly mutable borrows in ways that I wouldn't attempt in C to protect myself from future code from breaking invariants I'd have to rely on. This means I can write the hyper optimized version of an algorithm on any language, but I'm more likely to even attempt it in Rust.

An other bit I first saw observed by Armin Ronacher and which is indeed quite common is that Rust does not require you to be defensive (and pay the price) around protected resources e.g. you can hand out references to rc’d types or mutex’d, and you know it’s safe, if a bit constraining.

And so you save the overhead of the extra refcounting or the re-entrant locks (though that would be unsound anyway), and you can safely use anything which works off of references.

There were (or are?) some older call of duty games being sold on steam that had unpatched RCE vulnerabilities in them. Simply joining a server ran by a malicious host can result in the players system being totally compromised.

These games would go on sale once a year or whatever and attract new players, and people would post warnings in the steam forums and whatnot to try to stop people from being effected by the issue, but I am sure some people either didn't listen or didn't notice the warnings.

I haven't looked into the issue in a few years at this point, but it's very possible that the games are still unpatched and being listed in the store to this day.

Anything that connects to the internet needs to be strongly concerned about security!

That’s why managed languages in itself are not enough. Security on desktop OSs are a joke, with perhaps mac being a bit ahead than the rest.

But especially desktop linux.. every random bash script could encrypt your documents, or leak out your browser cache, do whatever it wants..

Similarly there was an RCE discovered in Dark Souls III about a year ago. And even though modders had s fix for it before the details of the exploit were even publicly revealed, it took the devs 8 months or so to finally fix it.

I wonder how much of the positive performance is just because we have not proved that some impossible case is really impossible and so we have if statements checking for a situation that mathematically cannot happen just out of caution. (note that in many cases we don't even have the theoretical tools to prove the code)

Though that makes non-memory safe code more reliable in the case of a bit flip. (this is not a serious advantage - only some bit flips can be prevented this way)

Reply on news.ycombinator.com