Hacker News Reader

Unless this was updated after your post, I think it was covered in the following line:

> It's certainly possible to build things where bounds checking makes a difference. This just isn't that.

I will add that idiomatic Rust causes a good number of bounds checks to be optimized away. Its impressive the difference this can make. I maintain a parser combinator library that started as a fork. With some simple changes, I've seen massive performance improvements and my best guess is that those changes allowed bounds checks to be removed.

Possible I missed it when I read it. Would need the author to indicate whether or not it was edited.

Yes, idiomatic Rust can cause a number of bounds checks to be elided when using iterators. Typically that's when you have all the data that you need to process up front. When you don't have that ability (e.g. data is being fed in piecemeal and you're copying into a temporary buffer), the bounds checking elision is something you need to do yourself.

can you elaborate? rust iterators seem quite capable of streaming in the fashion you suggest.

If I have a buffer in a library and the external API is “append to buffer”, each append will do a bounds check. I’m not aware of an iterator API would work especially considering that the iterator would hold ownership of the buffer which would be incompatible with the call flow returning to the user after the append.

Can you share what kind of changes, in your case, improved the performance?

Avoid directly indexing if possible. Use the collection functions and slices instead.

I'd add careful use of inlining so the compiler can see both the bounds check and the index to know the bounds check can be removed.

I think the general takeaway is that for most use cases, Rust’s additional safety comes at barely any cost. In some cases the safety gains performance (in not checking thing that the compiler can assert statically), in some cases there’s a hit, but it usually ends up somewhere in the middle.

The takeaway is that Rust’s safety and maintainability generally don’t impose corresponding penalties. And in the cases they do and it’s critical, you can always punt to unsafe.

*nod* Give https://blog.readyset.io/bounds-checks/ a read.

They tried doing a comparison between ReadySet compiled normally and ReadySet with bounds checking removed so thoroughly that they needed to use a patched toolchain to achieve it and found the difference to be within the noise threshold.

Their conclusion was:

> At the end of the day, it seems like at least for this kind of large-scale, complex application, the cost of pervasive runtime bounds checking is negligible. It’s tough to say precisely why this is, but my intuition is that CPU branch prediction is simply good enough in practice that the cost of the extra couple of instructions and a branch effectively ends up being zero - and compilers like LLVM are good enough at local optimizations to optimize most bounds checks away entirely. Not to mention, it’s likely that quite a few (if not the majority) of the bounds checks we removed are actually necessary, in that they’re validating some kind of user input or other edge conditions where we want to panic on an out of bounds access.

As I noted originally, that’s not the reason. It’s because no one is trying to write code that can do hundreds of millions of high level operations per second. There’s so much “inefficiency” in the software stack (or a problem domain where more work is being done in the hot path) that bounds checking is in the noise typically. Readyset’s experiment is flawed because any existence of bounds check on some critical hot path would have already been noticed and fixed as low hanging fruit.

So the remaining bounds check is off the hot path where it doesn’t matter. But in the hot path where you should be bounded by HW limits, it can be a significant slowdown. Prediction can help but it’s not free.

So for most people, they only need to care about bounds checking when they’re doing something in the hot path and even then only when their hot path is running into HW limits. If their hot path is some complicated CPU computation, bounds checking should be but a blip unless you do something stupid like check the bounds too frequently.

So the general advice to not worry too much about bounds checks when writing Rust is directionally correct for the vast majority of people, but recognize it’s incorrect in places and it’s hard to notice because it’s such a small thing hidden in code gen without an easy flag to test the impact.

That's fair.

I still think the "because it's not in the fast path" part of "Most software will not see a bottleneck because of bounds checking because it's not in the fast path" is a bit too much of a blanket statement and could detract from the admonition to benchmark very carefully before optimizing but, otherwise, I agree.

I suspect that information about memory aliasing from the borrow checker (that things are not aliased) allows the compiler to add in many optimizations that C/C++ can't perform, and the benefit from that is similar to the cost of bounds checking.

The memcpy vs. memmove thing comes to mind. So does stuff like iterating over the indices of a pointer to one std::vector while modifying another pointer to a std::vector. If you know you have exclusive access to the vector you are reading, then you know the size isn't changing, so you can just iterate from 0 to size and elide any bounds checking.

With C++, if the pointers to the vectors could alias, then it will have to bounds check at runtime. Templates and Link time optimization helps with this sort of thing, but I'm guessing Rust still has an advantage.

Throughputs on modern hardware are insane. Latency/time-til-first-byte can often still be pretty sucky.

Care to give an example? A disk read is comparatively slow to compute/memory but it’s still in the tens or hundreds of microseconds if I recall correctly and memory is hundreds of nanoseconds. I am convinced latency issues are a remnant of how we build abstractions into software stacks and not a fault of the hardware.

Cache misses will destroy the performance of any CPU-intensive program. You really really want to do mostly sequential reads of RAM if you're processing a lot of data.

Now have 10 or 15 apps hammering away at the same disk at the same time and see how it behaves. Even SSDs slow down as read sizes go down.

The paradox of bounds checking is that it both rarely matters and always matters.