Arnavion

The speaker was mistaken / misspoke.

The closest thing to "conditionally returned to the kernel" is if the page had been given to madvise(MADV_FREE), but that would still not have the behavior they're talking about. Reading and writing would still produce the same content, either the original page content because the kernel hasn't released the page yet, or zero because the kernel has already released the page. Even if the order of operations is read -> kernel frees -> write, then that still doesn't match their story, because the read will produce the original page content, not zero.

That said, the code they're talking about is different from yours in that their code is specifically doing an out-of-bounds read. (They said "If you happen to allocate a string that's 128 bytes, and malloc happens to return an address to you that's 128 bytes away from the end of the page, you'll write the 128 bytes and the null terminator will be the first byte on the next page. So they're very clearly talking about the \0 being outside the allocation.)

So it is absolutely possible to have this setup: the string's allocation happens to be followed by a different allocation that is currently 0 -> the `data[size()] != '\0'` check is performed and succeeds -> `data` is returned to the caller -> whoever owns that following allocation writes a non-zero value to the first byte -> whoever called `c_str()` will now run off the end of the 128B string. This doesn't have anything to do with pages; it can happen within the bounds of a single page. It is also such an obvious out-of-bounds bug that it boggles my mind that it passed any sort of code review and required some sort of graybeard to point out.

That's what (b) is about.

A read from an unmapped page producing a different value than reading from that same page after it's mapped is an OS bug (*). If this was an already allocated page that had something written to it, reading from it would page it back in and then produce the actual content. If this was a new page and the OS contract was to provide zeroed pages, both the read before it was mapped and the read after it was mapped would produce zero.

What could happen is that the UB in that code could result in it being compiled in a way that makes the comparison non-deterministic.

(*): ... or alternatively, we're not talking about regular userspace program but a higher privilege layer that is doing direct unpaged access, but I assume that's not the case since you're talking about malloc.

A non-idiot would use calloc(DEFINITELY_BIG_ENOUGH), and that will likely erase the difference because the impl will be able to rely on mmap(ANONYMOUS) creating zero pages for such a large allocation. A more realistic test would be to have a large number of small allocations that get calloc'd and then free'd repeatedly, because a) free'ing a small allocation will not free the underlying page and thus reallocation will not be able to rely on it already being zero, and b) zeroing small allocations doesn't amortize the cost of zeroing as well as zeroing large allocations does.

It does, though even for a small segment you end up needing a lot of boilerplate - a dependency on the cc crate, a build.rs that invokes the cc crate, extern declarations to tell the Rust caller about the C function, potentially some bindgen if your small segment is not so small ... and you still end up having to do some amount of thunking between & / * / MaybeUninit because of that anyway. So if there is a "pure Rust" way to do it with `unsafe`, writing that is often easier. The pure Rust impl also has the advantage that you can validate it with Miri, unlike the C impl case because Miri cannot emulate arbitrary C code.

As I wrote in https://news.ycombinator.com/item?id=44048391 , you have to get used to copying the libstd impl when working with MaybeUninit. For my code I put a "TODO(rustup)" comment on such copies, to remind myself to revisit them every time I update the Rust version in toolchain.toml

Also not every type has a valid "all zeroes" value in the first place.

There is a limited form of such "expression rewriting" using tagged template strings introduced in ES2015. But it wouldn't be particularly useful for the ORM case.

No. The correct way to write that code is to use .spare_capacity_mut() to get a &mut [MaybeUninit<T>], then write your Ts into that using .write_copy_of_slice(), then .set_len(). And that will not be any slower (though obviously more complicated) than the original incorrect code.

>I’ve also dealt with pain implementing similar interfaces in Rust, and it really feels like you end up jumping through a ton of hoops (and in some of my cases, hurting performance) all to satisfy the abstract machine, at no benefit to programmer or application.

I've implemented what TFA calls the "double cursor" design for buffers at $dayjob, ie an underlying (ref-counted) [MaybeUninit<u8>] with two indices to track the filled, initialized and unfilled regions, plus API to split the buffer into two non-overlapping handles, etc. It certainly required wrangling with UnsafeCell in non-trivial ways to make miri happy, but it doesn't have any less performance than the equivalent C code that just dealt with uint8_t* would've had.

This stopped being much of a problem when MaybeUninit was stabilized. Now you can stick to using &MaybeUninit<T> / &mut MaybeUninit<T> instead of needing to juggle *T / *mut T and carefully track converting that to &T / &mut T only when it's known to be initialized, and you can't accidentally use a MaybeUninit<T> where you meant to use a T because the types are different.

It's not as painless as it could be though, because many of the MaybeUninit<T> -> T conversion fns are unstable. Eg the code in TFA needs `&mut [MaybeUninit<T>] -> &mut [T]` but `[T]::assume_init_mut()` is unstable. But reimplementing them is just a matter of copying the libstd impl, that in turn is usually just a straightforward reinterpret-cast one-liner.

>So fixing approx. 5-10% of CVEs (by zero-initializing all stack vars) is a worse cure than letting these uninitialized stack vars be possible sources of exploits?

Read the comment you responded to again, carefully. It's not presenting the dichotomy you think it is.

Or let-else with a panic!() in the else {} if you want to use a format string.

The standard is for software to report its own EOL / EOS status. The website you linked is the opposite direction - it's aggregating that status for a certain set of software.

Some distros packaging Rust software (OpenSUSE at least) also transparently set up CARGO=cargo-audit to get embedded SBOMs.

It doesn't force someone who already wasn't checking their dependencies for CVEs / maintained-ness to start doing that. It does make someone who *was* doing that be able to show they're doing that in some standard way.

In other words it doesn't force you to add an SBOM + EOX checker step to your CI pipeline. But if your compliance auditor wants you to check your dependencies, adding such a standardized step makes it easier to satisfy the auditor.

Its new owner since 2010 is still part of the Asus group, but sure it's technically a different company from Asus proper.

Any mobo will let you download the firmware file to a FAT32-formatted USB drive etc, and then use that to update the UEFI within the UEFI UI.

Yes some mobos have the feature in their UEFI to connect to the internet and download the update, but it's best to not rely on that since you have no idea how securely that is implemented. Considering how the submitted article is about a shitty implementation in a regular Windows program, you can be sure the implementation in UEFI is even shittier (may not check certs, may not even use HTTPS, etc). Asrock used to have an "Internet Flash" feature in their UEFI and then suddenly removed it, probably because it was too insecure to fix.

And in fact is not even something that Rust does differently from C++. Memory releases in Rust are handled by dtors just like they are in C++. What a weird article.

(The only difference is that Rust defaults to moving while C++ defaults to copying, and Rust moves don't leave a moved-out object behind while C++ does, so the dtors in Rust are simpler and called fewer times than the equivalent C++ code.)

Yes, if you want to go that route, you'll be better off going with RISC-V.

Asrock (sub-brand of Asus but seemingly independent in the product and dev side) has been fine for me over the ~10 years I've bought their mobos. There was the thing a few months ago with X870 mobos that were apparently frying CPUs, but I think that was not sufficiently proven to be their fault?

That said, in their X670 / B650 they have the same setting as what this article is about, and it could be equally as broken on the software side as Asus's is, but I wouldn't know because I don't use Windows so I disabled it.

Yes that's correct, for exactly the reason that it is more likely to keep compiling and possibly not do what you intended if the original value's type changes due to refactoring. However there are still a few conversions that don't have alternatives to `as` - truncating conversions (eg i64 -> i32 that intentionally discards the upper half), int <-> float conversions (eg i64 -> f64, both truncating and checked conversions), unsized pointer casts (eg *const [MaybeUninit<u8>] -> *const [u8], `.cast()` only works for Sized target), and probably a few more.

Sorry, the code is closed source.

The allocator we wrote for $dayjob is essentially a buffer pool with a configurable number of "tiers" of buffers. "Static tiers" have N pre-allocated buffers of S bytes each, where N and S are provided by configuration for each tier. The "dynamic" tier malloc's on demand and can provide up to S bytes; it tracks how many bytes it has currently allocated.

Requests are matched against the smallest tier that can satisfy them (static tiers before dynamic). If no tier can satisfy it (static tiers are too small or empty, dynamic tier's "remaining" count is too low), then that's an allocation failure and handled by the caller accordingly. Eg if the request was for the initial buffer for accepting a client connection, the client is disconnected.

When a buffer is returned to the allocator it's matched up to the tier it came from - if it came from a static tier it's placed back in that tier's list, if it came from the dynamic tier it's free()d and the tier's used counter is decremented.

Buffers have a simple API similar to the bytes crate - "owned buffers" allow &mut access, "shared buffers" provide only & access and cloning them just increments a refcount, owned buffers can be split into smaller owned buffers or frozen into shared buffers, etc.

The allocator also has an API to query its usage as an aggregate percentage, which can be used to do things like proactively perform backpressure on new connections (reject them and let them retry later or connect to a different server) when the pool is above a threshold while continuing to service existing connections without a threshold.

The allocator can also be configured to allocate using `mmap(tempfile)` instead of malloc, because some parts of the server store small, infrequently-used data, so they can take the hit of storing their data "on disk", ie paged out of RAM, to leave RAM available for everything else. (We can't rely on the presence of a swapfile so there's no guarantee that regular memory will be able to be paged out.)

As for crates.io, there is no option. We need local allocators because different parts of the server use different instances of the above allocator with different tier configs. Stable Rust only supports replacing GlobalAlloc; everything to do with local allocators is unstable, and we don't intend to switch to nightly just for this. Also FWIW our allocator has both a sync and async API for allocation (some of the allocator instances are expected to run at capacity most of the time, so async allocation with a timeout provides some slack and backpressure as opposed to rejecting requests synchronously and causing churn), so it won't completely line up with std::alloc::Allocator even if/when that does get stabilized. (But the async allocation is used in a localized part of the server so we might consider having both an Allocator impl and the async direct API.)

And so because we need local allocators, we had to write our own replacements of Vec, Queue, Box, Arc, etc because the API for using custom A with them is also unstable.

Various kind of "desktop" applications like databases and video games use custom non-global allocators - per-thread, per arena, etc - because they have specific memory allocation and usage patterns that a generic allocator does not handle as well as targeted ones can.

My current $dayjob involves a "server" application that needs to run in a strict memory limit. We had to write our own allocator and collections because the default ones' insistence on using GlobalAlloc infallibly doesn't work for us.

Thinking that only "embedded" cares about custom allocators is just naive.

To be clear, the only difference between Rust and C here is whether the conversion happens by default or not. Rust doesn't do the conversion by default but will let you do it if you want to, with `as`.

There are also more type-safe conversion methods that perform a more focused conversion. Eg a widening conversion from i8 -> i16 can be done with .into(), a narrowing conversion from i16 -> i8 can be done with .try_into() (which returns a Result and forces you to handle the overflow case), a signed to unsigned reinterpretation like i64 -> u64 can be done with .cast_unsigned(), and so on. Unlike `as` these have the advantage that they stop compiling if the original value changes type; eg if you refactor something and the i8 in the first example becomes an i32, the i32 -> i16 conversion is no longer a widening conversion so the `.into()` will fail to compile.

Explicit allocators do work with Rust, as evidenced by them already working for libstd's types, as I said. The mistake was to not have them from day one which has caused most code to assume GlobalAlloc.

As long as the type is generic on the allocator, the lifetimes of the allocator don't appear in the type. So eg if your allocator is using a stack array in main then your allocator happens to be backed by `&'a [MaybeUninit<u8>]`, but things like Vec<T, A> instantiated with A = YourAllocator<'a> don't need to be concerned with 'a themselves.

Eg: https://play.rust-lang.org/?version=nightly&mode=debug&editi... do_something_with doesn't need to have any lifetimes from the allocator.

If by Zig-style allocators you specifically mean type-erased allocators, as a way to not have to parameterize everything on A:Allocator, then yes the equivalent in Rust would be a &'a dyn Allocator that has an infectious 'a lifetime parameter instead. Given the choice between an infectious type parameter and infectious lifetime parameter I'd take the former.

Result<Option> is the correct way to represent this, and if you need further convincing, libstd uses it for the same reason: https://doc.rust-lang.org/stable/std/primitive.slice.html?se...

>many functions allocate memory and what are they supposed to do when there is no memory left?

Return an AllocationError. Rust unfortunately picked the wrong default here for the sake of convenience, along with the default of assuming a global allocator. It's now trying to add in explicit allocators and allocation failure handling (A:Allocator type param) at the cost of splitting the ecosystem (all third-party code, including parts of libstd itself like std::io::Read::read_to_end, only work with A=GlobalAlloc).

Zig for example does it right by having explicit allocators from the start, plus good support for having the allocator outside the type (ArrayList vs ArrayListUnmanaged) so that multiple values within a composite type can all use the same allocator.

>Also many functions use addition and what is one supposed to do in case of overflow?

Return an error ( https://doc.rust-lang.org/stable/std/primitive.i64.html#meth... ) or a signal that overflow occurred ( https://doc.rust-lang.org/stable/std/primitive.i64.html#meth... ). Or use wrapping addition ( https://doc.rust-lang.org/stable/std/primitive.i64.html#meth... ) if that was intended.

Note that for the checked case, it is possible to have a newtype wrapper that impls std::ops::Add etc, so that you can continue using the compact `+` etc instead of the cumbersome `.checked_add(...)` etc. For the wrapping case libstd already has such a newtype: std::num::Wrapping.

Also, there is a clippy lint for disallowing `+` etc ( https://rust-lang.github.io/rust-clippy/master/index.html#ar... ), though I assume only the most masochistic people enable it. I actually tried to enable it once for some parsing code where I wanted to enforce checked arithmetic, but it pointlessly triggered on my Checked wrapper (as described in the previous paragraph) so I ended up disabling it.

The reason to introduce the Quantity wrapper is to not be able to swap the quantity and price arguments.