Zig's Memcpy, CopyForwards and CopyBackwards | Hacker News Reader

Zig's Memcpy, CopyForwards and CopyBackwards

openmymind.net

34 pointstodsacerdoti8 comments

> “If you try to run the above, you'll get a runtime panic: panic: @memcpy arguments alias.”

That’s an improvement on undefined behavior, but it still feels disappointing that the compiler can’t catch this.

Maybe Rust is spoiling me.

Is it possible for the rust compiler to statically determine if two memory regions will overlap at compile-time, especially with complex pointer arithmetic or when pointers are passed as function arguments? It would be super impressive if so.

Yes and no; the borrow checker requires that mutable references are exclusive, which combined with ownership semantics means that it isn't possible to construct references that overlap if either is mutable. This is only in "safe" Rust though; inside an a`unsafe` block (or an `unsafe` function, which can only be called in an unsafe block) certain invariants are relaxed, with the requirement that they need to hold again at the close of the block to avoid undefined behavior.

In practice, that means that when writing your own code, you can (and in the vast majority of cases should) just stick to safe Rust, and you can be sure that you won't ever make overlapping references if either one is mutable. Safe APIs can be built on top of unsafe APIs though, so you won't necessarily be able to assume your dependencies are doing the same (although there's tooling to help with that, e.g. requiring via the config to generate a build error if any dependency uses unsafe code)

Thank you for that explanation. In the original blog post an example is shared of reusing a buffer, and I was curious how Rust would handle that scenario. Perplexity suggested the following:

// Use a mutable slice to represent the buffer:

let mut buffer = [0u8; 16];

// Read into the buffer:

let n = socket.read(&mut buffer)?;

// To move the partial second row to the beginning, you'd use safe operations like:

let second_row_start = 7; // Index where second row starts

buffer.copy_within(second_row_start.., 0);

// Then read more data into the remaining space:

let remaining_space = &mut buffer[9..];

let additional_bytes = socket.read(remaining_space)?;

Is this idiomatic for Rust?

Your AI hallucinated some APIs, but in theory you could do something like that, sure. The issue is it's not following what the text said, where they read the entire buffer; it's reading a few bytes, then reading a few more bytes, to fill the buffer overall.

For the buffer example, the one after "As a made up example, consider:", I'm not sure I would personally write the code to move the previous stuff to the start of the buffer, but instead process the whole buffer, even if that means a partial parse, and then fill the buffer again, finishing the parse.

But if you wanted to translate the "Let's extract this specific example and try:" into Rust:

    fn main() {
        let mut buf: [u8; 16] = *b"D04GOKUD09OVER90";
        let dest = &mut buf[0..9];
        
        dest.copy_from_slice(&buf[7..17]);
        
        println!("{:?}", buf);
    }

copy_from_slice is a memcpy: https://doc.rust-lang.org/stable/std/primitive.slice.html#me...

This errors at compile time:

    error[E0502]: cannot borrow `buf` as immutable because it is also borrowed as mutable
     --> src/main.rs:5:27
      |
    3 |     let dest = &mut buf[0..9];
      |                     --- mutable borrow occurs here
    4 |     
    5 |     dest.copy_from_slice(&buf[7..17]);
      |          ---------------  ^^^ immutable borrow occurs here
      |          |
      |          mutable borrow later used by call
      |
      = help: use `.split_at_mut(position)` to obtain two mutable non-overlapping sub-slices

We cannot use the help suggestion to make this work, because the slices would be overlapping. But it is a compile time error, not a runtime one. If we did use split_at_mut, that would end up being a runtime panic, because the index comparison happens at runtime.

If I wanted to do what the copyForwards example, sure, that works, but you're in the realm of unsafe: https://doc.rust-lang.org/std/ptr/fn.copy.html

Thank you again, I'm not able to determine if any of this is contrived but it's turned into a great teaching opportunity. I am highly impressed with Rust. The compile error looks extremely expressive of the core issue. I'm also new to zig if you can tell but this has helped me position the two languages and their goals better.

I think Steve might have forgotten (so many helper functions...), but Rust's slices have a `copy_within`[1] function for exactly this scenario:

    fn main() {
        let mut buf: [u8; 16] = *b"D04GOKUD09OVER90";
        
        buf.copy_within(7..16, 0);
        
        println!("{:?}", buf);
    }

It does only support types that are trivially copyable, but it's implemented exactly you'd expect: bounds checking then memcopy handling overlap.

[1] https://doc.rust-lang.org/std/primitive.slice.html#method.co...

I did! Wow! Thanks :)

Reply on news.ycombinator.com