Hacker News Reader

Comment by _vvhw | Hacker News Reader

I worked on a static analysis tool to detect bleeds in outgoing email attachments, looking for non-zero padding in the ZIP file format.

It caught different banking/investment systems written in memory safe languages leaking server RAM. You could sometimes see the whole intranet web page, that the teller or broker used to generate and send the statement, leaking through.

Bleeds terrify me, no matter the language. The thing with bleeds is that they're as simple as a buffer underflow, or forgetting to zero padding. Not even the borrow checker can provide safety against that.

View on HN

I am skeptical until I see the details, and strongly suspect you are dealing with a "safe-ish" language rather than one which has Rust-level guarantees. Uninitialized memory reads are undefined behavior in basically all memory models in the C tradition. In Rust it is not possible to make a reference to a slice containing uninitialized memory without unsafe (and the rules around this have tightened relatively recently, see MaybeUninit).

I say this as someone who is doing a lot of unsafe for graphics programming - I want to be able to pass a buffer to a shader without necessarily having zeroed out all the memory, in the common case I'm only using some of that buffer to store the scene data etc. I have a safe-ish abstraction for this (BufWriter in piet-gpu, for the curious), but it's still possible for unsafe shaders to do bad things.

Hackers exploit any avenue (and usually come in through the basement!), regardless of how skeptical we might be that they won't. They don't need the details, they'll figure it out. You give them a scrap and they'll get the rest. It's a different way of thinking that we're not used to, and don't understand unless we're exposed to it first-hand, e.g. through red-teaming.

For example, another way to think of this is that you have a buffer of initialized memory, containing a view onto some piece of data, from which you serve a subset to the user, but you get the format of the subset wrong, so that parts of the view leak through. That's a bleed.

Depending on the context, the bleed may be enough or it might be less severe, but the slightest semantic gap can be chained and built up into something major. Even if it takes 6 chained hoops to jump through, that's a low bar for a determined attacker.

> For example, another way to think of this is that you have a buffer of initialized memory (no unsafe), containing a view onto some piece of data, from which you serve a subset to the user, but you get the format of the subset wrong, so that parts of the view leak through. That's a bleed.

If there's full initialization then this is just a logic error, no? Apart from some kind of capability typing over ranges of bytes (not very ergonomic), this would be a very difficult subtype of "bleed" to statically describe, much less prevent.

Yes, exactly. That's what I was driving at. It's just a logic error, that leaks sensitive information, by virtue of leaking the wrong information. File formats in particular can make this difficult to get right. For example, the ZIP file format (that I have at least some experience with bleeds in) has at least 9 different places where a bleed might happen, and this can depend on things like: whether files are added incrementally to the archive, the type of string encoding used for file names in the archive etc.

Makes sense! My colleagues work on some research[1] that's intended to be the counterpart to this: identifying which subset of a format parser is actually activated by a corpus of inputs, and automatically generating a subset parser that only accepts those inputs.

I think you mentioned WUFFS before the edit; I find that approach very promising!

[1]: https://www.darpa.mil/program/safe-documents

Thanks! Yes, I did mention WUFFS before the edit, but then figured I could make it a bit more detailed. WUFFS is great.

The SafeDocs program and approach looks incredible. Installing tools like this at border gateways for SMTP servers, or as a front line defense before vulnerable AV engine parsers (as Pure is intended to be used), could make such a massive dent against malware and zero days.

Thanks for the explanation. I would consider that type of logic error more or less impossible to defend at the language level, but I can see how analysis tools can be helpful.

Thanks for the question.

"I would consider that type of logic error more or less impossible to defend at the language level, but I can see how analysis tools can be helpful."

This is how I see it too, as a logic error that exposes sensitive memory in a way that can be as unsafe as a UAF. It's a great leveler of languages, and why I get excited about enabling checked arithmetic by default in safe builds, explicit control flow, and minimizing complexity, abstractions and dependencies—this all helps to reduce the probability of semantic gaps and leaks.

I would imagine that the scenario is simply reuse of some buffer without clearing it, maybe in an attempt to save on allocations. It can happen across so many (even safe) languages. It doesn't matter what guarantees you have around uninitialised memory if you're reusing an initialised buffer yourself.

This error is possible in Rust, but not easy to cause.

Uninitialised memory is considered unsafe. There aren't any loopholes even for "just bytes". You can't accidentally make an uninitialised buffer, and you can't expose one without explicit `unsafe{}`.

Sych unsafe code is typically wrapped in safe interfaces, so the risk is limited to implementation side, and not spread to every usage site.

For filling buffers the language pushes towards use of growable Vec with capacity that tracks how much has been initialised, or to collect straight from an iterator, which is foolproof.

Custom buffers are not even used that often, because things like zip writing or compression use io::Write streaming interface.

It's actually not very hard to cause. It would happen if your system reuses buffers without actually going through the allocator. Something like this:

    let mut buf = vec![0; DEFAULT_BUFFER_SIZE];
    let database_frame_len = database_socket.read(&mut buf)?; // use 1
    let database_result = Database::parse(&buf[..database_frame_len])?;
    let html_template = HtmlTemplate::new(database_result);
    let html_len = html_template.write(&mut buf)?; // use 2
    socket.write(&buf[..html_len]);

The above code makes the tenuous assumption that HtmlTemplate::write actually clobbers everything it's supposed to write. But it doesn't even require unsafe code for it to not do that, because the underlying buffer is entirely initialized according to the type system.

The only advice I can really give to avoid this kind of bug is: don't reuse buffers unless it's actually a bottleneck.

Couldn't you fix that by just clearing the buffer in between the two calls? It won't be measurably slower to do that because the buffer should retain its allocated capacity.

It's not a difficult bug to fix, but it is a very difficult bug to detect, since miri won't think anything is wrong.

This is why I've mentioned io::Write, because people don't chop buffers like that with C-style separately tracked length when they can have zero-copy:

   html_template.pipe_to(socket);

and there's a foolproof io::BufWriter adapter if you need these writes buffered.

You have my attention!

Wow, that's saying something!

The tool is called Pure [1]. It was originally written in JavaScript and open-sourced, then rewritten for Microsoft in C at their request for performance (running sandboxed) after it also detected David Fifield's “A Better Zip Bomb” as a zero day.

I'd love to rewrite it in Zig to benefit from the checked arithmetic, explicit control flow and spatial safety—there are no temporal issues for this domain since it's all run-to-completion single-threaded.

Got to admit I'm a little embarrassed it's still in C!

[1] https://github.com/ronomon/pure

Reply on news.ycombinator.com