A Rust curiosity: pointers to zero-sized types | Hacker News Reader

A Rust curiosity: pointers to zero-sized types

wabbo.org

64 pointsluu47 comments

> In fact, it’s not clear to me why this code insists on transmuting a non-zero value, because doing this seems to work just fine:

Because it needs to compare nonequal to the null reference.

So 'kam' had a reply to this which is now dead, and I don't understand why because it looked informative to me. Could someone explain what was wrong with it?:

"To expand on that: Rust references are guaranteed to be non-null. This allows the compiler to optimize Option<&T> to be the same size as a pointer by encoding None as 0 and Some(p) as the pointer value. When unsafe code violates that guarantee by transmuting 0 to a reference, Some(p) becomes None! Example here: h t t p : / / i s . g d / S I o g w i" (link expanded in case that was the issue).

Nothing's wrong with it, that explanation is accurate, and the link demonstrates it.

Did you have any questions about the explanation, or just whether it was correct?

The question in ajb's mind is probably "why is kam's response dead?"

Sure, although the question asked was "Could someone explain what was wrong with it?", which implies that ajb believed kam's response was killed because it was incorrect.

What am I being down voted for?

Are you unhappy with my response? I thought I answered your question. What was I downvoted for?

You answer is fine, thanks! (I was locked out of HN due to noprocrast)

Link shorteners are frowned upon. There is no need to have a short URL because you have infinite characters and they can lead to bad behavior.

I know reddit filters them, apparently HN does too.

It's been awkward. The reason that is from a shortener is because the program is encoded in the url. The actual URL there is

http://play.rust-lang.org/?code=fn%20main%28%29%20{%0A%20%20...}

HN displays that kinda okay, but if you don't have it looking like HN does, it can get to be almost two thousand characters.

Hmm...

Worst compression algorithm ever:

1) make a URL containing the data to compress (perhaps base-38 encoded or something) 2) Send it to a URL shortening service 3) store the returned link 4) use it every so often to ensure the link doesn't go dead 5) profit!

Not only that, but HN parses the full link incorrectly (the last } isn't included in the link) such that the only way for the link to have worked is to use some indirection.

Like you know, a url shortener.

Well, you can %-escape the '}'...

http://play.rust-lang.org/?code=fn%20main%28%29%20{%0A%20%20...

We've been toying with solving the link shortener problem by replacing urls that redirect with the urls they redirect to.

Would that have been bad in this case?

If this is done in an automated way, it could be an attack vector. EG. a malicious domain or link shortener owner could identify HN server IPs (or via other fingerprints, I'm not really a networks expert), learn when the HN server accesses it, and serve a different URL in only that case to replace the one in the comment.

I know AOL used a similar tactic when Microsoft was racing to keep the (competing) MSN client compatible with their chat protocol after each update[0]. They wouldn't block the MSN client at known Microsoft IP addresses, so the engineers thought their solution worked, while users couldn't access AOL.

Human replacement would probably work fine, and I personally think it would be a good way to deal with behavior that is often an honest mistake made out of ignorance. You might wish to surface the fact the the URL was changed subtly, however, with a color change or diacritical mark.

[0] https://nplusonemag.com/issue-19/essays/chat-wars/

I'd like to mention that we've been recently discussing ways of getting rid of the `::<>` syntax. Then

   mem::size_of::<T>()

would be

   mem::size_of<T>()

which is less visually noisy. This syntax is probably the noisiest thing we haven't cleaned up yet.

+++++1

That still has lots of noise, arguably. Angular brackets somehow feel more noisy to me than the other brackets/braces, they work better as arrows and inequality operators. A single colon would also look better, I think, and the underscore could be dropped...

: doesn't work because that's for type ascription (i.e. specifying what the type of something is).

The angle brackets were chosen because they match the way nearly all C-family languages except D notate generics: C++, Java, C#, Swift, Dart, TypeScript, ActionScript, …

The underscore improves readability here, IMO. We use underscores where they improve readability, similarly to PEP 8.

For a constant like size_of, can you not eliminate the function call and make it a value? Just mem::size_of<T> ?

Not without associated constants, which Rust does not yet support (and likely will not until after 1.0).

size_of isn't really a function but a compiler intrinsic.

Another one I've seen discussed is mem::size_of[T](). Though I can't find any references about it online.

There was an RFC: https://github.com/rust-lang/rfcs/pull/148

The author closed it after finding that there were no non-subjective reasons. Moving to []s causes other ambiguities.

Cool. Somewhat different but similar... In Julia, types with no fields are automatically singletons and an array size-zero immutable values takes no storage:

    julia> sweet_nothings = Array(Nothing,typemax(Int))
    9223372036854775807-element Array{Nothing,1}:
     nothing
     nothing
     ⋮
     nothing
     nothing

    julia> sizeof(sweet_nothings)
    0

This seems like a cute, useless trick but sometimes it's quite useful. For example, the Set{T} is implemented by wrapping a Dict{T,Nothing} and the array of values in the Dict object takes no storage and of course all access is optimized away.

I actually like that C++ enforces a struct to be at least a byte long because it means you can have pointers to them and they will compare as non equal even if you do not know the type. That is actually quite handy sometimes.

I wonder why Rust does not do that.

C++ has a zero-sized type too: it's called "void". But it's not first-class, which is a weird restriction.

Having first-class "void" is very useful. For example, you can make efficient hash sets out of hash maps by simply using () as the value.

How would that be useful? If you need a hash set, use a hash set.

For the same reason any generic type is useful: you can use one implementation for multiple purposes (as a hash map implementation doubles as a perfectly good hash set implementation) without having to duplicate code.

I think that's backwards: a hash set implementation can be used as a hash map simply by storing objects whose comparison is limited to their key fields. A hash map with values that are all the same and never used just seems like a waste.

It's not a waste, because the fact that unit is zero-sized makes it boil away to nothing at runtime.

I can have structs/classes that are zero sized, sometimes this is used for interfaces, or for "namespacing" as alternative to normal namespaces, or for functors, etc.

IIRC, C says that you get a real pointer back from malloc(0). Repeated calls return distinct pointers.

For whatever that's worth...

Actually it is implementation defined.

You may get a NULL pointer or a non-NULL pointer which may not be dereferenced.

This is intentional so that you can always call free on a pointer received from malloc functions, regardless of the allocation result.( passing a NULL to free is ok )

Not exactly; from §7.20.3 in C99:

    If the size of the space requested is zero, the behavior is implementation-
    defined: either a null pointer is returned, or the behavior is as if the size were some
    nonzero value, except that the returned pointer shall not be used to access an object.

Ah, my error.

But, quoting from my ancient copy of The C++ Programming Language, volume 2, §r5.3.3:

"This implies that an operator new() can be called with the argument zero. In this case, a pointer to an object is returned. Repeated such calls return pointers to distinct objects."

So my memory was from C++ new, not C malloc.

"or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object"

I love the C standards. Is that defining the behavior of malloc or the behavior of the code calling malloc?

The caller code. It is saying that the caller cannot dereference the returned pointer, under pain of undefined behavior.

In particular, you can't access (read or write) off either end of an allocated block. Well, if the block is of (nominal) length 0, there is nowhere to access safely.

As someone who is pretty good at allocators, if you're calling `malloc(0)` you're wandering into very undefined territory. I've seen everything from valid allocations of non-zero length, whatever was on the stack, and just 0. The only real constraint is that free accepts the returned pointer.

Because Rust doesn't let you compare references. Rather, comparing references actually compares the referred-to value (which requires that said value implements the comparison operators).

What you can do is convert a reference into a raw C pointer `const T`, and then you can compare those addresses. But the reference type itself doesn't make any guarantees about the address, which means you can't make any assumption about the address of the resulting `const T`. Reference types only make guarantees about the semantics of the reference, and since you can't compare the pointer value of references, the pointer value is not something the reference type guarantees.

It's interesting that this is one of the things that Go and Rust have agreed upon: that the addresses of zero-sized values are the same (in Go we spell a zero-sized type as "struct{}").

It has some unexpected surprises though. If you write `var key struct{}` in a package and then try to use &key as a map key (c.f. http://blog.golang.org/context) then you'll collide with other packages. You've got to use a non-zero type instead.

I don't think we actually defined the addresses of zero-sized values to be equal; I believe it's actually unspecified. I don't think this has ever come up in practice, though; we could define them to be equal if we wanted to.

That issue hasn't come up in Rust, probably because equality and hashing look through references (per the definitions of Hash and Eq in the standard library), so you'd really have to go out of your way to try to use unit as a way to make package-specific keys like that.

> As you might imagine, this operation is HIGHLY DANGEROUS.There’s nothing to stop you from transmuting 0 into a borrow, &T and then trying to dereference it. Voila, null pointer dereferencing in Rust!

Is this right? Why doesn't this function need to be marked unsafe?

transmute is marked as unsafe, so it can only be used in unsafe blocks or functions.

My mistake. I clicked through to the docs, and didn't see any annotation.

http://static.rust-lang.org/doc/master/std/mem/fn.transmute....

Erk—that’s a bug.

Reply on news.ycombinator.com