HNHacker News
TopNewBestAskShowJobs

aw1621107

4,617 karma · joined April 24, 2017

submissionscomments
aw1621107··on Much Ado About None: Sentinel Values in Python
> In both cases getting no owner should suffice?

Depends on what exactly you plan on doing with the information, I'd imagine. I don't think it's necessarily a given that "no document" and "no owner" should be handled identically (e.g., searching for the updated document to try again vs. returning an operation-not-allowed error).

> Why are you using get_owner to try and figure out if the document exists?

Potential TOCTOU issues, perhaps? As I mentioned the document could have vanished out from under you so detecting this situation could let handle things appropriately (though to be fair that argument could probably extend to future operations as well).

Maybe a versioned filesystem would have made a bit more sense, where None would indicate a new version is available? Another example I've seen mentioned is trying to get a key-value pair from a JSON where the key may be missing.

But in any case, I did say it was a bit of a contrived example, so cut me a bit of slack :P It was just the first thing that came to mind.

The examples sibling comments give are probably more practical.

aw1621107··on Much Ado About None: Sentinel Values in Python
The idea is that optional values are either some value or no value, but sometimes the inner "some value" bit can itself be an optional value (in Rust this would be Option<Option<T>>, in C++ it'd be std::optional<std::optional<T>>, in Python it'd hypothetically be Option[Option[T]], etc.). Such a situation gives you three possible states:

- You have an inner value and the inner value is something non-null (an integer, a string, another optional, etc.)

- You have an inner value and the inner value is null

- You do not have an inner value

The problem, from what I understand, is that Option[T] in Python is equivalent to the union type T | None, and union types are flattened (e.g., (T | U) | V is equivalent to T | U | V) and duplicates are ignored (e.g., T | U | T is equivalent to T | U). As a result, Option[Option[T]] is equivalent to (T | None) | None, which is equivalent to T | None | None, which is finally equivalent to T | None, which means you lose the distinction between "No inner value" and "Inner value is present but is None".

As a more concrete (albeit somewhat contrived) example, say there's some API you can query for the owner of a particular document ID where the document ID may or may not refer to an actual document (e.g., maybe you got it from a cache and the document has since been deleted). One way to express this would be (using Rust syntax)

    fn get_owner(id: DocumentId) -> Option<Option<UserId>> { ... }
So if you get None, the document does not exist; if you get Some(None), the document exists but it has no owner; and if you get Some(<id>) the document exists and it has an owner.

(This can probably be done better by creating new enums, but it's an example so give me a bit of slack)

A naive port of this signature to Python would be

    def get_owner(id: DocumentId) -> Option[Option[UserId]]: pass
But to type checkers this is equivalent to

    def get_owner(id: DocumentId) -> UserId | None: pass
So now you're in the pickle the OP describes: you can end up with a None, but you no longer knonw whether it means "no document" or "no owner". The Python solution is to basically create a different name for either the outer None (e.g., MISSING as in the OP) or the inner one (e.g., NO_OWNER or the like).
aw1621107··on Much Ado About None: Sentinel Values in Python
Pretty sure Python's union types don't nest, though. If you could nest optionals then I feel that reduces the need for separate sentinel types since e.g., you can distinguish between "value not present" (None) and "null value present" (Some(None))
aw1621107··on Switch Lowering in GCC
> restrict is also still broken there.

That's a bit surprising to me given how heavily Rust uses the underlying mechanisms. Do you mean specifically how restrict is used in Clang or is there still something in LLVM that is broken?

aw1621107··on The result of the C++26 DIS ballot
> and is no longer undefined behaviour to act on them

This is only partially true. From the proposal [0]:

> Moreover, a variable that has a well-defined, erroneous initial object representation under this proposal can still lead to undefined behaviour later on:

    void f(T&);
    void g(bool);

    void h() {
      T* p;    // uninitialized, has erroneous value (e.g. null)
      bool b;  // uninitialized, erroneous value may not be valid bool

      f(*p);   // dereferencing has undefined behaviour
      g(b);    // undefined behaviour if b is invalid
    }
[0]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p27...
aw1621107··on Bbtree: Fastest Ordered Map
Sure. I didn't mean to say that the author used CBMC, only that it's an example of model checking.

Incidentally, the docs don't seem to say what model checker was used. I don't think any of verus, shuffle, or ffs are technically model checkers (or at the very least they don't describe themselves as such)

aw1621107··on Bbtree: Fastest Ordered Map
My guess is https://en.wikipedia.org/wiki/Model_checking . C Bounded Model Checker [0] is a relatively well-known instance in the niche.

[0]: https://www.cprover.org/cbmc/

aw1621107··on Composite objects and C++26 the std:polymorphic type
It is similar in that the valueless state isn't absolutely necessary, but avoiding the valueless state in general is not free and it seems the committee was unwilling to force implementations to incur that cost.

The docs for Boost.Variant state that avoiding a valueless state requires some kind of additional storage beyond that strictly required for the intended value. In the current implementation this means that assignment may allocate (modulo optimizations described further down the page) [0], which is obviously unacceptable for some use cases.

The proposal for std::indirect and std::polymorphic describes similar costs for avoiding a valueless state, though for moves instead of assignment [1]:

> Without a valueless state, moving indirect or polymorphic would require allocation and moving from the owned object. This would be expensive and would require the owned object to be moveable. The existence of a valueless state allows move to be implemented cheaply without requiring the owned object to be moveable.

Presumably the committee made a similar decision with respect to a valueless state for std::variant as they did for std::polymorphic/indirect.

[0]: https://www.boost.org/doc/libs/latest/doc/html/variant/desig...

[1]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p30...

aw1621107··on C# now beats Rust on WASM size
IIRC no_std is a per-crate attribute and does not actually forbid use of std, so I think there's a decent chance you can use random Rust crates without changing them.

That being said, I do think it would have been nice if the author had chosen a more interesting benchmark, but alas...

aw1621107··on C# now beats Rust on WASM size
I think part of the complaint has less to do with language toolchains/extensions and more that the post processing performed on the C# output via Binaryen is not unique to the toolchain. It's analogous to comparing the toolchain outputs while zipping one output but not the other.

I think it's also a bit unclear exactly what you mean when you list nonstandard C extensions alongside Rust's no_std and then say "similar approaches". Rust's no_std is part of the language proper so the commonality surely can't be the non-standard-ness of what is done to shrink binary sizes.

aw1621107··on Composite objects and C++26 the std:polymorphic type
Ah, you're right, that's a bad example. I had assumed that that kind of precondition would be explicitly mentioned so I didn't read the rest of the text as closely as I should have.

Perhaps move-assignment would have been a better example? That one at least mentions construction using an rvalue so I think that specific operation would work for move-only types.

Though now that I think about a bit more I suppose it's a bit of a moot point since the constructors all look like they require a copy-constructible T

aw1621107··on Composite objects and C++26 the std:polymorphic type
Looks like it depends on the specific operation(s) you want to perform [0]. For instance, constructing a std::polymorphic<T> from an object of type T or derived from T requires that that object is copy-constructible (among other requirements), but copy-assigning one std::polymorphic<T> to another only requires that T is complete.

[0]: https://eel.is/c++draft/polymorphic

aw1621107··on C++26 Contracts: Introduction to Preconditions and Postconditions
Possibly? Though given its outputs apparently aren't completely out there I'd hazard a guess that the inferred context usually shouldn't be too far off of what can be deduced using more traditional methods.
aw1621107··on C++26 Contracts: Introduction to Preconditions and Postconditions
The paper explicitly says that if you use an LLM it is prompted function-by-function.

For the SAT solver I think classification can be done function-by-function if you manually drive the solver in the single-function mode, but I think you're more right in that the whole-program mode doesn't actually do that.

I think my misunderstanding stemmed from interpreting the start of section 5.4 and figure 3 as showing that constraint generation occurred strictly before the verifier was invoked and glossing over the bit at the start of section 6 that said that the process basically works the other way around when using the solver to build the p-model.

aw1621107··on C++26 Contracts: Introduction to Preconditions and Postconditions
> They globally derive the borrowing information for all pointers in the whole program by classifying the role of each pointer.

Right; by my reading this is done function-by-function by the constraint generator before passing that information to the solver for verification.

aw1621107··on C++26 Contracts: Introduction to Preconditions and Postconditions
At least for this iteration of contracts, no. From the proposal (italics in original) [0]

> Features Not Proposed

> To keep the scope of this proposal minimal (while still viable), the following features are intentionally excluded from this proposal; we expect these features to be proposed as extensions at a later time, building on the foundations laid by this proposal.

> The ability to assume that an unchecked contract predicate would evaluate to true and to allow the compiler to optimize based on that assumption, i.e., the assume semantic

aw1621107··on C++26 Contracts: Introduction to Preconditions and Postconditions
> That "test bed" description fits Rust way better.

This assertion seems rather debatable given Rust's general lack of trailblazing in the language design space.

> Rust is in many ways also significantly more complex than C++, believe it or not.

I'm a bit skeptical that "many" is the right qualifier there. "Some", perhaps, but it isn't obvious to me how you'd reach "many".

aw1621107··on C++26 Contracts: Introduction to Preconditions and Postconditions
Thanks for the link! That's an interesting approach indeed. If I'm reading the paper [0] correctly the concepts involved look pretty similar to what Rust uses. I think they turn the interprocedural analysis into an intraprocedural one as well by effectively deriving borrowing information from function implementations and attaching that to signatures for use by callers. It's a compatibility hazard, which is why Rust chose to not do the same, but in principle it makes it easier to derive more precise borrowing information.

[0]: https://dl.acm.org/doi/epdf/10.1145/3814943.3816182

aw1621107··on C++26 Contracts: Introduction to Preconditions and Postconditions
> lifetime analysis can be done fully automatic for most C code as long as it can be done independently from other program state

Is this an interprocedural or intraprocedural analysis?

> I have seen quite promising work on this.

Is this work public? I'm interested in seeing more!

aw1621107··on C++26 Contracts: Introduction to Preconditions and Postconditions
> The problem is that this still runs into complexity problems and does not scale to non-toy programs.

Right; that is why Rust's borrow checker was designed to be intraprocedural only. That obviously is a tradeoff of its own: now functions need to be annotated to convey information across function boundaries, and the annotations picked will influence what can and cannot be expressed.

> IMHO a middle ground with some annotations is realistic and would be far more convenient than a borrow checker when supported by good tooling.

I'd be interested in seeing what such a middle ground looks like, particularly in respect to what tradeoffs it makes!

aw1621107··on C++26 Contracts: Introduction to Preconditions and Postconditions
IIRC Rust's borrow checker analysis is more or less a subset of what SMT solving is capable of. It's "just" a question of tradeoffs made with respect to performance, complexity, precision, etc.
aw1621107··on Your C is not standard C
> That's an assumption that you're treating as though it somehow informs reality.

At least to me, section 4 seems to imply that the "assumption" is a bit more than just that. For example:

> We know of at least four optimizing compilers (three of them production compilers) that performed this transformation at some point during their lifetime; sometimes at least partially reversing the decision when the implications on multi-threaded code became known.

i.e., there are real-life instances of the lack of compiler/language support leading to miscompilations. Yes, that example is talking about threads, but I think a similar argument should apply to atomics.

> I gave you an example of atomics being handled by a library

The thing is that nothing in that man page indicated to me that that library wouldn't be subject to problems similar to that described in Boehm's paper; e.g., for all I know that library is the Plan 9 moral equivalent of pthreads and is subject to similar problems pthreads is when a non-thread-aware optimizing compiler is used.

> Kencc is a very "dumb" compiler that does little magic

This could (partially?) address things though. If Plan 9 always uses that compiler and if there's no interest in implementing the optimizations Boehm describes then that would obviate 2 of the 3 correctness issues Boehm describes (though that would also imply that Plan 9 uses its own dialect of C with its own set(s) of guarantees, both explicit and implied). The only thing I'm uncertain about is to what extent hardware reordering would be relevant; I thought I remember reading about that making things interesting, but I don't remember off the top of my head to what extent.

aw1621107··on Your C is not standard C
I don't think that man page on its own is sufficient to draw that particular conclusion. You need compiler/language support to ensure atomics are handled properly, so you'd need to know how Plan 9's compiler/language specifies atomics.
aw1621107··on Your C is not standard C
> You can do it all with library functions that just take regular int* (and so on).

It's not just about atomics, but you might still find Hans Boehm's 2005 paper "Threads Cannot Be Implemented As a Library" [0] interesting. It describes (among other things) correctness and performance issues that can arise if the compiler and/or language don't know about thread/atomic semantics.

[0]: https://web.stanford.edu/class/archive/cs/cs240/cs240.1236/o...

aw1621107··on 11 year old unsoundness bugs found in Rust borrow checker
> but given these bugs, Java blows Rust away in memory safety.

It's not like Java is immune to memory safety-related implementation bugs either (e.g., [0]):

> A resource area allocated object is probably passed/used outside of the scope of the corresponding ResourceMark (similar to JDK-8325672).

So "blows Rust away in memory safety" doesn't seem quite supported enough?

[0]: https://bugs.openjdk.org/browse/JDK-8336095

aw1621107··on 11 year old unsoundness bugs found in Rust borrow checker
> Given the type declarations it's "obvious" it should not compile as it directly contradicts the types 'static lifetime. You don't deliberately write code that doesn't compile.

You don't need 'static to trigger the bug [0]

[0]: https://github.com/rust-lang/rust/issues/164250#issuecomment...

aw1621107··on 11 year old unsoundness bugs found in Rust borrow checker
> however the ability to leak memory is pretty demonstrable, no?

This bug wouldn't be nearly as interesting if it "just" resulted in a memory leak. Memory leaks aren't considered unsafe in Rust because they can't result in UB in the same way as e.g., OOB writes. In fact, there are APIs accessible in safe Rust which explicitly leak memory! [0]

This bug is effectively a use-after-free, which is definitely something that the Rust compiler should catch.

> Would be cool to see clippy pick up on this bug in older versions!

No need for that. A future version of the Rust compiler that incorporates a fix should reject the incorrect code, so all you'd need to do is compile older code with that new compiler.

[0]: https://doc.rust-lang.org/std/mem/fn.forget.html

aw1621107··on 11 year old unsoundness bugs found in Rust borrow checker
> In fact, C is even safer, because it actually does have a specification that lists all the invariants that the programmer has to uphold themselves because the compiler doesn't check them.

The difference is that the C spec intentionally frees the compiler from upholding those invariants. In this case the Rust compiler is failing to catch something it is required to catch.

A more accurate analogy would be a C compiler incorrectly failing to reject an assignment through a pointer to const without a cast. The spec requires that such a thing be rejected, so it's a compiler bug.

aw1621107··on C for Rust programmers
Yeah, it's one of those things that is not exactly intuitive if you didn't already know about it. The menagerie of other byte-ish-sized types added in recent versions of C and C++ probably doesn't help either.

Bit of a (not so?) fun fact: char* being a universal alias can lead to some potentially unexpected slowdowns [0], especially if the char* bit is behind a typedef.

[0]: https://travisdowns.github.io/blog/2019/08/26/vector-inc.htm...

aw1621107··on C for Rust programmers
> The linked-to essay points out that char is only "at least 8 bits", and links to https://cppreference.com/c/language/arithmetic_types which confirms that point.

Yes, but in C standardese a "byte" is not necessarily the 8 bits that it's normally thought to be these days. From C89 Section 2.2.4.2 Numerical Limits [-1]:

> maximum number of bits for smallest object that is not a bit-field (byte) CHAR_BIT 8

i.e., CHAR_BIT is the number of bits in a byte. C23 has a similar definition, and further defines CHAR_WIDTH that is defined to expand to the same value as CHAR_BIT.

> So I don't believe your statement "sizeof(char) is guaranteed to always be 1" is correct.

From C89 section 3.3.3.4 The sizeof operator [0]:

> When applied to an operand that has type char, unsigned char, or signed char, (or a qualified version thereof) the result is 1.

This wording remains basically identical through C23 [1].

> Where does the spec say that a char* can be used to address any point in an object?

From C89 section 3.3 Expressions:

> An object shall have its stored value accessed only by an lvalue that has one of the following types:

> <snip>

> * a character type.

This also remains the case up through C23.

I think you're thinking of the strict aliasing rule with your example, but character types are one of the exceptions to said rule so I think your example is actually fully defined. It'd be UB if you casted to an incompatible type like a float, I believe.

[-1]: https://port70.net/%7Ensz/c/c89/c89-draft.html#2.2.4.2

[0]: https://port70.net/%7Ensz/c/c89/c89-draft.html#3.3.3.4

[1]: https://port70.net/%7Ensz/c/c23/n3220.html#6.5.4.4

[2]: https://port70.net/%7Ensz/c/c89/c89-draft.html#3.3

[3]: https://port70.net/%7Ensz/c/c23/n3220.html#6.5.1

← PreviousPage 2 of 34Next →