T* makes for a poor optional<T&>
brevzin.github.io
brevzin.github.io
The last `<T&>` there is key, but seems to have been removed from the HN version of the title.
This blog post is not arguing for the use of optionals in general (it assumes everyone already agrees with that). It is instead making a much more wonky argument that std::optional<T> should support T being a reference type, which apparently is not supported right now.
For most purposes, it seems std::optional<T&> would behave very similarly T*, but the ergonomics would be better in certain template scenarios. It seems like a valid argument to me, but definitely getting into the weeds a bit.
Aside: IMO std::optional in general is disappointing because it doesn't actually solve the worst problem with nullable pointers: dereferencing a null std::optional is still undefined behavior. I wish the standard had gone with a design that forces the developer to write an explicit check when unwrapping the optional...
If this is how std::optional worked, hardly anyone would use it.
OK... Well, this is how kj::Maybe works and having written several hundred thousand lines of code using it, I think it works well.
https://github.com/capnproto/capnproto/blob/master/kjdoc/tou...
The actual implementation of Maybe doesn't seem to force anything, it just returns a pointer to the stored value. There's no enforcement of any kind that anything is checked as a result. There's some hoop-jumping with friend classes to "hide" the readMaybe() function to make the non-macro usage as ugly as possible to strongly encourage using the macro-convention, but that's hardly "enforcement".
It's a cute syntax & macro pattern, but it's hardly a good fit for a C++ standard, either.
> There's no enforcement of any kind that anything is checked as a result. There's some hoop-jumping with friend classes to "hide" the readMaybe() function to make the non-macro usage as ugly as possible to strongly encourage using the macro-convention, but that's hardly "enforcement".
Sure fine. Anyone who wants to reach into the internal APIs could also just as easily do `#define private public` and bypass every protection C++ has. We're not trying to defend against malicious developers here.
See also value_or and the C++20 monadic methods on std::optional.
http://open-std.org/JTC1/SC22/WG21/docs/papers/2021/p0798r6....
You can use `.value()` to deref an optional instead of `operator*` or `operator->`. It will throw an exception on empties.
IMHO the primary purpose for a function returning a `std::optional` instead of either returning a value on success or throwing an exception on failure is that you can use it in codebases where exceptions are disabled for whatever reason. If the behaviors of `.value()` vs `operator*` were swapped, such that `operator*` throws exceptions and `.value()` is undefined behavior, all the people who wanted it for the purposes of eschewing exceptions, the crowd who wanted `std::optional` the most, would be getting the verbose code.
Which is bad at every level if resolution: it’s more verbose to do “the right thing” (obviously), it’s more expensive to do “the right thing” (because if you’ve properly checked you’re paying the check twice and praying the compiler can remove the redundant check), and it’s incompatible with other pointers to do “the right thing” (because pointers use ! And *).
So .value() it mostly worthless, if it’s being used it’s an assertion with overhead which can take down your program.
> all the people who wanted it for the purposes of eschewing exceptions, the crowd who wanted `std::optional` the most, would be getting the verbose code.
And nothing of value would be lost (except for it not behaving as a “standard pointer” anymore).
Eh that's hardly a solid complaint here. Since these are header-defined templates, it's all inlined. And any compiler will absolutely remove those redundant checks as a result. And an unchecked .value() would still be available if you really did object to that "overhead", so it doesn't violate any C++ principals here.
I do kinda think that operator-> & operator* as a result did the assertion, and value() bypassed it. But this would also be the opposite behavior of other std:: classes, like std::vector (operator[] being unchecked vs. at() being checked).
It can in pretty much every single case even on -O1.
I do not think there is something bad with it. It does the same for the same syntax and you have a `.value()` checked access anyway.
Checked exceptions were a nice idea in theory that definitely didn't work well at all in practice. IMO exceptions should be reserved for "panic" situations, as a friendlier alternative to crashing. (But I also feel they should be used liberally in such situations...)
* This is technically wrong, because, e.g., destructors don't run on the way out; I'm going for pithiness here rather than strict accuracy.
> The last `<T&>` there is key, but seems to have been removed from the HN version of the title.
HN seems to like stripping out HTML-tag-like blocks instead of escaping them. Why though?
I'm simplifying a tad, look up 'null pointer specialization rust' for more. What's cool is the zero-cost abstraction over type layout specification, while allowing the generic Option<T> type to work exactly as expected.
[1]: Assuming T is a non-nullable pointer type.
You can even do the same thing for a few special primitive integer types, such as std::num::NonZeroU64.
https://doc.rust-lang.org/std/num/struct.NonZeroU64.html :
> Option<NonZeroU64> is the same size as u64
Behind the scenes, the type is defined with a magic compiler-internal incantation that isn't usable in user code, unfortunately:
#[repr(transparent)]
#[rustc_layout_scalar_valid_range_start(1)]
#[rustc_nonnull_optimization_guaranteed]
I would also like being able to tell the compiler about other niches. If a Foozle can be Ding, Ping, Zing or Number(n) where I promise n is from 0 to 199, that all fits in a byte if the compiler is willing to work with me by storing Ding, Ping and Zing as magic sentinels in the niche range 200 to 255 and I'm willing to trade the small runtime cost for the memory usage win.
Yeah, that would be cool. I think it would make sense to represent this as a repr(u8) enum with a range-limited Number variant. Something like:
#[repr(u8)]
enum Foozle {
Ding,
Ping,
Zing,
Number(u8 in 200..=255), // or whatever range syntax
}In practice, people use a zero value to indicate that a value is missing, but this only works if the zero value isn't also a valid input.
Hopefully the coming generics will resolve this pain point.
This is obviously not possible, since you can imagine lots of complex objects are all integers or themselves composed of integers, and for this entire class of objects, the zero value usually makes sense.
For example, a 3-vector (in the physics sense) is a 3-tuple of integers `type Vec3 struct {x, y, z int}` , `Vec3 {0, 0, 0}` being the origin. How does a function specify that it can take an optional 3-vector? Even worse, you can imagine a struct for a potenitally 0-volume cube represented as 8 Vec3s, one for each vertex, whose 0 value is again a valid object.
Of course, you could gratuitously add an `IsValid bool` flag which must be true to the class, but the cube example shows how this quickly becomes annoying and bloated.
1. Take a *Vec3, using nil as the "optional is missing" value.
2. Take a Vec3, but consider some value as "invalid", often the 0-value as possible.
1 invokes exactly the issues in Hoare's essay. 2 doesn't work for all types, as some types don't have any bit pattern that are not useful, this is what I was trying to point out.
Additionally, a *Vec3 is more than `Vec3` or `null`. For example, the following code will behave very differently based on whether optionalVec is Vec3 or *Vec3.
optionalVec := Vec3{0, 0, 0}
foo(optionalVec)
fmt.Printf("theVec: %+v", optionalVec)
// would print {0, 0, 0} with an Optional[Vec3] type
// will actually print {2, 0, 0} with *Vec3
...
func foo(optionalVec *Vec3) {
if optionalVec != nil {
optionalVec.x += 2 //we use a different origin
}
[...]
}For my team, we would have loved to have something like C# (or even Java), with generics, exceptions, lambdas, LINQ/Streams etc., but with Go's low overhead and large ecosystem. Go the language was a major negative point, just not bad enough to outweigh the advantages of the runtime.
``` var map1 map[int]int = nil var map2 map[int]int = &map1 var map3 map[int]int = nil ```
Interface types also can be nil even if they're not explicitly pointers because heap pointers are needed under the hood due to the size of the value not being knowable at compile time. You can pretty quickly get into some hairy situations trying to ensure that a value isn't nil when combining pointers or maps/slices with interfaces.
I really wish that separating the concerns of present versus not present from value versus reference was more mainstream. I understand why this was traditionally the case for lower-level languages, but even higher-level ones like Java are mostly based on the paradigm that only references can be "not present". This irks me in a similar way to how earlier versions of Java wouldn't allow default implementations of interface methods and instead requiring abstract classes, which classes could not extend from more than one of.
This is one of those things which sounds great in practice, and survives approximately 17 seconds of working on a real codebase.
Go supports returning multiple results. Maybe I don't read enough Go code to understand what's actually popular, but AFAIU the idiomatic way to accomplish this in Go is something like:
if v, exists := lookup(key); exists {
do(v)
}
if r, err != foo(); err != nil {
do(r)
}
I'm not sure Generics can help here. What you really want is compiler enforcement of comprehensive condition checks. For example, via pattern matching switch statements. That's easiest to do with optional types as the unwrapping operation creates a simple point for static constraint checking, but in principle the same thing could be accomplished with annotations on multi-value return types that describe the association.
How would you write a function that takes a string, but can use some default if none is provided?
Common way:
func f(s string) {
if s == "" {
s = "default"
}
...
} func f(s *string) string {
ret := ""
if s == nil {
ret = "default"
} else {
ret = *s
}
return ret
}
func main() {
s := "foo"
fmt.Println("Hello, " + f(&s))
fmt.Println("Hello, " + f(nil))
}There won't be overloading * for getting the value inside and overloading bool conversion to check if the value is there. Still, it's an improvement.
> Whenever the idea of an optional reference comes up, inevitably somebody will bring up the point that we don’t need to support optional<T&> because we already have in the language a perfectly good optional reference: T*.
But i have never heard this said even once. Rather, inevitably someone will bring up the point that you can do it with reference_wrapper. Check it out:
https://en.cppreference.com/w/cpp/utility/functional/referen...
Also if you're going to optionally store a reference why not just use a unique_ptr or a shared_ptr so ownership is much more obvious and safer. (I realize it might not be zero cost, but optional<T&> just seems like an invitation to hard to find bugs)
If X isn't Y, and doesn't even try to sort of be a Y when you need it to in useful circumstances, then X is a poor optional Y for that excellent reason.
A tail is a poor optional leg, for starters, and so on.
What you need is some_kinda_smart_pointer<T> instead of T*, which can be T* when you need it to, but also quacks like T most of the time; like when passed as an argument to a function needing a T, it will convert to that by yielding the T value and so forth.
The name "some_kinda_smart_pointer" can be "optional", and QED.
A T* isn't a class type; it's a basic type: a pointer. You can't do too many clever things with basic types in C++. They can't have any member functions, so they cannot substitute anywhere where something is required that has member functions.
TLDR: naked pointers inherited from C have disadvantages compared to smart pointers in all sorts of generic programming and whatnot; ask any C++98 programmer.
The problem is that you can't have optional<T&>. So if you want to pass an optional reference to something, you're currently forced to pass T* instead (using null as the proxy for "not passed", of course). And thus, T* makes for a poor optional<T&>.
Didn't say there is (or, for that matter, that there isn't).
The author does say there is an issue with it: he would like optional<T&> but cannot have it. (That, of course, is not optional's fault, though).
> nor is the article complaining that T* is a bad optional<T>.
But it is poor (not just in comparison to a fictitious optional<T &>).
> So if you want to pass an optional reference to something, you're currently forced to pass T* instead (using null as the proxy for "not passed", of course). And thus, T* makes for a poor optional<T&>.
If you want an optional reference to something, maybe you don't entirely understand references. A reference isn't optional; it has to refer to something.
References in C++ are odd; they have limitations. For instance, you can't declare an array of them, either.
To simulate an optional reference, we use a pointer, and we can wrap that in a class:
class optional_int_ref {
const int *maybe_int; // null if we aren't referencing an int, otherwise valid pointer
};
std::optional is a container: it contains a value, which may be indicated as not being present. A reference isn't a value which is contained; it's a value which is somewhere else. If you contain a reference, the reference must be resolved to a valid object.
Basically, you don't want optional<T &>, but a parallel optional_ref<T>: something that optionally references a value, and if so, that value is somewhere else.
So, today, we could make optional_ref<T>. If we did so, then T * would be a poor optional_ref<T> for the same reasons that it's a poor optional_ref<T &> that we don't have. Therefore, T * is not the best optional we can have.
This exact consideration is discussed in the article and arguments are brought forth against it. I would love to give you benefit of the doubt, but you clearly didn't read the article.
Thanks for pointing that out; see the "§ What about optional_ref<T>?" just before the conclusion (same name and all).
I probably missed it because it has no code box; if I get tired of reading something long, I start to gravitate toward code rather than walls of text.
optional_ref being "spelled differently" is a given. It has different semantics from optional, and doesn't replace it (as is noted) and so of course it deserves a different name. An optional reference to something (which itself isn't optional) isn't the same as an optional something (which either exists or doesn't). This is a fluff objection.
Regarding the objection against duplicating code in the implementation for things like map:
1. So what? C++ is full of duplication.
2. All the same, perhaps the implementor could find a way to share some implementation bits between optional and optional_ref. I haven't tried a cut at it, but it doesn't seem self-evident to me that the only way to implement optional_ref is complete copy and paste from optional.
The author has a valid points about there being issues with having to hard code either optional or optional_ref in places in the user code.
Maybe a combind super_optional<bool ref, T> which goes to optional<T> if ref is false otherwise to optional_ref<T> could help.
That is similar in spirit to the workaround, except the choice of "ref or not" isn't extracted from a T& versus T argument; it's an explicit template parameter.
In the end the author's conclusion is indeed that "optional_ref<T> is a much better optional reference than T*, but it’s still a poor substitute", where by substitute I think he means "alternative for the nonexistent optional<T&>".
#include <optional>
#include <functional>
#include <type_traits>
using std::optional;
using std::conditional_t;
using std::reference_wrapper;
using std::cout;
using std::endl;
template <typename T>
class optref {
T *ptr;
public:
optref<T>() : ptr(nullptr) { }
optref<T>(T &obj) : ptr(&obj) { }
optref<T> &operator =(const T &rhs)
{
if (!ptr)
throw 42;
*ptr = rhs;
return *this;
}
};
template <bool ref, typename T>
using opt = conditional_t<ref, optref<T>, optional<T>>;
int main()
{
int x = 42;
opt<true, int> rox = x;
rox = 3;
cout << x << endl;
}
The output is
3
We can build abstractions on opt<ref, T> but they have to carry that pesky parameter. If we can move it to the right and default it to false, that helps if most of our optionals aren't references.
It's not obvious how to get rid of it.
I'm not sure how you wrote this but then didn't immediately realize why optional<T&> would be useful.
> std::optional is a container: it contains a value, which may be indicated as not being present. A reference isn't a value which is contained; it's a value which is somewhere else. If you contain a reference, the reference must be resolved to a valid object.
And yet std::reference_wrapper exists and basically ruins your entire thesis. There's no conceptual problem with storing references in containers; `struct Foo { int& i; }` is exactly that and has worked since forever. Rather it's a problem with the implementation of optional<> specifically that prevents it. And why reference_wrapper exists, to "fix" those implementation issues - primarily with types that already existed like vector, which is why it's silly it's still needed on new types like optional.
C++ references cannot be optional by design; that is one of the aspects of pointers that C++ references "fix", and you cannot code around it. A key idea behind references is to get rid of the null-based optionality of pointers (nowadays referrred to as the "billion dollar mistake").
The author of the article would like to use the declaration syntax of references T& in the middle of some templateology to produce an optional reference; but that optional reference itself cannot have anything to do with C++ references.
If the abstraction doesn't actually have C++ reference semantics in it, why would you want to specify any aspect of it using C++ reference syntax? Because it is poetically memetic or something?
reference_wrapper doesn't actually have anything to do with references; it doesn't have to be used with references and isn't implemented literally as a wrapper around a reference: implementations contain a pointer. It really should lose the _wrapper from its name, which provides no information about it.
Where is the reference here?
#include <iostream>
#include <utility>
#include <functional>
using std::reference_wrapper;
using std::cout, std::endl;
using std::swap;
int main()
{
int x = 42, y = 73;
reference_wrapper rx(x);
reference_wrapper ry(y);
swap(rx, ry); // swap pointers
cout << rx << ", " << ry << endl; // 73, 42
cout << x << ", " << y << endl; // 42, 73: stayed where they are
}
You keep repeating this as if it's at all relevant. The single purpose of std::optional is to make things optional that cannot themselves be optional by design. That C++ references cannot be optional are exactly why it's a perfect fit for std::optional.
You seem to be talking about the difficulty in storing T& if the value isn't present. That's not actually hard to do, that's what std::reference_wrapper does already. The C++ standard library already solved that, and trivially so.
Rather the issue is a difference in assignment behavior, as the Boost.Optional discusses: https://www.boost.org/doc/libs/1_69_0/libs/optional/doc/html... (and Boost.Optional does allow T&, because optional<T&> is well-formed & well-defined)
> reference_wrapper doesn't actually have anything to do with references
What? Literally from the documentation:
"std::reference_wrapper is a class template that wraps a reference in a copyable, assignable object. It is frequently used as a mechanism to store references inside standard containers (like std::vector) which cannot normally hold references."
The only thing it does is take a T& and return a T&. That it stores it internally as a T* is an implementation detail.
As for your example snippet, you just have a bug in usage since you didn't read reference_wrappers docs. It needs to be `swap(rx.get(), ry.get());` since the _wrapper part of its name is actually significant, you have to get the wrapped value out of it. Alternatively `swap((int&) rx, (int&) ry);` would also have worked.
It's judging a book by it's cover.
Where does the article suggest using optional<T&> in all contexts?