Much Ado About None: Sentinel Values in Python | Hacker News Reader

Much Ado About None: Sentinel Values in Python

florianwilhelm.info

78 pointssebg51 comments

So much trouble to avoid introducing the optional monad.

Python’s type system has union types (including Option), and using them is pretty ergonomic. As TFA points out the problem is that `None` gets contextually reused in all kinds of places where a proper sentinel type is more appropriate.

Pretty sure Python's union types don't nest, though. If you could nest optionals then I feel that reduces the need for separate sentinel types since e.g., you can distinguish between "value not present" (None) and "null value present" (Some(None))

Can you explain or give an example of nested optional types? I don't quite see how that makes sense

The idea is that optional values are either some value or no value, but sometimes the inner "some value" bit can itself be an optional value (in Rust this would be Option<Option<T>>, in C++ it'd be std::optional<std::optional<T>>, in Python it'd hypothetically be Option[Option[T]], etc.). Such a situation gives you three possible states:

- You have an inner value and the inner value is something non-null (an integer, a string, another optional, etc.)

- You have an inner value and the inner value is null

- You do not have an inner value

As a more concrete (albeit somewhat contrived) example, say there's some API you can query for the owner of a particular document ID where the document ID may or may not refer to an actual document (e.g., maybe you got it from a cache and the document has since been deleted). One way to express this would be (using Rust syntax)

    fn get_owner(id: DocumentId) -> Option<Option<UserId>> { ... }

So if you get None, the document does not exist; if you get Some(None), the document exists but it has no owner; and if you get Some(<id>) the document exists and it has an owner.

(This can probably be done better by creating new enums, but it's an example so give me a bit of slack)

A naive port of this signature to Python would be

    def get_owner(id: DocumentId) -> Option[Option[UserId]]: pass

But to type checkers this is equivalent to

    def get_owner(id: DocumentId) -> UserId | None: pass

So now you're in the pickle the OP describes: you can end up with a None, but you no longer knonw whether it means "no document" or "no owner". The Python solution is to basically create a different name for either the outer None (e.g., MISSING as in the OP) or the inner one (e.g., NO_OWNER or the like).

This seems really silly to me. Why does get_owner need to differentiate between the document exists and maybe has an owner or the document doesn’t exist and thus has no owner. In both cases getting no owner should suffice? Why are you using get_owner to try and figure out if the document exists? Can you give an example where this distinction actually matters?

> In both cases getting no owner should suffice?

Depends on what exactly you plan on doing with the information, I'd imagine. I don't think it's necessarily a given that "no document" and "no owner" should be handled identically (e.g., searching for the updated document to try again vs. returning an operation-not-allowed error).

> Why are you using get_owner to try and figure out if the document exists?

Potential TOCTOU issues, perhaps? As I mentioned the document could have vanished out from under you so detecting this situation could let handle things appropriately (though to be fair that argument could probably extend to future operations as well).

Maybe a versioned filesystem would have made a bit more sense, where None would indicate a new version is available? Another example I've seen mentioned is trying to get a key-value pair from a JSON where the key may be missing.

But in any case, I did say it was a bit of a contrived example, so cut me a bit of slack :P It was just the first thing that came to mind.

The examples sibling comments give are probably more practical.

> I don't think it's necessarily a given that "no document" and "no owner" should be handled identically (e.g., searching for the updated document to try again vs. returning an operation-not-allowed error).

If you care whether a document exists or not, you should use an API that exclusively tells you that instead of expecting everything related to documents telling you if the document exists or not. Otherwise you’re overloading the meaning. A non existent document has no owner. An API to fetch the owner of the document should not care if it exists or not, it does not matter.

As I said, that sounds like a recipe for potential TOCTOU errors.

But again, this was an off-the-cuff example that I explicitly described as contrived from the start. I could probably flesh it out to make more sense, but I think it'd be easier just to refer to sibling comments rather than bikeshed this to death.

Not really? It returns no owner if the document is deleted or just does not exist, which is accurate. Why do you need get_owner to tell you if the document exists or not? If anything it should be an error if the document doesn’t exist because you’ve violated a precondition.

> It returns no owner if the document is deleted or just does not exist, which is accurate.

"I don't know" is not the same as "definitely no owner". You may handle the two answers the same for a particular scenario, but then again you may not.

Again, consider the case where get_owner is working on some kind of cache (in which case sure, the function name could stand to be improved). If the document is no longer in the cache it could be because it was actually deleted or because it was simply evicted to make room for other entries. In general you cannot assume that a missing cache entry is equivalent to the document existing and having no owner.

Or similarly (and also yet again), consider the case where get_owner is working on a versioned filesystem, in which case the outer optional could indicate that the (generational) document ID is out of date. In such a case treating "I don't know" and "no owner" the same is incorrect because the owner could have been updated from "no owner" to something else (such a case would likely be more realistically handled by returning a Result<Option<UserId>, SomeErrorKind>, though, given the myriad other ways a file system handle can go stale)

> Why do you need get_owner to tell you if the document exists or not?

Because sometimes there is no good alternative. Asking a cache whether it has an entry and then trying to get the entry value in a separate operation (particularly in a concurrent environment) is a classic setup for TOCTOU errors. You can say something similar for the versioned file system example.

> If anything it should be an error if the document doesn’t exist because you’ve violated a precondition.

I mean, enabling that kind of handling is precisely the point of the outer optional?

Maybe a different example works better for your mental model:

A user can set custom settings, and those settings may or may not have a value assigned to them. For example, one setting is optimization level, another option is a debug flag.

You want to have a unified interface to get information about settings, queried through some intermediate system. (lets just say it is limited to querying specific settings, rather than getting the entire list, due to legacy constraints in this case) Because for some settings you just care about them existing and for others you care about the actual value, you have to differentiate between a setting that contains something, a setting that contains nothing, and one that doesnt exist.

It's analogous to how we use undefined and null in JS.

Undefined = No value known. Think 0 rows found, the 1st row's column is undefined.

Null = Value known. It is void of value. The 1st row is populated, and this column has no value.

You run into this issue any time you do a LEFT JOIN in sql and the field you care about is nullable.

The right side value could be null because it was missing, or because it was literally null. You resolve by checking the join key, and if that is itself nullable, you’re forced to introduce a sentinel

Most languages can do this if it's what you want. In Python it would be:

    class Some[T]:
        value: T

    type Option[T] = Some[T] | None

    def get_owner(id: DocumentId) -> Option[Option[UserId]]: ...

    match get_owner(document_id):
        case None:
            ...
        case Some(None):
            ...
        case Some(Some(x)):
            ...

Yeah, that's more like what you would do if you wanted to translate the entire concept to Python instead of just the signature, and I probably should have mentioned that alternative as well (or should have said "A Python solution"). Can't say I've seen this approach used in practice, interestingly, though I'm probably not nearly as familiar with the Python that exists out there as most other commenters here.

The name clash is unfortunate, but luckily naming things is famously one of the easiest problems in CS so that should be trivial to solve :P

I see, thanks. Though this feels like something that would always have a better alternative than returning Some(None). Probably a new enum, or in Python's case, exceptions (e.g. instead of returning a value if the document is missing, you'd raise DocumentMissingError or something)

Say you select nullable column into vec, Vec<Option<T>>. vec.get returns Option<Option<T>>, which distinguishes null from out of bounds

I'd like nested Nullable in ClickHouse to use CoalescingMergeTree to support postgres unchanged TOAST values easily without getting mixed up on nulls

If you have a map, get(key) returns Option[ValueT]: None if the key is missing, Some(value) otherwise. If the value type itself is, say, Optional[String], your return type is Optional[Optional[String]]. Here None still always means the key is missing, while Some(None) means the value is just None.

The answer is they don't in practical code but they come up in generic code, where you want to wrap something that is already returning an optional in your optional. In languages that do support optional[optional[whatever]], if you every actually write that you know you've messed up your API.

They absolutely do in practical non-generic code. A case I personally encounter regularly is json APIs which distinguish between a key being missing and one being set to null in a structure.

Indeed. Option[Foo] really just means Foo | None - you can’t have an Option[Foo] at runtime. Fundamentally, Python’s static typing system describes a language that is a sort of mediocre match for Python.

Another fun one is List[Foo]. Runtime Python has lists but you can’t ask a list what type of list it is, and you can’t write isinstance(a, List[Foo]).

I’m curious what that would practically look like in Python.

I bet this would be a good use of vibe coding. Experiment with what-ifs. I can imagine being able to “show not tell” and reduce the signal-to-noise of comment opinions.

It would look like a very different language that would be thoroughly incompatible with the existing ecosystem.

Python makes it very easy to do these experiments without changing cpython itself. For example, a lot of the cases for the "not set" Sentinel can be handled by making it MyClass | None. Treat None as "not set". If you need a "no-op" then you can give MyClass a field or something which implies it does nothing. And then override the bool dunder method to make it return false if the no op field is set to true. If you really want, you can have a classmethod which returns a "no op" instance of MyClass so people can easily call your func with MyClass.no_operation() when you want no op, and null when its not actually set.

Sentinels just help you wrap that up nicely and for more general use cases, but theres value in stepping back and thinking about where the unset vs no op case matters.

Or, you know, you can use the exceptions

"A simple cache returns None both for a missing key and for a key that maps to None."

No, it returns KeyError. Or at least it should.

Perhaps the author has had their fair share of run-ins with libraries that didn't think their API through like that. But yes, they should.

Very many people have been taught that exceptions are bad and should be avoided, because google avoids them.

The adverse knee-jerk hate for exceptions far predates Google also saying they're bad, but certainly that didn't help.

Very many people have learnt that exceptions are bad and should be avoided, because they are bad and should be avoided. Some of those people work at Google.

Author here. Especially in Python, I think exceptions are not evil in any way but when it comes to typing they are a pain. Once gave a somewhat provocative talk about this that exceptions are gotos in a way :-) https://florianwilhelm.info/2025/04/evolution_of_goto/

I wouldn't go as far as evil but they are bad for code quality because they obscure the behaviour of functions. You never know what exceptions can be thrown. Can a function even fail? Impossible to tell. (Don't say "documentation" lol)

The standard answer is checked exceptions, which I'm on the fence about, but they don't seem to be very popular in any case. Python doesn't have them. C++ removed support for them which shows you how unpopular they were. Java has them and IIRC they were kind of ok to use in Android. Kotlin doesn't have them though so I guess the Java people didn't really like them either.

it’s maybe less “pythonic” but at $dayjob we’ve very actively moved away from exceptions for things in favor of things that are statically type checked. too many things that become runtime bugs instead of type checking bugs.

with match now being available, we even return explicit error objects and the ergonomics are pretty good (reminds me of rust).

How are these two things related? If you don't use exceptions, you just have to implement them yourself by manually bubbling up the error, all the way up the stack, rather than using the dedicated side channel to do so.

The problem I see is there's no way to enforce the check that you actually handled/bubbled up the error. With exceptions, you just crash if you don't handle it, which is much preferred if determinism is the goal. Is there some way to use the type system to do this? Maybe incoming are UnhandledResult outgoing are some HandledResult or something like that?

If you're going to use the returned value you'd probably return the moral equivalent of Rust's Result/Haskell's Either/etc. types. As long as the returned types don't happen to share the necessary API(s) the type checker should yell at you if you don't branch on the returned value.

I don't think Python has the equivalent of C++'s [[nodiscard]] or Rust's #[must_use], though, so if there are cases where the return value can legitimately be ignored then return values risk missing errors that would show up under exceptions.

Yeah, this is basically what we do. It ends up a bit more ergonomic to not have explicit result types because union types are pretty easy in Python. For example, we have functions like

    # courses.py
    def create(...) -> Course | ErrInvalid:

and they get used like

    match courses.create(...):
        case courses.Course() as course:
            # We use `course` below.
            pass
        case courses.ErrInvalid() as err:
            # Error conditions are usually translated into early returns
            raise fastapi.HTTPException(
                status.HTTP_422_UNPROCESSABLE_ENTITY, err.message
            )
    
    return APICourseCreateResponse(id=course.id, ...)

It has worked (for us at least) pretty well to reduce the number of unhandled errors that make their way into Sentry. The granularity of the error types depends on the need (e.g., sometimes we just have an `ErrInvalid` that returns a message which gets displayed to the user, but sometimes we'll have ErrInvalidAddress or ErrInvalidTaxCode to be more specific for the context).

And yeah, without nodiscard, we try™ to only do this on functions that return values that the caller will definitely need to use. Also combined with exhaustiveness checks on switch statements by default.

The author’s example chose to use the syntax that returns None on missing. If they wanted clarity the standard API does raise.

  cache = {“key”: “value”, “some input”: None}
  cache.get(“missing_key”)

  cache[“missing_key”]

There are better examples that could have been highlighted.

> No, it returns KeyError. Or at least it should.

Surely you meant _raises_? Or did you literally mean to return an exception class?

Yes, raises. My bad.

When I was writing a bunch of Lua, I was a big fan of having a None sentinel value (Lua uses 'nil' for representing absence) and it was useful for a lot of things similar to Pydantic's MISSING.

This distinction is something that I think is actually handy about JavaScript, and distinguishing between 'null' and 'undefined' is sometimes useful for similar reasons.

I feel like faster exceptions would be a better solution for this sort of thing. Leave functions returning the happy path with excepted values. Exceptional things, like missing values being handled at the level where of abstraction that actually cares to catch it:

I went back and forth on this for a long time. Lua's exception mechanism is actually pretty fast but they're just strings, so there's no useful information unless you match on those strings (eek!)

I think encoding these things into a better type signature is the way to go. In Go and Lua, that means multiple return values like `success, result` or `result, err`. It's pretty clunky and verbose in both of those languages though and so my years of Lua made me instantly skeptical of Go doing the same thing.

Languages with cheap data-carrying enums like Rust are my favourite these days. No unhandled exceptions, super obvious where the failure is, and Rust has a lot of niceties for handling, mapping, and adding context to errors without a lot of runtime cost.

I have a library that's approximately half joking that I use for (sentinel and other) special value constants. If you like reading test suites, this one will get ya:

https://github.com/nucypher/constantSorrow/blob/master/tests...

I like it. I can't see how to do better while remaining compatible with Python idioms.

You could even say you never knew what you were… MISSING ;-)

Here is the sample code from the article:

    > MISSING = sentinel("MISSING")

    > def update_email(user_id: int, email: str | None | MISSING = MISSING) -> None:
    >     if email is MISSING:
    >         return  # nothing to update
    >     elif email is None:
    >         db.delete_email(user_id)
    >     else:
    >         db.set_email(user_id, email.lower())  # narrowed to str

I see this style frequently from long time Python programmers. The language is so dynamic that people begin to think this is a good way to program. Hot take: It is not. Why not two different functions called delete_email() and set_email()? That would be more clear.

I think this sort of pattern makes more sense when the fn is update_user and allows the caller to mutate multiple fields at once.

I just return a customized Exception type. Works about the same.

I hate sentinel values. Even/especially with AI-generated code.

1. If your date does not have a value, do not return 0, an empty string, or None. Use a discriminated union and model your domain rigorously.

2. If your data model accepts None/undefined as a value, that should translate to your business model as well. For example

3. Do not make your `name` field as `string | undefined` just because there might be an intermediate loading or processing state. It's better to model that as something like `Loaded<string>` where Loaded is something like a Result or Optional type (though ideally more rigorously defined)

Here is what I've done for years in TypeScript, though now in a nice package thanks for AI: https://github.com/shepherdjerred/monorepo/blob/main/package...

Ah Python. The leaking sieve that keeps on giving problems to solve.

Reply on news.ycombinator.com