Maybe Not [video] | Hacker News Reader

Maybe Not [video]

youtube.com

160 pointskgwxd42 comments

I really like this idea of composing selectable attributes as schemas (ala RDF). I've been fairly reluctant in adopting specs into my own code, mostly because in a lot of cases I just didn't see the benefit (not for the things I typically do, not saying I wasn't interested in generative testing and the likes ... those still seem very promising but also hard to get right). Especially, when dealing with "distributed" things (like the world wide web, where everything is maybe and only some things useful). And, RDF got this surprisingly right. I was sort-of half expecting him to also introduce PROLOG into it, but maybe next time. The idea of collections of subject-predicate-object statements forming an isomorphism with graphs, and sets of these statements form concrete "aggregates" plays extremely well with the idea of unification through search.

Somewhat tangentially, it also keeps reminding me somehow of Alfred North Whitehead's concepts of reality as process flows https://en.wikipedia.org/wiki/Alfred_North_Whitehead#Whitehe...

"In summary, Whitehead rejects the idea of separate and unchanging bits of matter as the most basic building blocks of reality, in favor of the idea of reality as interrelated events in process. He conceives of reality as composed of processes of dynamic "becoming" rather than static "being", emphasizing that all physical things change and evolve, and that changeless "essences" such as matter are mere abstractions from the interrelated events that are the final real things that make up the world."

Here's Rich Hickey giving a talk in 2009 guided by Whitehead quotes [1]

[1] https://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hi...

Hickey's `Maybe` example feels misguided (about breaking existing callers). If Maybe is an input just keep the original function around and provide a new one that wraps it.

    originalF :: x -> y

    f :: Maybe x -> y
    f None = newDefaultBehavior
    f (Some x) = originalF x

If `Maybe` is an output then existing callers SHOULD be broken, because they must now handle a new possibility of failure they didn't before. The fact that this doesn't happen in Clojure is actually a source of pain for me.

It's also rather interesting that Hickey also comes to the same conclusion that optionality doesn't belong in a data structure, but for slightly different reasons: https://news.ycombinator.com/item?id=17906171

Maybe someone figured out they would get half the response the first time and the rest the second time.

If x represented a set with some values

  x = {RED, GREEN, BLUE}

It would be really swell if the optional type `Maybe x` were a strict superset:

  Maybe x = {RED, GREEN, BLUE, None}

This means functions that map `Maybe x -> y` can accept values of type `x` without issue.

But in Haskell (and OCaml) it's not. What we actually have is something more like

  Maybe x = {Some(RED), Some(GREEN), Some(BLUE), None}

which is a totally different set from `x`, which is why functions which shouldn't break, ultimately do.

The downside of that (which can be provided by union types) is that `Maybe (Maybe X) == Maybe X` which may or may not be what you're after. The particular weakness is the interplay with generics where `forall t. Maybe t` depends upon the `Maybe` and the `t` not interacting. If you've got `forall t. t | null` then this type may secretly be the same as `t`---and if you handle nulls then it might conflate meanings unintentionally.

So you're saying that the one Nothing has a different meaning than the other Nothing, and that this is something that anybody would want? Or that we have a Nothing and a Just Nothing? I'm sorry, but I don't get how that would be considered good design in any context. Do you have any example where this would be considered sensible?

Imagine you're working on a compiler. You need to represent compile-time computed value of type Maybe Int (e.g. you are precomputing nullable integers).

You see 1 + null. So you have add: Maybe Int -> Maybe Int -> Maybe Int, that takes two precomputed values, and returns new precomputed value for the operation result.

However, you can't precompute Console.readInt().

For some expression, you can either be able to compute value at compile time, or not.

What is the output type of compileTimeCompute: Expr -> ???

I don't understand your example. What does compile-time computed stuff have to do with readInt()?

I get that it might be possible to do that, use a Maybe Maybe T. But it's like an optional<bool> in C++. It can be done, it's just not a good idea. So if you design your system not to allow that in the first place, nothing of value was lost.

If you have specific error cases that you want to communicate, like "what was read from the console didn't parse as an int" as opposed to "the computation didn't find a result", then using the two values "Nothing" and "Just Nothing" as the two distinct values to encode that is not a sound design. Either you have meaning, or you have Nothing. Nothing shouldn't have any meaning attached to it.

> "what was read from the console didn't parse as an int"

I meant what you read from the console can not be computed at compile time.

Having a situation where one handles both `Nothing` and `Just Nothing` in the same context should be rare.

But you might be writing some functions on some abstract data structure with a type parameter `a` (say it’s a graph and users can tag the vertices with values of type `a`). And (maybe just internally) there are situations in your algorithms where a value might be absent, so you use `Maybe a`. If `Nothing == Just Nothing`, your Users can’t use a maybe type for `a` anymore because your algorithm wouldn’t be able to distinguish between its `Nothing`s and the user’s.

I am.

In any concrete context, Maybe (Maybe A) could _probably_ be simplified to just Maybe A as we expect. Alternatively, we could be in a situation where there are two notions of failure (represented here as Nothing and Just Nothing) in which case we'd be better off simplifying to Either Error A where Error covers you multiplicity of error cases.

But while these are all obvious in a concrete context, what we often are doing is instead compositional. If I want to write code which works over (Maybe a) for some unknown, library-user-declared type `a` then I may very well end up with a Maybe (Maybe Int) that I can't (and mustn't) collapse.

As a concrete example, consider the type of, say, a JSON parser

    ParserOf a = Json -> Maybe a

We hold the notion of failure internal to this type so that we can write

    fallback : ParserOf a -> ParserOf a -> ParserOf a

which tries the first parser and falls back to the second if the first results in error. We might also want to capture these errors "in user land" with a combinator like

    catch : Parser a -> Parser (Maybe a)

If we unwrap the return type of `catch` we get a Maybe (Maybe a) out of a compositional context (we can't collapse it). Additionally, the two forms of failure are distinct: one is a "handled" failure and the other is an unhandled failure.

Similarly, a polymorphic function may want to put values of an unknown type in a Map or MVar. From the outside it may be completely reasonable to call that function on Maybe Foo, which would mean a Maybe (Maybe Foo) somewhere internally and I would struggle to call that bad design.

> In any concrete context, Maybe (Maybe A) could _probably_ be simplified to just Maybe A as we expect.

And doing so is just what `join :: m ( m a ) -> m a` specialises to for `m = Maybe`!

That's just one way to do it. Another might be

    collect :: Maybe (Maybe a) -> Either[Bool, a]

But you mentioned simplifying `Maybe (Maybe a)` to `Maybe a`, which your `collect` doesn't do (at least not directly).

(Also, shouldn't `Either[Bool, a]` (which I don't know how to make sense of) be `Either Bool a`? Even with this signature, I'm not sure what the implementation would be, and Hoogle doesn't turn up any obviously correct results ( https://www.haskell.org/hoogle/?hoogle=collect https://www.haskell.org/hoogle/?hoogle=Maybe+%28Maybe+a%29+-... ), but that's probably my fault.)

I think the obvious implementation is `maybe (Left False) (maybe (Left True) Right)`

If we have a value then we clearly have both layers. If we don't have a value then we need to distinguish Nothing from Just Nothing by way of the book.

Oh, I see; we're thinking of `Maybe a` as `a + ()`, and then identifying `(a + ()) + ()` with `a + (() + ())` and then `() + ()` with `Bool`. Thanks!

Right, exactly!

Yep that's why as Hickey says people are excited about union types. But in the context of a refactor or a change to an API where you care about breaking existing users, there's a perfectly valid way forward.

Basically I don't think Option types are as unwieldy as Hickey is making them about to be. Indeed Dotty (one of the examples in the talk) will be keeping Option even in the presence of union types (mainly because union types fall down with nested Options and with polymorphism).

That being said Haskell (and Scala) is annoying when it comes to trying to combine different kinds of errors together, if you want an error type more expressive than Option.

OCaml here (and Purescript) have a much better story with polymorphic variants.

In OCaml you have polymorphic sum and product types, so the function

    val f : [`Red | `Green | `Blue | `None ] -> y

Would accept a value of type [`Red | `Green | `Blue ] quite fine.

Though I'd still prefer a haskell false positive to a clojure's false negative. Also, if something is considered a string, I expect it to be any string but NULL.

> If Maybe is an input just keep the original function around and provide a new one that wraps it.

The problem with that is now you have to maintain two functions that do the same thing just to satisfy the type system.

> If `Maybe` is an output then existing callers SHOULD be broken,

He agrees with you on this, according to the talk

1. They don't do the same thing though, i.e. it's not just a wrapper. Dumb wrappers are a problem though in a lot of typed languages in other cases.

2. Ugh this is what I get for watching a video while coding on the side. You're right.

Even without wrapping, I'm not sure that "it breaks existing callers" is such an awful thing. Yes, it is slightly annoying. And if it was a language like Python, making a function suddenly start returning None is going to silently break existing callers, which is pretty awful. But in a language like Haskell, the compiler will find every broken callsite immediately, and the cleanup needed on the caller side is very straightforward.

All that aside, in my experience the case of parameters becoming optional or return values losing nullability actually does not happen that much in practice. It's a bit of a strawman.

This only works if you own the codebase. If a library maintainer changes the signature from X to maybe then it breaks all of the users of the libraries code.

The problem with breaking changes is that the person who breaks things doesnt feel the pain.

No, I'm arguing that this kind of breaking change is fine-ish, because the compiler helps the end users upgrade their code to the new API in a very straightforward way.

In other words, I'm arguing that this kind of breaking change is (1) rare, and (2) ok anyway if you have a strong enough type system that the compiler will just tell you what callsites need modification, and the modification is simple (as in this case).

I think "don't break anyone's code" > "break code but tell user about" > "silently break code".

Rich Hickey has very strong feelings about breaking other people's code, and I tend to agree with him about: don't do it. Breaking other people's code is mean and unnecessary.

I don't disagree, except to say that "don't break anyone's code" is a bit better than "break in a way that the compiler catches and helps you fix", which is a GREAT deal better than "silently break code".

Paranoia about breaking code makes sense in languages that have few mechanisms for aiding in refactoring. It makes less sense when the language is acting as your teammate.

I'm an outsider of Elm community, but from my perspective, breaking changes in Elm 0.19 is still a pretty big deal where lots of people complain about.

Talking about refactoring, I think it's much easier when you are working with a codebase that built by composing lots of small pure functions together. The place where it gets tricky is the part with side-effects where you have to carefully test it either manual or automate anyway. Type error will likely to be caught during that process. With Clojure Repl, we can test that out pretty quickly.

Elm’s 0.19 update was not super-well handled because it had quite a few breaks that are not easily handled (removing all custom operators, which make certain code structures different, disallowing Debug in packages where the new (arguably correct) way requires restructuring, etc.) Just introducing Maybes in a function parameters or removing them in return types would have been quite painless, I think.

>If `Maybe` is an output then existing callers SHOULD be broken, because they must now handle a new possibility of failure they didn't before.

Look at the example again. The output was Maybe y "yesterday", "today" the output is y. Existing callers shouldn't have to change - there is no new possibility they need to handle - they no longer need to deal with the possibility of Nothing.

It's an interesting one - they shouldn't be broken, but they should update their code when they can since they have a bunch of code to deal with a case that can't happen any more.

Could the title be changed to include "- Rich Hickey". I know I am certainly more likely to click the link when it's one of his talks.

Thanks dang

It did originaly. It is the full title and, without it, it's very vague. Maybe if I had remembered to put [video] in there, the mods wouldn't have touched it. Or maybe not.

It's pretty meaningless currently ("Maybe Not [video]"). It would be nice to know, at the very least, that it's a video of a talk, but info. about the actual content of the talk would be preferable.

Great talk, really excited to see how he is going to solve imports and reification for Clojure [so far only rust and haskell have this figured out].

I'm just so glad that Clojure exists, it is doing all the good design things to make programming pleasant but it has a different approach from the usual suspects which is incredibly valuable!

Being dynamic and interactive is great for UI/UX design since that, especially, benefits from the tight feedback loop and I'm becoming more and more bullish on making the type system separate like this and thinking of it as synonymous with tests, it makes a lot of sense (separation of concerns...)

Also coming from mathematics I can see a analogy to the preference of working in open sets (so that you can always take a point arbitrarily close to the edge and still make an open neighborhood around it) and what he is calling an open system. This preference for open sets over dense ones seems counterintuitive at first sight but is the basis on which analysis is built. Type systems feel a bit 'dense' in the sense that they force you to write programs that are overspecified?

Anyway I just look forward to seeing experience accumulate with these different technologies so that we may learn more about how we ought to design systems !

"I got six maybe sheep in my truck."

I have this strange feeling, that first half-hour was totally unnecesary for what the sencond half is. I mean, that it feels that he have to rant always about. Like ok, this ideas are wrong, this is my idea that is totally new(not really) and is the right thing to do. Nowadays, that ranting about Java is not so cool, he have to rant about types. Only to finish saying that he is going to add to Spec something that have already been implemented in Clojure https://github.com/plumatic/schema. So at the end of the talk, what's the point?

Punchline of the talk starts at 38:00

I'm really excited about the way this is progressing. As a way to think of it (and not demote it in any way) is that it's like PropTypes with React done with reusability and less verbosity. The reference to GraphQL really makes it clear that we want deep separated specification.

Great talk! I’ve watched a lot of Hickey’s talks over the years, and thought he might have "run out of steam" by now. But there are some new insights here that I enjoyed.

One of his main points is that Maybe is context-dependent, and other aspects of schemas are as well. This is a great and underappreciated point.

It’s related to a subtle but important point that Google learned the hard way over many years with respect to protocol buffers. There’s an internal wiki page that says it better, but this argument has spilled out on to the Internet:

https://stackoverflow.com/questions/31801257/why-required-an...

In proto3, they removed the ability to specify whether a field is optional. People say that the reason is “backward compatibility”, which is true, but I think Hickey’s analysis actually gets to the core of the issue.

The issue is that the shape/schema of data is an idea that can be reused across multiple contexts, while optional/required is context-specific. They are two separate things conflated by type systems when you use constructs like Maybe.

When you are threading protocol buffers through long chains of servers written and deployed at different times (and there are many of these core distributed types at Google), then you'll start to appreciate why this is an important issue.

(copy of lobste.rs comment)

Reply on news.ycombinator.com