Hacker News Reader

Comment by kodablah | Hacker News Reader

They could have gone hardcore and said any non-primitive, unanotated platform type is nullable but that would have made interop really ugly. And all of the null checks would have muddied up code and added (admittedly minimal) runtime costs.

I want something in between Kotlin and Scala. I want option as a real type that is treated as a max-single-element collection that can be flattened with the same APIs as other collections. But I want no runtime penalty. People in Rust are so lucky to have zero cost abstractions for these things. I suppose I'll need better compile time support (even more than Scala macros) and whole program optimizations (i.e. cross JAR) to get zero cost optional on the JVM. Things like Scala Native use LLVM and surely Option things are inlined or otherwise optimized out.

View on HN

I wrote a zero-allocation `Option`-style (monadic) data structure for Scala a while ago [1]. Unlike all the previous attempts, it supports the distinction between `None`, `Some(None)`, `Some(null)`, `Some(Some(None))`, etc., which is what allows it to remain monadic. The surprise is that it does not use `null` as the `None` value. The downside is that `toString()` is altered: `Some("hello").toString()` returns `hello`, not `Some(hello)`.

There was an experiment to use it as a replacement for the implementation of `scala.Option` in the dotty compiler code base [2], but it is inconclusive so far; it should be tried directly in the collections library.

[1] https://github.com/sjrd/scala-unboxed-option

[2] https://github.com/lampepfl/dotty/pull/3181

Just curious, if Some(hello).toString produces `hello`, what does None.toString() and Some(None).toString() and Some(Some(None)).toString() produce? (My guess is `None` for all)

I think it's impressive† that you got nesting to work, I'm really curious how you pulled that off for longs and doubles without incurring extra overhead.

† Though, given the magic you worked in getting Union types working in the ScalaJS facades I shouldn't be surprised.

Oh primitive types do get one level of boxing (on the JVM): a `Double` becomes a `java.lang.Double`. But it doesn't become a `Some` with a `java.lang.Double` inside, so we gained one allocation anyway. It is not possible to remove that box without compiler support, and even then not in all cases (return values for examples) because `Double` contains a finite amount of values 2^64, and `Option[None]` has 2^64 + 1 values.

And in that implementation, `None.toString == "None"`, `Some(None).toString == "Some(None)", etc. Although that could be changed.

Seems like one could use the wasted digit of signed numbers to stor options rather than have asymmetric range (two's complement) or positive/negative zero... IE have a [ed:bit string] that indicates none/some?

It's no longer monadic in the presence of a pervasive `null` value. Such is the power of null - it breaks abstractions that have nothing to do with it.

It is monadic if the monad laws about it hold. `null` has nothing to do with it. In the context of my unboxed option, `null` is like `5`: one of many primitive values, uninterpreted by the abstraction, and therefore it does not break the abstraction.

If you think my unboxed option is not monadic, please provide a counter-example to one of the monad laws.

> They could have gone hardcore and said any non-primitive, unanotated platform type is nullable but that would have made interop really ugly.

Instead you just get a crash if it is null which is worse in my opinion.

> And all of the null checks would have muddied up code and added (admittedly minimal) runtime costs.

This already happens automatically for parameters to a Kotlin function; check out the Kotlin Intrinsics checks.

> Instead you just get a crash if it is null which is worse in my opinion

Only in the same instances you would in Java-to-Java. So the ergonomics aren't improved or reduced. And really, this only helps on returns anyways. Marking every Java param nullable gets you nothing if the implementer didn't handle it well.

> This already happens automatically for parameters to a Kotlin function; check out the Kotlin Intrinsics checks.

Yup, and I don't like it. So even public Kotlin-to-Kotlin calls suffer. Haven't checked in a while, but I would like an option to use annotations only and skip those top-of-method checks.

> Only in the same instances you would in Java-to-Java. So the ergonomics aren't improved or reduced.

This isn't correct. For example you have a String that was passed as a param and were shoving it into JSONObject. In Java if it's null nothing bad happens (except shoving a null value into your JSON). Shoving a null into a JSONObject in Kotlin won't crash in the put operation; you'd get the crash as soon as the method is called and it does the Intrinsics check.

Relevant: https://discuss.kotlinlang.org/t/run-time-null-checks-and-pe...

I've measured the null-checks and other people have too... the runtime performance cost is negligible, so disabling them would get you nothing useful. If they did, Kotlin would have made the compiler flag to disable them (which exists) public... but until someone comes up with an actual reason other than "I feel like it's better" that won't change.

I have reasons. For example, I need to have as few instructions between a JNI call and the next line (I am starting a SingleStep JVMTI callback as part of my development on a fuzzer) and those intrinsics don't help. It's gotten to the point on my advanced JVM projects where I have to just drop into Java all the time to avoid this stuff, get the proper MethodHandle.invokeExact semantics, etc, etc.

I'm tired of being surprised by the bytecode that Kotlin generates. You could argue javac has some magic too, but not near as much and it's pretty well spelled out whereas you won't find the docs for all of these intrinsics.

It's even worse when things just get dismissed as "I feel like it's better" and then when you provide reasons people will say it's not normal. Languages (and their proponents) should not treat the end user (i.e. the dev) like they are stupid, or at least give them the option to turn things off. The trade-off of hiding this stuff is not worth it...it's more like it's the language people doing the "I feel like it's better".

Null checks are effectively free on the JVM. One of the advantages of using null-as-real-null that is rarely discussed by functional programming fans is that null is well supported by the hardware. Just keep the bottom pages unmapped and an attempt to access a null pointer will trigger a hardware fault. If you don't do it that way then you have to do the null checks with branches in software, which bloats your code footprint and now with Spectre perhaps requires slower code too.

> Null checks are effectively free on the JVM

I don't want the bytecodes added. I have tracing I am doing and I need more predictable instructions. It's not just a performance issue.

Also, you should not assume cost of a JVM implementation of things based on your empirical evidence from a single, popular VM. For example, how does TeaVM translate it?

As for your point wrt to options in Scala, there were some libraries that used marker values for none (e.g. NaN for double) that then used value classes to basically provide a zero alloc option type. I’ve done my own internal versions for similar purposes. They work well with the caveats of value classes in Scala having their own set of issues.

For doubles you don't even need to take over NaN. There is a huge range of unused bit patterns in IEEE floats that could be used instead.

I don't think the JVM guarantees the bit pattern will be preserved when passing it around. I know this is the case for NaNs where I have experienced the bit pattern change [0].

0 - https://stackoverflow.com/questions/43129365/javas-math-rint...

It would be really depressing if java normalized NaNs, but i wouldn't put it past some author thinking it was a good idea.

It's not that they normalize them per se, it's just that NaN bit patterns are mostly undefined behavior. I think the JVM spec (or JLS, I forget) implies that you can't really rely on the NaN bit patterns being retained. Double.longBitsToDouble mentions that some processors might not return a double in the same bit pattern as passed.

Such as?

Reading up on the format [1] it seems that every value is defined, which includes a ton of possible NaN values [2].

[1] https://en.wikipedia.org/wiki/Double-precision_floating-poin...

[2] If you read the actual IEEE 754 spec, one suggested use for a NaN value is the address of the offending instruction(s); otherwise there is no defined pattern for NaNs in IEEE-754.

> it seems that every value is defined,

> there is no defined pattern for NaNs

I was referring to using a NaN value without a zeroed mantissa.

A NaN is defined as the maximum exponent (all ones) and a non-zero mantissa. By "no defined pattern for NaNs" I meant there was no fixed value for a single NaN.

I think you can make it feasible with a few features.

Not sure which of these kotlin has if any

1. declaring that a type isn’t null with ! appended.

2. Suppression annotations beatable at the file / class / function / line levels to disable the check

3. Facades / Decorators that can be added by anyone that tell the compiler retrofitted additional typing information on existing libraries (the defintely typed / typescript approach)

ad 1: `!` in kotlin actually mean "platform type that could be nullable or not". When you add !! to a value of that type, you declare it as non-nullable.

Reply on news.ycombinator.com