Add detailed message to NullPointerException describing what is null
openjdk.java.net
openjdk.java.net
https://www.fxsitecompat.com/en-CA/docs/2018/improved-javasc...
This is an unfortunate example of Hyrum's Law: "With a sufficient number of users of an API, all observable behaviors of your system will be depended on by somebody."
Similarly, I would argue that by relying on undefined behavior it was the FlipKart engineers breaking the web.
1) It's not just Flipkart. It's a library that was being used on multiple sites. For example, lego.com was broken too, for the same reason.
2) Firefox can't tell Flipkart to go to hell per se. All it can do is tell its users to go to hell and not use Flipkart (or lego.com), or at least not use them in Firefox. Phrased that way, I think it's clearer why Firefox cared enough...
https://github.com/facebookincubator/idx
>The idx runtime function exists for the purpose of illustrating the expected behavior and is not meant to be executed. The idx function requires the use of a Babel plugin that replaces it with an implementation that does not depend on details related to browser error messages.
"Every change breaks someones workflow..."
I am Jack's complete lack of surprise.
And by all means, break them. They're undefined behaviour.
This is why we can't have nice things.
How long would have taken to fix? How much time would we all now safe?
if (ex instanceof NullPointerException)
catch (NullPointerException ex) {
window.foo.bar
//=> TypeError: window.foo is undefined
window.foo.bar
//=> Uncaught TypeError: Cannot read property 'bar' of undefined
TypeError: Cannot read property 'bar' of 'window.foo' (undefined)
Edit: Formatting
I like the solution proposed here.
Sounds like they were considering trying to get sites to fix this and then redo the change, but it's not clear what happened to that in the last 5 months.
I read all the comments, didn't get that impression, can you say why you do?
I suppose that the "normal" response is just to say "yeah people will write bad code in any language," but maybe there was something we missed out that could have signaled to someone that this was a very bad thing they were doing? It's something that could easily happen to a junior developer. So if we had the language of our dreams, is there some other option?
Erlang's "just let it crash, do not attempt recovery but rather monitoring" comes somewhat to mind. Standardizing the error message also comes to mind, but seems a bit banal. Error codes are handy but with C I have noticed that every single piece of software has some different idea of which integers mean what things, so that developers have to absorb a hodgepodge.
Really it's I guess that you don't want any sort of conditional dispatch to happen on such a value? You would like the ability in an hypothetical programming language to mark a field as "advisory", and then the programming language checks that it never performs any sort of conditional dispatch on an advisory field. That gives you a way to make the API contract, "this contains a string but I might change that string at any later date." So if a junior dev runs into that, they are intimidated and get that sort of instant feedback they need.
Interestingly this happened with Java exception stack traces too. Joshua Bloch remarked that one needs to provide access to all info through standard API, if it's not there people start using toString and others and it bacames a de facto API.
Also, you may still want dispatch on "what sort of exception was this?" so then you have to provide an alternate mechanism for that...
Every change breaks someone's workflow. [1]
With a sufficient number of users of an API,
it does not matter what you promise in the contract:
all observable behaviors of your system
will be depended on by somebody. [2]
That said, it sounds like the "real" problem here is that there's no programmatic way to investigate details an exception, and the answer should likely be to make an extensible structure (to which you can add new fields later) that contains the meaningful information in the exception string.... Ideally the exception string should be a static, probably translatable template string that you can use with the language's printf equivalent to fill in the per-exception data in the structured fields.
https://philsturgeon.uk/php/2013/09/09/t-paamayim-nekudotayi...
TypeError: Cannot read property 'bar' of undefined 'window.foo'It happens quite often that we get a bug report with "We got a NPE". Our users often attach a screenshot, just showing the standard "NPE error" message, which does not really help.
"Frequently" is obviously a relative term, but are NullPointerExceptions really common enough to be a performance concern? It's good that they are taking performance overhead into consideration of course, I'm just surprised it's even an issue.
Throwing an exception in Java is a non-trivial expense since the constructor for Throwable populates a stack-trace structure for the current thread. If you're using exceptions as a routine flow-control mechanism, you're incurring that cost too frequently.
Like you said, sure, tune the performance of this new feature because it's part of the standard library and should be reasonably well-tuned. But if people are frequently throwing NPEs, they should consider alternatives such as simple null checks.
InterruptedException are rare (very) and used mostly to signal termination attempt. LockSupport.parkNanos doesn't throw the exception (unlike sleep, and sleep should never be used in code) but the interruption signal/flag has to be respected somehow. Most of the time is rethrow in library code or bailing out with IllegalStateException in 'userspace'
But it's reasonably easy to imagine ways to produce an API that only uses tokens like that instead of using exceptions to signal cancellation/termination requests. And I suspect, given how many bits have been spilled trying to explain to people just how they're supposed to deal with InterruptedExceptions, that the resulting system would have better ergonomics.
Long story short, I don't know that it's that InterruptedException was a necessary evil so much as that people were still fairly new to this whole exceptions thing when Java came out, and it's really hard to get subtle things like this right on the first try.
Unfortunately, the JVM has special code to detect this case (!) and stop recording the stack trace after a while (!!). Then when you go look at your log to see why the system isn't working, you see only `NullPointerException` without the expected stack trace, and you have to guess where it's coming from (after wondering for a few hours why the stack trace went missing, only to find the explanation at https://stackoverflow.com/questions/2411487/nullpointerexcep...).
I'm not exactly sure what it tells us, but it definitely tells us something.
try{ .. business logic .. } catch (NullPointerException e) { .. else .. }
Rather than a null guard. That's what occurred to me.
It's the code that is the problem here.
If splitting on separate lines makes code more readable, then sure, do that. But often it just makes the code longer and harder to follow.
A task I thank the developer who creates a variable to hold such a reference, for it creates a much better experience.
Long method chains with a strong type system or a functional (also called "fluent") style are extremely expressive and common in languages that obviate the need to handle null values at every member.
Everything is just as testable and decoupled, it's just a different project structure.
Method chains have nothing to do with the things you stated.
There is no null check in the generated assembly code, when a null dereference occurs - it's an effective kernel trap (as 0 is not mapped to the process). The latter uses the code execution pointer to understand what has been attempted to execute and throws the exception. This may or may not involve stack crawl (which is very expensive) depending on JIT ability to prove if the stack would be unused.
Nulls should be avoided, all fields should be initialized, etc. Nulls are great for =very= high performance code as null checks are virtually free.
On Win32, a segfault / access violation - and similar low-level errors, like division by zero - is represented as something called "structured exception". These have a standard OS ABI such that they can be caught, and stack can be unwound, across different languages.
In MSVC, normally, they are not treated as C++ exceptions for the purpose of catching them, although it will still participate in stack unwinding (if something else below is catching them). However, there is an opt-in compiler switch that does make them look like C++ exceptions in a sense that catch(...) will catch them. It's not something you'll see in most code written in the past 15 years or so, for obvious reasons.
Letting null values propagate through your program is frequently a code smell, you want to constrain it at the boundaries of your program logic if possible. Unfortunately, Java does not have a semantic way to do this, just documentation and discipline.
https://www.oracle.com/technetwork/java/javase/relnotes-1391...
https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...
See also Brian Goetz opinion on that subject: https://youtu.be/FdkPHShh628?t=50m17s
https://msdn.microsoft.com/en-us/magazine/mt829270.aspx?f=25...
But even with the setting turned on, you're still free to ignore the compiler warnings (assuming you haven't also told it to treat warnings as errors) and blithely assign null to a reference that wasn't flagged as nullable. So it's not quite non-nullability.
Which isn't to say that it's a misfeature. It's a huge step in the right direction that will be immensely helpful, and they managed to do it in a way that doesn't break the mountains of C# code that already exist. These sorts of methodical, well-considered design decisions that keep the language moving steadily forward without causing chaos for its users are exactly what's so great about the C# team.
Failed to write HTTP message: org.springframework.http.converter.HttpMessageNotWritableException: Could not write JSON: while trying to invoke the method de.hybris.platform.catalog.model.CatalogModel.getId() of a null object returned from de.hybris.platform.catalog.model.CatalogVersionModel.getCatalog();
In fact I relied on it so much that I thought every JVM does this.
I work on mainframe (z/OS), and it is completely normal there that when an application fails with exception (they're called ABENDs - from ABnormal ENDing), you can get a dump of memory of the application. From that, you can see all the values of the offending variables and all the relevant system areas and so on.
Why not have a top level try catch then?
So instead, in Java, it's your responsibility to catch and correctly handle all the exceptions.
Also, variables in the last function (where exception was raised from) are unlikely to give me understanding of the bug, because it's usually thrown somewhere deep in others code, and also I'm more interested in _why_ it happened, not just _what_ happened.
We could dump all the variables in the stack trace, but I'm not sure I would like to read the whole dump of every variable in stack trace, that's just too much. I'd better write my code with proper logging, so I would be able to read the logs in order to understand not only _what_ happened, but _why_ it happened. With good logs you rarely even need to debug anything, I'd say 95% of the time it's pretty clear where to fix.
I know from experience on z/OS how useful full dump can be, and I sorely miss it in Java. You wouldn't have to read the full dump yourself - a good postmortem debugger can do that work for you - displaying stack trace, locals, etc.
Good logs can certainly save you.. if you have them, and you happen to log exactly what you need. But it's much easier to look at the memory and see what went wrong. (Personally, I think logs are great on the boundary, to describe inputs and outputs.)
I also think that the exceptions are kind of wrong approach (compared to z/OS ABEND conditions), and it's not entirely true that failing loudly is a bad practice, but that's for another discussion.
In short, that's what logging is for and there are some awesome options for that in Java. We log all exceptions with stacktrace + full MDC (mapped diagnostic context) to a central logging cluster. Also, we start alerting if that happens. Most application servers will have a top level catch and a way of logging/handling exceptions (e.g. sending a 500 response for unhandled exceptions).
The JVM crashing usually means you are hitting a jvm bug; it's not supposed to do that. It's been a while since I had that going on. Even out of memory exceptions tend to happen in a somewhat orderly fashion. It will print some message probably. Setting -XX:+HeapDumpOnOutOfMemoryError does what you want here: dumping the memory. Of course the use of that on an ephemeral disk in e.g. Docker is kind of limited. These heap dumps are super detailed though and you can dig through them using special tools. A common issue with docker is not setting limits correctly and then docker killing your process instead of it exiting the normal way.
You can dump a stack trace of all threads to std out by sending kill -QUIT to the process (does not kill the process). Great for debugging deadlocks.
-XX:OnError="jcmd %p GC.heap_dump /heap-dumps/%p.dump"
Pretty much all byte code editing tools would be obsolete as the constant pool needs change and so on. Also the generated byte code would be more verbose (worse load times)
The current change is easy to implement and it'd work on code compiled for/with java 1.0.
I've left this year the JVM for TS, but glad for everyone who uses the JVM.
service.withName(customer.getName().firstName);
String s = customer.getName().firstName;
service.withName(s);
I'd prefer this be fixed in the JVM, so you can decide to write your code in the way that's most readable, and not be forced to write it a certain way because the tools suck.
And it's even
val n = customer.getName()
val f = n.firstName
service.withname(f)
The L in Java stands for Low Memory.
each dereference where NPE can occur has had 'metadata'. It's compiling metadata not object one.
Examples:
https://en.cppreference.com/w/cpp/utility/optional/value
https://docs.oracle.com/javase/8/docs/api/java/util/Optional...
If you're given an Optional that's guaranteed to have a value, it's better to call get and blow up if someone violates this invariant than to silently ignore the invalid input via pattern matching or optional chaining.
Crashing is better than silently ignoring logic errors.
Of course, one could ask what's the point of an Optional that's guaranteed to be non-nil. Well, it often happens that there's some relationship between a set of values that's hard/awkward to express via the type system. E.g. a toy example would be that A and B can both be nil, but it also holds that (A == nil) == (B == nil).
There are unsafe methods to extract the value in those rare occasions when you absolutely know it's there, but you seldom need them. You're supposed to map and pattern match. Neither is silent, so I don't understand what you meant.
> Crashing is better than silently ignoring logic errors.
Map and pattern matching are not silent.
----
[0] https://www.oracle.com/technetwork/articles/java/java8-optio... (search for "this is not the recommended way of using Optional", and bear in mind this was for Java 8, which doesn't have pattern matching like Scala or an ML language)
I'm writing a complicated algorithm and am at a point where I know for certain that a given value must be non-nil. Sometimes it can be nil, but at this point it's provably non-nil (E.g. at the end of Dijkstra's algorithm every entry in the distance vector must be non-nil - assuming the graph is connected). The type system can't prove this, but I know that it's non-nil at this point.
So, when I access it, I want the program to fail loud and clear, because it means that I've messed up somewhere. Handling it via map or pattern matching makes no sense here. What would you write in the nil case of the pattern matching anyway? return -1? That's absurd. abort()? That's exactly what get() does.
If it's provably non-nil, use get. But you must provide the proof, and your first reaction must always be "I'm not sure I can assume this".
> What would you write in the nil case of the pattern matching anyway? return -1? That's absurd.
Yes, it's absurd. You don't return this type of sentinel values in statically typed languages with optional types. If you don't need to discriminate the error, sometimes returning Nothing will suffice. If you need the error, you can use an Either type (where traditionally Left is used for the error).
Again, this isn't something I made up. It's how Optionals are used. I suggest you read the link I posted above.
I think that sort of pattern is a bad code smell, you are propagating doubt where there is none leading to misleading code that has assumptions that differ from what's on the page - obviously I don't have a real example in front of me so there may be good allowances for understanding but I don't find your core premise to be sound and I feel like any counter example would have a lot of work to do to appear convincing.
Nope that's bad programming. If you know the Maybe is always there, PROVE IT! Step 1 to proving anything: Make a proposition. In other words, swap out `Maybe a` for `a`. Then prove it upstream. Maybe at some point you want to crash, but you definitely want to do it at the boundary and not within your function. Writing a function that takes a Maybe but assumes it's there just seems like asking to break things.
> a toy example would be that A and B can both be nil, but it also holds that (A == nil) == (B == nil).
Isn't this just `Maybe (A, B)`? If you have both sums and products in your type system, these sorts of situations go away.
A more complicated example: You have A and B and A, B, or both are non-null. But both are never null at the same time.
The way to lift this invariant into your type system (i.e. make a proposition about your program) is:
data IOr a b = ILeft a | IRight b | IBoth a b>If you have both sums and products in your type system, these sorts of situations go away.
Like I said, it’s a toy example. If you need a different example, here’s one: A, B, C and D are Optionals, but at most two of them are nil.
> A, B, C and D are Optionals, but at most two of them are nil.
That sounds like something that's quite fragile and should either be wrapped up in its own type or not depended on.
You can write a verbose sum-of-products to encode that ABCD example although if it's really true that only two tops can be nil and it can be any combo and they're all disparate type, I'd love to see what real problem that would be about! Seems nuts.
Lemme see if I can have some fun and encode it more elegantly than "unrolling the permutations" :)
{-# LANGUAGE GADTs #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE KindSignatures #-}
module ABCD where
import Data.Kind (Type)
data N = Z | S N
data AtMostN (n :: N) (ts :: [Type]) where
AMN_Nil :: forall m. AtMostN m '[]
AMN_Pass :: forall t m rest. AtMostN m rest -> AtMostN m (t ': rest)
AMN_Cons :: forall t m rest. t -> AtMostN m rest -> AtMostN (S m) (t ': rest)
λ: AMN_Cons True (AMN_Cons True AMN_Nil) :: AtMostN (S Z) '[Bool, Bool]
<interactive>:7:1: error:
• Couldn't match type ‘'S m0’ with ‘'Z’
Expected type: AtMostN ('S 'Z) '[Bool, Bool]
Actual type: AtMostN ('S ('S m0)) '[Bool, Bool]
• In the expression:
AMN_Cons True (AMN_Cons True AMN_Nil) ::
AtMostN (S Z) '[Bool, Bool]
In an equation for ‘it’:
it
= AMN_Cons True (AMN_Cons True AMN_Nil) ::
AtMostN (S Z) '[Bool, Bool]
We can say at most two are set and now it compiles:
λ: AMN_Cons True (AMN_Cons True AMN_Nil) :: AtMostN (S (S Z)) '[Bool, Bool]
λ: AMN_Pass (AMN_Cons False AMN_Nil) :: AtMostN (S Z) '[Bool, Bool]
In many languages that support Optional types as first-class citizens, it is a compile-time error to ignore the fact that it may be null/none. Your code must deal with this, and you won't have any surprise exceptions, even if you are lazy.
However, they also tend to provide escape hatches for getting at the type directly, which defeats that and will still let you NPE.
Might as well beat up on Java: One of the best belly laughs I had during a debugging session was tracing a `NullPointerException` to the source, and finding that the reference was to an `Optional<T>` that was being used to handle a value that might not be present. Of course, the null reference here was to the `Optional<T>` itself.
There are also also some ergonomic issues. Obvious ones like the lack of pattern matching making them a bit more verbose to work with, of course. But also deeper, more annoying problems, such as Java's standard library lacking a really well-thought-out equivalent to the various abstractions on sequences that are standard in functional languages, and, by extension, functions for operating on them. Though it does have plenty of bits and bobs that were diligently cargo culted on without really capturing the essence of what this stuff is supposed to be about. So, for example, while (at least in Java 8) `Optional<T>` does have its own method called `flatMap`, it doesn't implement `Stream<T>` or offer an easy method for producing a stream. So the idiom of flatmapping a function that returns `Optional<T>` over a sequence to yield a (possibly shorter) sequence of items for which the function returned something, which is so simple and readable in an ML-like language, becomes hopelessly verbose, awkward, and even error-prone in Java.
(Also, Java Streams are, by design, not referentially transparent, so much so that they cannot be made to behave in a referentially transparent way. Which is a total tangent from what I'm talking about here, but does at least give another angle from which to see the depths to which Java misses the point on these sorts of things, and possibly isn't even able to catch it.)
All of which kind of leaves me thinking, "When in Rome, do as the Romans do." Yeah, in hindsight, the ALGOL family of languages' standard method of dealing with cases where a value might not be available is hopelessly fraught with problems. But importing a second approach from a separate language family that split off about 40 years back and started going in a whole different semantic direction into an existing language that already has its own ways of doing things doesn't appreciably improve that situation. It just introduces two competing ways of doing things that end up interacting poorly with each other.
So, to wrap it up: I'm inclined to say that keeping null and just improving the error message when it happens unexpectedly probably is the best band-aid for this problem.
That's a big leap. That's like saying Haskell isn't pure because of unsafePerformIO. You still get nice static checks with Maybes. If people cheat with fromJust, it's at least obvious they are cheating. Both big and obvious improvements over null :)
This reminds me of old C/C++ parsers that couldn't tell you what line was expecting a bracket or semicolon and you got to manually find the parse error.
IMO overhead should not even be a consideration. It doesn't make sense to sacrifice detailed reporting just so someone can repeatedly handle NPEs as part of their application's logic. I can't imagine a situation in which I would look at code that throws enough NPEs for overhead to be a concern and say "yes this is well designed code"
I don't program in Java at all, and frankly, I was flabbergasted to see a change as obvious and useful as this wasn't already in there a decade ago. Or at least 3-4 years ago where a lot more languages started advertising better compilation error messages.
It's proposing to reverse-engineer the corresponding source code from the bytecode when encountering the exception. That does sound like a potentially unreliable hack.
In Haskell, it's pretty much impossible to have unexpected nulls because the ability to return nulls is something that is expressed in the type and needs to be handled before working with the possible type to be able to pass the type-check of the compiler.
For example, if I had a function:
findFistOdd :: [Int] -> Maybe Int
1 + findFirstOdd [2,3]
1 + (fromMaybe (error "couln't find odd number") $ findFirstOdd [2,3])
(1 +) <$> findFirstOdd [2,3]
addOne :: Int -> Int
Kind of wish all languages were like this. Nulls are probably the cause of most unexpected exceptions and it could be something that could always be caught without even running the program.
a) Interop with Java APIs (Kotlin doesn't force you to check every return type from a non-Kotlin call)
b) If you use !! to cast away nullness. Good Kotlin style can almost always avoid such cases.
https://docs.oracle.com/en/java/javase/12/docs/api/java.base...
Of course, an Optional reference can still be null because it's still Java. There are also other implementations of Maybe-like types, and implementations of Either-like types, etc in the Java ecosystem.
This is a petty nit on what sounds like some great work but please do not use the word "simple" in technical contexts like this. Having you assert that it is simple does not make it easier for anyone to understand, and it is really quite irritating. I don't even know what "data flow analysis" is and I've been involved with computer programming and data analysis for 15 years.
You probably haven't been involved with compiler development in these 15 years; data flow analysis is one of the techniques used by optimizing compilers to reason about the code. Given the context (not only there is a compiler converting Java source code to Java bytecode, but also the Java Virtual Machine acts as a "just-in-time" compiler from the Java bytecode to machine code), everyone involved most probably already know what "data flow analysis" is. And by "simple" the author most probably means that a basic data flow analysis, with no extra bells and whistles, would be enough for the described purpose.
It's sort of funny that their example doesn't follow the usual Java style guidelines of encapsulating member variables in the parent and using set() and get() methods for access.
a.to_b.to_c.to_d.num = 99;
This could be rewritten as a method in 'A' as...
protected void setDNum(int num) {
C c = b.getC();
D d = c.getD();
d.setNum(99);
In this case, a nullpointerexception would include the line number (using default compiler options) clearly indicating which is null.
You could lower the overhead by not producing these messages if the exception is thrown frequently. It would be similar to the existing OmitStackTraceInFastThrow mechanism [0].
[0]: https://www.oracle.com/technetwork/java/javase/relnotes-1391...
Is that an ART/Dalvik feature ?
the name of the method causing the crash (toString in that sample) + the line number and location thanks to the stacktrace (although in some rare case it is 100% accurate, but still pretty close, maybe that's due to runtime inlining) and it is very trivial to know where the exception comes from.
Nice humble brag. :D
Yes, ART has had this for a few years now. For example in Oreo this code:
static Integer nums[] = new Integer[100];
public static String stabSelf() {
return nums[new Random().nextInt(100)].toString();
}
Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.String java.lang.Integer.toString()' on a null object referenceThe debugger in VS2017 does give the information, but I suspect thats just by re-evaluating the expression.
Makes me feel that complaints about null aren't really about null itself but languages and OS's not generating useful error messages.
However now I see just about everyone stacking this method calls one after the other. Is this just acceptable now?
state = stateFromPostcode(customer.getAddress().getPostCode());
I realize that all sounds very obvious, but like most things... it really does depend. It can certainly make debugging harder if you need to step through. Personally I try to write code which simply doesn't allow nulls below the first layer they may be encountered. Not always possible though (e.g. nullable fields in entities and whatnot.)
https://dzone.com/articles/eclipse-annotation-based-null-ana...
No, but where before it was a hard requirement in Java to deal with NPEs, newer libs are less likely to return nulls, so it's usually a bit safer than it was. You also avoid leaving references around that could potentially leak.
There is also a bit of fashion with streams and JS-inspired call-chaining.
Not only acceptable but a great idea. Vertical space is a much more limited resource than anything else.
That’s a matter of opinion, and my opinion is hell yes that’s not only acceptable but recommended! If creating a variable makes the code more readable, then do it; if it doesn’t, then don’t. But certainly don’t worry about NPEs.
A similar piece of advice is that you shouldn’t do this:
if (a.equals("hello"))
if ("hello".equals(a))
It turns out that the real advice is not “structure your code to avoid NullPointerExceptions”, but “avoid null”. With this in mind, rules like this one, as well as guidelines telling you not to have too many function calls on the same line, are rendered useless.
Its not a very clear error, but you can narrow down the issue to a few things even when doing application support.
I believe that if you support it long enough, it's your application now, so no one else is to blame. How long is long enough? I'd say 1k-2k of lines of code per month of supporting it should be pretty reasonable. So if you supported a project of 10k lines for a year -- it's yours now, stop blaming original authors.
Good engineer would start improving it little-by-little, or convince manager that they need to spend some time on refactoring some piece of it. That differentiates a coder from an engineer.
Optional.ofNullable(foo).orElse(new Foo())Although I'd rather creating objects if not needed to be easier on memory.
Optional.ofNullable(foo).orElseGet(Foo::new)
And in performance critical code you might find them as well. Sometimes dereferencing an option is too costly, so you put up with an anti-pattern in order to get lower latency or higher throughput.
My advice:
- avoid frameworks like the plague
- write in a functional style
- lean on the type system as much as possible
- don't chase the trendy stuff
- avoid annotations
- use code generation