Bruce Eckel: Strong Typing vs. Strong Testing (2003) | Hacker News Reader

Bruce Eckel: Strong Typing vs. Strong Testing (2003)

docs.google.com

24 pointsadambyrtek29 comments

Bruce nails why strong typing is important, but misses the point. He says, "realizing that the compiler was just one (incomplete) form of testing,"

Strong typing is really just a form of testing. But it has a couple of advantages. It is a form of testing with strong mathematical underpinnings. If someone says their application is correct with respect to the type system, I know what that means in a pretty precise way.

This has a couple of side benefits. When working in modular units or decoupled teams, when one team gets some code they know that if type checking has passed a set of requirements have been met. They understand what that means. While unit tests are useful, you passing your unit tests means very little to me, until I spend a few days going through your unit tests.

Another benefit is that all of your code gets these tests run on them. One of the few tests that gets 100% code coverage all the time.

Examples of famous bugs that could have been avoided with a strong type system include the NASA Mars orbiter bug. If they used a language like Ada that supports having English and Metric integer types, they would have been sure to mix them properly. And of course the compiler could enforce this through all of the code.

While Bruce does point out that weak typing (or dynamic typing) may result in less typing, it generally doesn't result in easier thinking. For me that makes it a relative wash in that regard.

> When working in modular units or decoupled teams, when one team gets some code they know that if type checking has passed a set of requirements have been met. They understand what that means. While unit tests are useful, you passing your unit tests means very little to me, until I spend a few days going through your unit tests.

Except that "type checking has been passed" doesn't actually tell you anything about correctness either without spending a few days going through the types. It's very easy, and very common, for bad programmers to build a few BigBallOfStuff types and essentially re-invent an ad-hoc dynamic type system that will pass static analysis in a way that is utterly meaningless when it comes to behavioural correctness

There is a very big difference. It's that when I write code I need to use your types. If I'm getting a BigBallOfStuff I instantly know that your type checking is constrained based on the types.

With virtually any external testing (unit, integration, UI, or most other) working with the product code is not sufficient to determine the quality of the testing. The tests must be looked at specifically.

Type checking and other forms of static analysis run on the code themself, which means that I only need to inspect the product code -- which I pretty much have to do to write any code with it.

A BigBallOfStuff type should stick out like a sore thumb in a code review, though.

Static typing is a DSL for specifying assumptions in a way the compiler can check. If you misuse it, it can backfire, but that's true of pretty much everything.

That's just a recasting of "if you rigorously specify all of your assumptions in your unit tests, you needn't worry about anything". You can take maximal advantage of the type system to encode assumptions, but it's still on you to do it. This isn't a silver bullet that makes static typing better than dynamic typing, because nothing forces you to be competent about specifying your assumptions here, either.

You can take maximal advantage of the type system to specify as many assumptions as it can check or you can take maximal advantage of the unit tests to specify the same assumptions. You end up in the same place either way, and the OP's claim that you can "trust" the existence of passing type-checks without looking at the types to see what assumptions they encode in a way that you cannot trust unit tests doesn't hold up.

Agreed. I think some mix of declarations and automated (regression, etc.) testing is probably most practical. I like randomized testing, as well.

The difference with test-driven development and continuous integration is that everyone relies very heavily on the tests. So, there is no necessity to check the tests, you start from a position of knowing that the tests are complete.

The NASA issue could equally have been caught by better testing. It's hard to debate that point, because we don't know the strength of the testing in that case. Other than it was clearly not adequate.

Regarding your last point: I find that having excellent unit tests and coverage means that I can concentrate singularly on the design problem at hand and experiment very broadly without any fear of regression. That's an enormously liberating place to be.

I should be clearer. Tests are great. Static strong typing is a built-in automatic test that hits all my code at compile time with known semantics. I like that testing also.

The tests themselves can be broken too.

Test-driven development also requires a sustained effort to not miss all the possibilities of something going wrong. Developers aren't wired like that, that's why many companies have QA engineers (i.e. people specialized in breaking things).

And not everyone has the resources to do test-driven development. Sure it helps in the long run, just like eating healthy and going to the gym, but when your software was supposed to get released yesterday and you ain't done ... tell me you're thinking about branch / path coverage.

But if they had set up this specific metric integer (float?) type for measuring force in, that implies to me that they had already anticipated that the mix-up of units was likely to be a problem. From what I've heard about it, that wasn't the case - neither side thought about it until after they knew it had failed.

It seems like it'd be great if the type system had caught that, but if they expected all software to be using SI units they may not have seen the necessity for it even if they were working in a language which permitted that kind of thing.

So this is one of those things where language features and language culture play a role (like unit testing in Ruby).

In Ada, you attempt to capture the assumptions in either the type of the name of the type. In general, you don't ever use primitive types directly. If you're going to use integer types questions you, for example ask, what are the assumptions? Is the range of integer values from 1-100? If so, then you make an integer type of that range. When you say "50", it's "50 what"? "50 decibals", "50 meters", "50 days"? So then you have types with these names.

If they followed standard Ada dev practice this bug would have probably been obvious. Probably less obvious in standard practice for many, if not most, other languages.

I agree with almost everything in this article, except for one crucial point. The article says "And if a Python program has adequate unit tests, it can be as robust as a C++, Java or C# program with adequate unit tests."

So what is an "adequate" unit test in Python? I know some advocates of weak typing who believe that the unit tests should include tests of expected input types. 99+% of the time that's going to mean checking exactly what you'd get from static type checking. Rarely, there will be something more complex, e.g. the foobars variable can reference a FooBar object or a list containing 0 or more FooBar objects. (And then I'd question the design of the API.)

I really don't want to write those unit tests. I'd much rather have the compiler do that testing for me.

Without a comparison to Haskell, ML, or similar best-of-breed statically typed languages, this truly misses the point. Using C++ and Java as the poster children for static typing is like using DOS batch files as the exemplar of scripting languages.

One thing I've noticed is that a strong type system does force me to think harder about interfaces.

Sure, you can think just as hard about interfaces without static typing. But speaking personally as someone who does mostly Ruby but is building increasingly complex infrastructure in Ruby and trying to break things up sanely -- it helps me to think clearly when I prototype the interfaces and the types in a statically typed language like Scala.

Surely it's meant to be the other way around? But an interesting realisation for me.

On the other hand, there's that feeling of code which does exactly what it needs to do to pass the tests, nothing more, nothing less. Which while very economical and very suited to rapid development, often doesn't stand up in its own right as a respectable design without regular and very disciplined "and now I go back and refactor" steps. Often those steps correspond (to my way of thinking) with going back and thinking deeper about the types and the interfaces.

So, I think types are more than just a limited kind of testing which happens to run statically - they also give you a conceptual framework and structure, in the mathematical as well as the regular sense of the word. An ontology, if that's not too grand a word for it. And as such they can help shape your design and spur you into phrasing your design clearly. Despite occasionally getting in the way of it.

If C++ and Java are your chosen models of "strongly statically typed" languages, it would only be fair to use PHP or Visual Basic as the contrasting language, in particular if a big chunk of your essay will discard the actual issues at hand and instead focus on unrelated topics like brevity. (all of the above are on my personal, subjective, "list of languages that are painful to work in.")

That aside, "strong typing" and "strong testing" are absolutely not on some sort of axis where you have to choose one or the other, and it's absolutely ridiculous to pretend that it's some sort of trade-off.

... It re-occurs to me that this particular issue is flamebait of high pedigree and ancient origins, so that's all from me on this.

It's worth noting that Python 3.0 supports function annotations, a feature that can be used to implement type checking in Python. What's more PEP 3107 explicitly states that one of the expected use cases is "providing typing information" and "type checking".

http://www.python.org/dev/peps/pep-3107/

I think the real argument for testing vs. types is that for most jobs "programmer productivity" has to do with producing code that works in practice rather than code that is perfect under all possible circumstances. Most of us are not developing satellites, nuclear power plants, or pacemakers. Premature optimization for robustness probably wastes more programmer-hours than premature optimization for speed.

That said, if you haven't used something like ML or Haskell you can't possibly imagine how many subtle bugs can be caught by a type checker, particularly edge and corner cases that will eventually dominate your test suite - in the unlikely event that your code lives that long.

I admire Bruce Eckel for seeing limitations of the language he invested lot of time and effort in (he is the author of Thinking in Java). Many would stick to a field they feel comfortable in, but Bruce had a courage and an open mind to move forward. He actually advocated Ruby and Python during his talks on Java conferences!

A great classic. I basically started learning Python because Bruce Eckel recommended it. And I'm never tired of recommending "Thinking in Java" as a must-read for all programmers, regardless of whether they aspire to become Java programmers or not :)

Even though I spent a lot of my childhood programming in C, I never understood why people started strong typing was so necessary when Ruby started becoming popular. This article explains that thinking for me, finally.

It's funny that it's the same article that also explains why it's wrong. At least, now it is.

Unit testing not only replaces the need for strong typing, it does so without any extra effort on top of what you'd already do for unit testing. It's not like you have to do extra work on duck-typed languages than you do on strong-typed languages when you're unit testing. (Assuming you're doing it correctly, of course.)

How do you ensure that the unit tests are correct?

Unit tests are "correct" by definition, because they represent the specification for the code that they test.

You aim to encapsulate all the intended behaviour in your tests.

Thus, when you encounter a bug, you start by creating a test that replicates the bug, correct the test to the expected behaviour (so it now fails), then fix the code. That is: red, green, refactor.

Not only do you fix the bug, but you improve the specification and improve the efficacy of your tests.

In the way that you cannot pragmatically prove that code has no bugs, neither can you prove your tests are correct (bug-free). But by having a test suite that can run continuously, and that you add to, you are increasingly improving code quality and, most importantly, ensuring that regressions don't occur.

> Unit tests are "correct" by definition, because they represent the specification for the code that they test.

No, those are acceptance tests. (It's really too bad they can't be the code. Talk about DRY!)

Besides, how does one write tests for bugs they don't know about? (Or can they only be written after the fact?) Wouldn't it be helpful to have some sort of declarative specification so that the compiler itself can generate exhaustive tests?

Unit testing not only replaces the need for strong typing, it does so without any extra effort on top of what you'd already do for unit testing

That's not true. I work on an e-commerce system. Ensuring that people don't accidentally use Floats where they shouldn't requires quite a bit of additional effort which could be avoided with a type system.

Exactly! And that's just the kind of things the computer should be doing for you, not the other way around.

If you are able to invest some time I highly recommend you take a look at Haskell or OCaml.

I used to think strong typing was something stupid, inconvenient, etc. Then I started learning Haskell and realized I didn't hate strong typing, I just hated sucky implementations of it (Java, C++, I'm looking at you...).

And Haskell is an eye opener in more ways then just the strong typing. This being said I do agree non-strong typed languages can work perfectly fine, Python is for now still my goto language until I get more Haskell experience.

> "[I] realized I didn't hate strong typing, I just hated sucky implementations of it[.]"

Same here (but Lua, not Python). I didn't get the point of static typing until I learned OCaml - the static typing in C, C++, Java, etc. is a joke. (Especially C!) Type inference also singlehandedly removes one of the biggest problems with common static typing - structuring code so that types reflect assumptions (and changing them with the code) is no longer incredibly verbose.

I think the most pragmatic approach is probably dynamic typing with support for static annotations and as much inference as is feasible, though. (As with Common Lisp or Erlang's dialyzer.) Static and dynamic typing is more of a continuum than an absolute fork.

Also, minor point - you mean "static" typing, not "strong" typing. While this is a bit of an oversimplification, static vs. dynamic means compile-time-checked vs. runtime-checked, while strong vs weak means "has enforced typechecks" vs "passing around void pointers and casting freely".

I disagree with Eckel on this one. The reality is that most of your tests end up being a matter of testing the things that the compiler would have done for you for free and that's the kind of thing that computers should be doing, not people.

Also, tests don't have 100% coverage whereas the compiler will enforce type correctness everywhere.

I've spent days hunting down bugs in Javascript which would have been automatically caught by a type checking compiler. That's just a waste of my time, as is reproducing all the type checking that the compiler would have done in the first place.

Reply on news.ycombinator.com