Hacker News Reader

>If you can't keep the "types" of your design in your head, your design might be broken.

Either that, or you might not work on toy projects.

Granted it's difficult to remember the type of the casual e-mail address or of a list of integers. In Haskell, where there are a thousand ways of encoding the most trivial things, that is.

In my own practice, most transformations don't involve relations over more than 5 variables with mostly obvious types. Yes, this is hard to use:

https://tronche.com/gui/x/xlib/window/XCreateWindow.html

But then 1) there might be nicer ways to design an API and 2) at these few places the implementor of the API can just insert value assertions.

>Granted it's difficult to remember the type of the casual e-mail address or of a list of integers.

It would be a toy project that only (or mainly) has those as all the types you need to remember -- so that just proves my point.

Also the 'type' of an email address is an unsolved problem -- if we're talking type checking. It's not a string, and it's not solvable with regex either. So bad example all around.

This is the point. Encode email addresses as strings and be done. Not a problem in practice.

Less aggressively, you may also be inheriting someone else's design.

You cannot do any kind of Domain Driven Design with dynamic typing. In F# you can have a type email address and it will be impossible to pass an unverified string there. It has to be a valid email address, and you will know it at compile time. Good luck to try to achieve the same effect in python without writing tons and tons of useless tests just to check the type. And if you can keep all the type of your system in your head I don't think you ever worked in a real system.

You could make a property in Python that validates it's given value and throws a ValueError if it's not an email. 6 lines of code max, with maybe two tests.

How can this prevent you at compile time to pass a username instead of an email?

I have gone that route in Haskell and it was a mess. In the end you need to extract the string again (basically everytime you do something meaningful with it). All these pattern matches kill code readability/writeability.

Also, in Haskell one needs to encapsulate the real constructor in its own module to make sure the data is only constructed through a sanitizing constructor. That's quite some overhead (and makes pattern matching impossible).

What might be useful instead is first class "tags". In Perl6, you can subset types to values that match a condition (for example, strings that match an email regex (cough)). They are still strings so can be used unchanged with other functionality that does not need the tag. It seemed quite usable to me (applications are limited, though). However the creator said it's still expensive because they haven't figured out / implemented how to transfer invariants, so they are checked at each function boundary.

In Python, you can use duck typing: subclass str, it's a dirty hack but works very nicely.

The real world solution to the whole mess seems to me to be just to design a good dataflow into the system, so it's clear where you have a valid email address. Typically it's very easy: Just assert the invariant at the boundary to the external domain (for example, HTML form parsing). Now whenever there is a variable "email" in your program you "know" that it is a valid email address.

In python nothing can guarantee at compile time that you are not passing a username variable in a method that accepts an email address. You'll just notice at runtime when you realise that you have to rollback your last deployment because new users are unable to receive the activation email that is being sent to their username instead of their email.

> In python nothing can guarantee at compile time that you are not passing a username variable in a method that accepts an email address

Is there no reason you could not create an Email class and have the method require its argument to be of that type?

> guarantee at compile time

If you’re talking about runtime type checks rather than mypy, well, they’re just verbose static types with less safety. (May as well just embrace the duck typing at that point by accessing email.local_part and letting the AttributeError propagate.)

I’ve gone that route in Haskell and it works great. Yes, you need to extract the string again. Not an issue unless you’re golfing all your code. You also don’t need to hide the real constructor if it’s a type your module should be working with.

Subclassing str seems like the most horrible way ever to represent an e-mail address. I would definitely write a wrapper instead – exactly like Haskell.

You’re making out having to explicitly specify a type when type inference doesn’t do a satisfactory job as some kind of extremely painful thing, but it’s not. You just define the type of your object, which is something you had to know anyways; now it’s written down and people reading don’t have to guess at or remember what it is.

Annotating even the last bit of code that uses type class methods until the error message becomes clear is very cumbersome. For example, I can't say (traverse :: The List instance), but have to dig out the whole annoying signature, or go look for a more strategic value where the same information can be put in fewer characters. For another thing, it often duplicates the amount of lines needed (or more if you like to put in whitespace before types). It's definitely very annoying.

> I can't say (traverse :: The List instance)

Actually, in GHC 8.0 you can do exactly that, with the TypeApplictions extension.

    Prelude> :set -XTypeApplications
    Prelude> :t traverse @[]
    traverse @[] :: Applicative f => (a -> f b) -> [a] -> f [b]

I'd be interested to hear more about Haskell projects where the type system causes bad designs. Maybe you could write a blog post about it or something?

I will think about it, but would probably not be comfortable with it.

For now a few words that I hope I won't regret later (apart from the things I mentioned): The bulk of invariants / assertions in many real-life programs can't be expressed with mainstream type systems at all. They are too specific or only come to life dynamically. Things like: This array of integers is sorted ascendingly, and that index is a valid index into this other array which points to a number between 48 and 58. Or this table's column over there contains only values that are also present in that other table that just happens to be present now (but must be expunged from working memory after the next subtask). Checking if an integer fits in 8 bits and then converting to a byte is no safer in Haskell than in any other language.

So much to put the usefulness of mainstream types for static correctness in relation.

Now much of the tinkering that is going on in Haskell projects seems to be shoehorning yet one more real-world invariant in another crazy type. Maybe the next brilliant idea works for one more toy case in the end. Great, but the cost/benefit ratio is just bad.

I think the most important thing in software architecture is modularity, because the most pressing need in software development is the need for change. If my project is very modular, it's sufficient to have clear entry points to each module with very simple data types in the signature. The internals can be a dirty hack and rewritten later, or the module thrown away when the idea turns out bad. Interfaces on the other hand can't be changed as easily since they affect much more code. So a good approach is making interfaces that accept tables, augmented by a textual description of invariants so the user easily "gets" the meaning of the data and can reshape/preprocess the data as needed. This enables isolated rewrites of modules. It also enables creative use of data in unforeseen ways. Clever types on the other hand have a very strong influence not only on the design of module interfaces but also on the dependent modules' implementations, because 1) it's more cumbersome to reshape complex structures in a way suitable for clean implementation, and 2) because type systems can't usually transfer invariants to reshaped representations (and after all the goal was to let the compiler handle the invariants, so reshaping is a no-go).

In short, complex types _kill_ modularity because they have far too much influence on data layout.

For real world programs, what Haskell is great at is statically finding trivial interface usage errors like, don't call this with a NULL pointer, or this parameter must be a functor instance (because the actual data might be too ugly to admit). Turns out that that's all nice and well, but the machinery is a huge cognitive and economic overhead, while running some python client code of a sane API once typically brings such usage errors to light just as well (profane things. TypeError raised. Seems it tried to iterate over an int, so you must have gotten the argument order wrong etc).

>For now a few words that I hope I won't regret later (apart from the things I mentioned): The bulk of invariants / assertions in many real-life programs can't be expressed with mainstream type systems at all.

Which is beside the point, since partial coverage of invariants is still very useful.

For when it's not, there's Idra, Coq and such.

Having a small subset of invariants checked by the type system for sure is beneficial, however I was saying that the cost/benefit ratio is just not good enough (IMO of course: as a non Haskell expert, who is not a genius and has only spent 2-3 years learning it...)

> This array of integers is sorted ascendingly, and that index is a valid index into this other array which points to a number between 48 and 58

These are difficult to encode in a way that lets the compiler check the code that sorts or calculates an index.

They are trivial to encode in a way that simply tags the values. I still need to be concerned with making sure my sorting is correct, but that's a very isolated concern. I still find it very useful to know whether I've called a function on the array that depends on it being sorted, and that doesn't require any heavy lifting at all.

I haven't had good experiences with tagging. There are just too many "tags" (often one per line or group of lines).

They also have disadvantages in that they separate data which is only different by some aspects, but not most others. (For a simple example, the printing function is not interested in a "safe HTML" Tag. The effort needed for constant tagging and untagging (or using "Stringy" classes, or the cognitive overhead you need to pay if you found a clever way to remove the syntactical burden) is too big except in very few cases.

> I haven't had good experiences with tagging.

Fair enough. I have.

> There are just too many "tags" (often one per line or group of lines).

Where the burden outweighs the benefit, you always have the freedom of typing things too loosely - to be sure, without something like refinement types, there is a granularity you can easily try for that's just not worthwhile. But that's a pretty different thing than "can't be expressed with mainstream type systems at all." For me, the questions are 1) how far does this type move information in its use through the program, and 2) what part of these are likely to change in ways such that I'll want to be notified about this piece of code here?

> Looking at C++, a practical example is vector. It's a useful tool (automatic memory management), and very likely to be all over in your function signatures. This is very bad in terms of modularity / reusability, because your functions are not defined using reasonably general types (pointers to arrays).

That's why you define your functions as taking generic Iterators instead, which generalize pointers to arrays. Or, in more modern style, generic containers.

You can wrap that in a type erasure layer (lookup any_iterator) if you want separate compilation.

> That's why you define your functions as taking generic Iterators instead, which generalize pointers to arrays. Or, in more modern style, generic containers.

Or, in yet more modern style as slices. Or, ...

The only thing I miss from pointers + indices is optional bounds checking. Otherwise, it's the perfect tool for productivity (because readability, writeability, portability, modularity).

Pointer+size only allow passing memory-contiguous containers.

> Most of my code now is structured as tables, which greatly simplifies many transformations. There is no pressing need for typing tables, and anyway most of these type systems really suck at "tables transformation algebra".

What do you mean by "tables"? I think of a database table of rows/columns, or a matrix.

Exactly. Arrays of C structures. Python lists of (named?) tuples. Mathematically, simple relations.

It helps if the language supports creation of indices / reshaping (table joins, filters, making hierarchies).