Hacker News Reader

Comment by howling | Hacker News Reader

The article elaborates on this point a bit:

> Like pipes in UNIX, the functions are clever enough to be performant when composed together–we don’t traverse the whole list and generate a new list each time, each item is generated on demand. In fact, due to stream fusion, the code will be compiled into one fast loop.

I imagine it is hard to do the same optimization in a non-lazy language.

View on HN

It's interesting that he says you also need purity to do the optimization, but Unix pipes are not necessarily pure.

Unix pipes don't do stream fusion, which is the optimisation to which he was referring.

So if I put (x1, x2...) through pipes f and g, instead of calculating all the f(x)s first followed by all the g(f(x))s, it calculates g(f(x1)), then g(f(x2)), etc.

How is that order of operations different to stream fusion? Why does stream fusion require more purity than Unix pipes?

Let's say there's a global mutable variable M (doesn't have to global, can be x1.M = M, x2.M = M, for example, but easier to think with global) which is used by and modified when f(x) or g(y) runs. Then, the order of operations would matter and so you get different results with and without stream fusion.

Right, but couldn't the same thing happen with Unix pipes if the components write to a global file?

You can do the same optimization Unix pipes does without language enforced purity, just with a caveat that functions sharing state might behave weirdly when fused.

You could... I guess language designers generally avoid those types of optimizations since it can bring about unexpected results. As in, if syntactically the code is using one order of operations and semantically the code is using another order of operations, even if it's documented, lots of programmers would stumble into difficult to find bugs and be surprised by this behavior. Or worse, just get wrong results and never even find out it's wrong.

Reply on news.ycombinator.com