Isn't this article just stating the obvious fact that programs at some point do I/O, so no program can be purely functional?
Every Haskell tutorial seems to introduce the IO Monad to illustrate this distinction.
Every Haskell tutorial seems to introduce the IO Monad to illustrate this distinction.
Unfortunately there are some roadblocks, like little information and conversation about the topic and no real mainstream implementation of these ideas.
I've been working on a system, that has is based on an immutable data layer and it is just a wonderful thing to work with, as long as you have enough disk space.
Where I think this could lead to something great is if the DB is alike this:
- Relational
- You have normal tables. Consider them your up-to-date cache. Most scenarios will be covered here (ie: The log recomputing is NOT at the app level, but at the DB level).
- You have a event log where everything is stored, plus your "tables" for the info you need up-to-date. But you don't need all to be a table if not make sense, and do full re-computation if desired.
- Your backup, sync and load history is around the event log.
- You don't even need a "table" if you wanna only a "index", this lead to me:
- You submit a data request (like a POST), and the DB convert it to the tables and log, but:
- You can instead (or also!) hook here and build secondary indexes and do other stuff. You can do it async, put the computation in a queue, do validations, etc as fit.
So, what is not here and need tooling is marry the "normal tables" + "event log" + "logic that route this", so putting:
Data -> WAL -> Router -> Event Log | Tables | Index | ping external tools
And
Request Data -> Router -> Get it from:Event Log | Tables | Index | ping external tools
I'm not affiliated, not even a user, it just sounded interesting when it was submitted here.
PipelineDB excels at workloads where you know the queries you want to run ahead of time, where your workload fits within the confines of SQL, and where there is a high degree of distillation (aggregations, sliding windows, etc.). We have large customers including Charter Cable / Time Warner Cable, MediaMath, SmartNews, MOAT Analytics, Cradlepoint Networks, Paddy Power Betfair, and others using the system at scale and we offer a clustered edition of PipelineDB under a commercial license, but PipelineDB's single server edition is open-source.
We have a live chat room here for technical questions:
It is great that you mention that 90% of the time you want "the last version". That was one of my discoveries with event sourcing as well.
So what I came to is that re-computation (foldl) is a nice generalization, but isn't very practical. Instead, index the events and build the state on demand by querying the indices. This way, [plurality enabling] domain modelling moves to the code (as opposed to the database) and allows for quick changes in how you see the information.
You can read more about it at https://blog.eventsourcing.com/why-use-eventsourcing-databas... and https://blog.eventsourcing.com/lazy-event-sourcing-ed7e59007... as well as other articles on that blog.
The classical example is how avoid duplicated data or do validation across tables. One of the requirements of my app is deny a sale if the customer have pending invoices. The tricky part with invoice is that entry must be fast (I'm competing against DOS apps that fly at data-entry).
So when ES say "Record the event, then build the read side at later time", instead me (and I think, many of us) wanna the opposite "Put into the event log after we have do validations".
Of course, some actions can be delayed, but some need be as-now. How do this elegantly?
My initial attempt is code against the tables and use triggers for recording the story, but that is not as nice...
In lazy event sourcing, you don't "build the read side at a later time", it's built upon receipt — in a form of indices, as they are THE read side and you pull this data and form domain objects on demand (by querying your indices), just transforming one representation (relevant events) into another (domain objects).
Does this help?
And how you build the indices?
I saw Rich Hickey's talk about Datomic recently and immediately thought it would be ideal as part of a distributed runtime. Without having actually programmed in Erlang, but having seen plenty of Joe Armstrong's talks, I understand that the point of replicating data in messages rather than passing around pointers is so that if a machine dies, the data isn't lost. But if you built a distributed runtime that used Datomic as a distributed heap on which you allocate large amounts of data which you expect to pass around a lot, you could instead pass around only pointers to the data and not worry about data loss, and save big time on message throughput. Does anybody know if something like this already exists?
What monads allow you to do is have a purely functional model to represent IO, among a great many other things. Haskell is, in a sense, the best imperative programming language ever created, because it lets you manipulate imperative statements in a purely functional manner. You can stick statements in data structures, compose them, replicate them, generate them and only evaluate them when you want to.
But that's not the goal. Functional programming is there to provide a better formalism for writing programs, it doesn't want to change the way the computers actually execute the programs (or how the systems eventually internally work).
That means more work for the compilers to figure out how to make the same old imperative program out of a functional one, which is a good thing (it's generally the direction the compilers evolve).
Benefits of functional programming are not in how programs are executed (quite to the contrary - it's actually more effort to make them as efficient as good imperative programs), but in how the programs are written by people. I think the article is a little confused about this.
For another example to see this, even notion of non-pure function goes already in this direction. Today, having lots of (small) functions seems to have negligible downsides, but historically this wasn't the case at all, look at any better program that had to fit into memory of an 8-bit computer. These were rarely written in this style, because compilers were simply not capable enough to produce a reasonably compact spaghetti machine code out of those (by means of inlining, for instance).
Let me return to that last example. Take a program written in procedural style, and an identical program written in a big ball of mud style (all variables are global, one big control flow). This is a discussion from like 1960s. It's harder to write for human, and harder for compiler to compile and make efficient program in procedural style.
Yet, I am sure today, you would prefer procedural style. It makes sense (trust me, I actually work with big-ball-of-mud programs written 30+ years ago at work, AMA). I don't care what work the compiler has to do, I care about my own time.
Functional programming is just another step on that road.
As a side note, I really think people who prefer FP should avoid (over)selling it as more computationally efficient approach, because it still isn't (even though it probably will be in the future). It's a little bit of optimistic dishonesty that IMHO hurts adoption of FP and feeds the misconception that I talk about.
But what you said is true too.
It's important to distinguish between imperative code and IO. Many pure algorithms are most naturally expressed imperatively. Viewed in this way, Haskell is actually pretty bad at imperative programming.
Myers Diff is an example. See diffST2 at https://hackage.haskell.org/package/uni-util-2.3.0.1/docs/sr... for the Haskell version. Notice how simple expressions in Myers's paper, such as `V[k−1] < V[k+1]`, explode into ugly things:
vkplus <- readArray v (k+1)
vkminus <- readArray v (k-1)
(x,l0) <- if vkminus < vkplus...
So in practice, Haskell is rather painful to use as an imperative programming language.
So you could write
case readIntArray# v (k +# 1#) realWorld# of (# vkp, _ #) ->
case readIntArray# v (k -# 1#) realWorld# of (# vkm, _ #) ->
case vkp <# vkm of
0# -> ...
1# -> ...
The problem is the state tokens; I ignored them by using realWorld# and _ above, but that makes the code not referentially transparent. So in practice, Haskell always passes them around in a monad (Except in all the core libraries, which use things like accursedUnutterablePerformIO to go back down to the thunk level: https://github.com/haskell/bytestring/blob/master/Data/ByteS...). But reads are (mostly) pure so it shouldn't break anything to provide a pure interface.
Alternatively, the verbosity can be seen as a syntax problem, with the corresponding solution of a better do-notation: https://www.reddit.com/r/haskell/comments/4dsb2b/thoughts_on...
so you'd get something like
(x,l0) <- if (<- v ! (k-1)) < (<- v ! (k+1)) ...
That's still not quite as good as C, it really needs the notion of sequence points: https://en.wikipedia.org/wiki/Sequence_point
But in terms of I/O with the outside world, monadic patterns become useful in Haskell because the type of your main function is also `IO ()`. That's the "escape hatch", not the use of monads in itself.
no program can be purely functional
By this I mean that a function that takes an Int and outputs an Int will have type `Int -> Int`, and a function that takes an Int, does logging, and outputs an Int will have type `Int -> IO Int`. Both are purely functional -- they're pure transformations from an Int to some other thing. For the first function that other thing is another Int, for the second that other thing is an action that can be performed by the machine which will return an Int.
So from the perspective that matters to us a developers programs can be purely functional.
(This becomes even more useful with something like PureScript's granular effect tracking https://github.com/purescript/purescript/wiki/Differences-fr...)
Do-notation shows that there's a trivial mapping between monadic code and imperative code. For all practical intents and purposes, things happening in the IO Monad are not 'functional', because your mental model needs to consider the mutations they represent.
What functional programming encourages you to do (as opposed to OOP) is segregate the pure aspects of your program, under the theory that pure code is easier to write, maintain, and test.
Classic OOP encourages modeling your problem as a collection of inherently stateful objects, making effect segregation more difficult.
Of course, you can write in a functional style in Java, or an imperative OOP style in Haskell, but the language isn't going to help you as much.
Pure object-orientation is a bad idea for other reasons (e.g., the binary method problem), but “inability to control effects” is totally bullshit.
Functional programming is a paradigm, and it emphasizes the segregation of effects. OOP traditionally does not.
(0) Functional programming is a paradigm that encourages expressing computation as expression evaluation, and discourages relying on more computational effects than are necessary to obtain a desired result.
(1) Object-oriented programming is a paradigm that discourages directly manipulating data structures from different parts of a program. Objects expose operations that act on a data structure hidden from the rest of the program.
(2) Effect segregation is a feature of some programming languages, notably Haskell and its descendants. An effect system classifies computations according to their possible effects, just like a type system classifies computations according to their possible results. You wouldn't call static typing a paradigm, right?
The fact still stands that it's closely aligned with the functional programming paradigm, and not OOP. Programing paradigms and language features are closely related: certain features make certain paradigms easier, and paradigms spur the development of particular features.
I agree with the point that a program can't be purely functional, but lately I've been wondering if you could still have a programming language that is purely functional (preferably also total). It is designed for writing data transformation libraries and needs to be called into by a stateful language that handles IO and orchestration. If you've seen Gary Bernhardt's talk Boundariers [1], this language would be the purely functional core.