Why is learning functional programming so damned hard? (2019)
cscalfani.medium.com
cscalfani.medium.com
These uncommon FP frameworks and languages can be good tools in the right, experienced hands. They can also be fun for side projects and as learning exercises. But every time I’ve watched programmers try to use uncommon functional languages for real projects they end up like this article:
> More accurately, learning Functional Programming concepts used in Haskell in 3 months after having thrown out 30,000 lines of code on a project that was now monumentally behind schedule was the hardest thing I had to do in my career.
When you’ve reached the point of being severely behind schedule, throwing out mountains of code along the way, and struggling mightily just to get basic things accomplished: It’s time to stop. Don’t double down on a new language that you also have to learn from scratch. Pick something tried and true and get the work done. Revisit the functional language at a later time for an unimportant project or a side project, not something with a deadline.
If the programming language or ideology has become more important than shipping the project, we’ve lost the point.
These two points are closely related, though.
Common tools will always have more available programmers, more documentation, more tutorials, more help, more libraries, more maturity.
We had a server written in a functional language at a company I worked at. It was fine, but when the two people who wrote it left the company it became a huge pain point to even hire someone to work on it. Consultants knew this and demanded exorbitant fees for basic work on the project.
Eventually we just rewrote it from scratch in a common language and saved a huge amount of time and money compared to trying to build teams and schedules around this obscure language.
Applicative programming is an older term for functional programming[1], but note this isn't pure functional programming like in Haskell; it's functional programming like in Scheme and Javascript.
[0] I can't find a pdf of the book, but it was mentioned here https://en.wikipedia.org/wiki/Futures_and_promises - the book calls it promises
[1] https://en.wikipedia.org/wiki/Applicative_programming_langua...
According to Wikipedia, Promises were, Futures (which are similar but not identical) were proposed in a 1977 paper. All of the closely related concepts of promises, futures, logic variables, and dataflow variables were used in functional languages first, and long before their use in, or the existence of, Twisted.
I use functional paradigms in mainstream languages. No need to be snarky or sarcastic.
That’s not what this article is about, though. Using Elm and Haskell for projects is not mainstream.
Haskell's got all that but I can't argue with it needing more programmers.
The fork wouldn’t necessarily need to be public either.
Forking a framework and trying to maintain a new, separate open source project is a huge burden. There is no way that would have fixed their problems of being behind schedule, but it definitely would have permanently worsened their maintenance overhead.
They needed to scrap the alternate language/framework plans and return to something safe and proven as soon as it became obvious that they were too far off track. Continuing to double down on commitments to unpopular frameworks (or worse, creating your own niche fork to maintain) would only worsen the problem.
I've found that people who go out of their way to learn things like that would be more productive regardless of the language itself.
> stuff like deferring side effects and avoiding state
one reason i asked is because while i don't really come from an FP background (as a student, i started with c/obj-c), after working in the field it was always appaling at how many variables and global state people would put in their code and how many side-effects were present... its just something that seemed so obvious even in "oop-land" but maybe im just an outlier...
Pattern matching, for example, is not itself 'functional programming', and never has been. It's a feature common in functional languages because it compliments algebraic types.
OO/Imperative languages (C# and Rust) getting pattern matching is useful but doesn't make them functional languages. C# and Rust are compositional in the sense types may implement interfaces (or traits), but with varying degrees of power. However, C# can't have HKT's until some work on the CLR is done, Rust is much closer.
Programmers (particularly web programmers) these days don't really consider the computational cost of their actions, so pure functions with immutable data types are now possible, despite the unnecessary allocation. Almost all popular OO architectures now are some variant of 'functional core, imperative shell'. There is definitely a 'functional shift'.
It makes me a little sad tbh, to see OO languages embrace FP principles. I just like writing tiny functions and types, gluing them together somehow (not how Haskell does it*), and building up to a bigger system. You can follow that approach in F# or OCaml, but it's not really possible in an OO language, regardless of how 'functional' it now is.
* Haskell is cool, but trying to explain to someone the difference between `.`, `$`, `<|>`, `|>`, `<$>`, `>>=` and more is quite painful.
For those who aren't familiar, I'll explain:
First of all, these are all infix operators, meaning they take two parameters: one before the symbol, and one after. You already know many infix operators: +, -, %, etc. I'll be surrounding them in parentheses, as that's idiomatic when they're not being used in the infix position.
(.) is compose: run one function, then feed its result into the other.
($) is just a tool for avoiding parentheses. It means "wrap everything after this in a set of parens".
(<|>) is alternative. Try one computation that can fail. If it doesn't work, try the other.
(|>) is either snoc (the opposite of cons) or pipe--as in bash--depending on what you have imported.
(<$>) is the general form of map, called fmap in Haskell (since map is just for lists). Given a function and a value inside a container, return the function applied to the value, inside the container.
(>>=) ah, bind. One half of the interface to the famously difficult monad. It's really not that hard, conceptually: run a computation, then use the result of that to run another computation. You might say "that sounds like compose!" and you'd be right. The difference is that a "computation" (or "action", or whatever your local monad tutorial calls it) is a function in a context. That context can be "it might not exist", which is called Maybe, or "there are a lot of values in order", which is called List, or "it can do side effectful IO", which is called, well, IO. If you want to compose those kinds of computations, you need to also "compose" their contexts as well. The implentation of that composition varies from context to context, but the interace is the same: (>>=), or bind.
Of course, conceptually is the easy part. This is the one operator in your list that can be a little difficult to gain an intuition for.
So learning functional languages is both learning the language, but to also improve your cognitive capabilities to have more thoughts.
I’m sure this can be observed with any profession. A good cook can work with a cheap knife and mediocre ingredients, but give them a nice set of professional kitchen tools and fresh, tasty vegetables and they will happily cook you a meal that makes your evening.
Oof, it may indeed be the second, but what do they get out of the deal?
A lot of FP programmers refuse to go back, simply because it feels nicer.
It's a hell of a lot easier to "persevere" on a project when your language doesn't make you want to dig your eyes out with a spoon.
Yes if you're running a C++ shop and someone leaves behind a small program in Haskell, you're probably better off reimplementing it than becoming or hiring a Haskell guru just to maintain that thing. That's a question of whether to learn Haskell, not how to learn it. In the case of the linked article, the author was the one who decided to get the company involved with FP, and gave his thoughts on whether it was worth it.
I can certainly support the idea that someone juggling all the plates required to keep a company running, has no time to embark on a deep and nerdy self-education project in something as abstruse as FP. Better to keep it at the level of a side interest or hobby until you're comfortable with it, before even thinking of doing anything important with it.
most "failed" FP projects I've seen mostly lack the former - often due to managers or engineers that come on the project after it first hits prod
then comes the rewrite to add a notch in said managers' and engineers' belts
then the original FPers leave and now there's systems in prod with little to no people who can work on it. that rewrite suddenly got a lot more business-critical!
the original FPers weren't as PM-savvy, so the rewrite is successful despite having less functionality and still taking a year or more. doesn't matter - the savvier managers and engineers know how to set goals & milestones that they know they'll hit and can say that they hit when evaluating the project's success
This is just a summary of my years of experience as an FP professional. I've seen this happen 3+ times across 3+ companies.
---
Haskell is by far the most worth-it skill I've cultivated. It wasn't easy but I really do do a 10x job on my personal projects in large part due to it. It's just not a good language to use in a corporate setting. It's better to get paid big bucks to be less productive. Save the technical brainpower for things you yourself find valuable :)
(and I sneak Haskell in all my jobs anyways - scripting, for instance - so I still develop my Haskell knowledge on company time & dime, despite the language being all-but-banned by the higher-ups)
(Granted, the most ported and maintained software I know of is either Linux or SCUMM.)
You combine the potential for side effects in the code with a tree that can't easily be changed because of how many other things it will negatively affect and you end up with a code base that is harder and harder to change the larger it gets.
In a FP approach, this is a class of problems that you can't easily replicate. Yes there's a learning curve to operating in a FP style, but once you're able to move quickly with it the long term rewards are a byproduct.
The OO solution to avoiding this problem is to build with microservices instead, which forces different parts of the system to be isolated and minimizes the negative effects from entanglement...but microservices come with their own maintenance and speed of development headaches as well.
I would love to know that there is a panacea that automatically improves productivity. Unfortunately, I haven't seen one in my career, as yet.
I am certainly more happy in many functional languages. And I can see happiness contributing to some. But I can't count that as data, without RCTs or something similar.
I'm not going to go so far as to claim that computer science isn't science. But I do fear that there are more claims than there are checks in our industry.
I suspect you've seen many of them. They're just the ones that are so normal you don't notice anymore. For instance, no one does pure waterfall-style project management anymore. "goto" programming has completely dissappeared. Etc.
I absolutely do not agree.
FP binds your particular choice of implementation to the code architecture. FP actually makes refactoring in the small trivial and refactoring in the large ferociously difficult.
A good example would be a program that runs fine suddenly now needs a "timeout" on an operation. FP implementations now need to thread the notion of time from somewhere near the top of the implementation the whole way down the chain to the function that needs "time".
This is painful.
An imperative programmer throws in a global time variable, possibly a local timeout variable and gets on with life.
Now, if I'm trying to manage a high-complexity codebase, I will probably eat the FP penalty. Having complete determinism, decoupling, and visibility in, say, a network stack makes debugging possible that will be very difficult with sorta-state smeared across a bunch of variables at various level of hierarchy.
However, I accept and acknowledge that I am making a tradeoff.
I mean, 'FP' is so broad a term that a lot of languages fall under that, almost all of which can introduce the concept of a timeout quite easily. E.g. OCaml's Lwt, or Scala's ZIO are two that quickly come to mind.
This is not at all my experience, in fact I'd say I encountered the opposite. Large scale refactoring in C++ and python were nightmarish and are a breeze in Julia and Elixir.
Which actually destroys the ability to refactor in the large. You have no idea what might use that global time variable from where or why. Choosing to use FP is choosing to ban yourself from taking on that kind of tech debt (although we should note that most FP languages have an "escape hatch" if you need it - even in Haskell you can always unsafePerformIO). There are definitely times where you want to do that, but I don't think it's right to frame that as the non-functional language making things easier to refactor in the large - rather the non-functional language makes it easier to not refactor in the large because you don't fully decouple things in the large in the first place.
A sufficiently complicated FP codebase almost certainly has some form of effects management in place (monad transformers or algebraic effects, for instance), so adding a timeout is as simple as adding the effect to the type signature at the top level and then letting the compiler tell you all the places you need to wire it up. I've done this in many codebases. It's actually a dream, as the type system won't allow you to make a mistake.
That may or may not be true, but most programmers don't and aren't really all that interested in learning. Most know what they are doing in a handful of Algol-derived mostly-imperative OO-ish languages and aren't comfortable goony much afield from that.
There is a fundamental mindset here that the Functional languages aren't task-first. Not in the languages, mind, it is in the community. And the tasks themselves can be so simple once the abstractions are under control that people don't write tutorials about the task.
It is visible in this article - the Haskell community is trying to convince this guy that he needs to know a wall of jargon from a relatively obscure branch of maths in order to program effectively. They may or may not be correct, but it is going to be a while before his attention makes it to the task at hand.
This seems to be the core problem in the communities, like you said: a tendency to believe that time spent learning the language or learning concepts or fighting with libraries or struggling with documentation doesn’t actually “count” towards time spent getting the job done.
I think this plays into our tendency to view learning and education as investments rather than costs. That may be true on a personal level when using your own time, but using your employer’s time to experiment with difficult new concepts and languages when you have a deadline approaching (or long since past) is not cool.
As kindly as I can put it: I’d argue that within this sentiment are the foundations of everything wrong with the ecosystem of professional software engineering.
Let’s start with, if you don’t let people learn on the job, you have to have at least 20% annual turnover to be only 5 years behind the knowledge curve.
At 10% turnover, you’re 10 years behind. Managers think people shouldn’t be learning on company time, HR tells the board lower turnover is a goal.
Inevitable next step is pausing all productive work to undertake a “Digital Transformation™” to try to replace 20 years of old tech. But the firm won’t know how.
> As kindly as I can put it: I’d argue that within this sentiment are the foundations of everything wrong with the ecosystem of professional software engineering.
Almost all Software Engineering, as practiced, is not a form of engineering in any way. Many times Software Engineering is really a bunch of commodity workers, who are learning, assembling commodity components that don't really work, under the oversight of a more senior developer who helps everything not fall apart.
https://www.hillelwayne.com/post/are-we-really-engineers/
The upshot is that "engineering" isn't the paragon of excellence we like to think it is. Whether or not you agree, it's a good read!
Learning in the job is great, within reason.
Abandoning all of your existing experience and trying to write new projects in a completely unfamiliar language with zero prior experience is not reasonable, though.
Learning isn’t a binary yes/no feature of a job. There is ample room for learning without allowing reckless decisions like trying to use Elm for server-side code when even the Elm authors are hostile to such a use case.
You may feel different, which is ok, but don’t bring made up numbers into it.
It's in fact very cool. I get to take those skills with me when I leave in a few years :)
To defend the counterpoint, artificial scarcity is abundant in the industry. Deadlines always approach, projects are already late. If employees don’t learn and improve on the job, employers will be happy to replace them.
Well... if you learn them on this project, they don't count for the next project. So in that sense, if it levels up your group, then you only pay the price once, but you reap the benefit for a long time.
For purposes of any one project, though... the time counts as overhead. It needs to pay off, or it's a waste.
Boss makes a dollar, I make a dime, that's why I learn zygohistomorphic prepromorphisms on company time.
A lot of people put the cart before the horse - the whole point of using a monad or whatever is to let you write clear, straight-through code with plain functions and values, if you can do that without using a monad then that's even better. But I don't think that's an FP problem per se so much as a hammer-nail mentality or, worse, a mentality where if someone's spent a lot of time and effort learning something then they want to prove that it was worthwhile by forcing everyone else to put the same effort in.
Switching fresh to FP: Now you have two nemeses.
However, it's not a silver bullet. I found that basic software engineering principles are way more important than the language. I've seen extremely messy OCaml code and super clean C code. What is important is how the code is organized at high level. Whether you use a for loop or a fold, an error monad or an exception mechanism matters less.
I'm also wary of functional programming gurus that tend to over-abstract things and use all the language latest features, making code very hard to read.
Also, when developing in a niche language, you tend to miss important tools and need to rely on unstable third-party libraries.
I used to be quite enthusiastic about FP, but I think I'd stick with more mainstream languages unless there's a good reason not to.
Mariners use "port" and "starboard" for vehicle relative directions not because it's "cool" but because it's clear and unambiguous. Problem decomposition works the same way: some problems decompose better with FP (I would argue databases do) some with OOP, some with EF etc. The more you know the clearer code you can write and the faster you can extract the theory from other people's code.
“You want me to turn right, right?”
After that Port/Starboard would make sense and I'd be chucking someone over one of them.
b) With the possible exception of "forward", if you hear one of these words you always know it's specifying a direction relative to the direction the boat is pointing, which is not true for "left" and "right".
Lots of other jargon too for other directions or places (leeward, windward, amidships, athwartships, etc) but that is all much less common than the four listed above.
Also, there's a fun additional distinction for forward and aft, which is that they tend to refer to locations within the vessel. Ahead and astern are used to refer to locations outside of the vessel.
After about 8 years of sailing fairly regularly with various crews, I still get many of the terms mixed up. For instance, stanchions and shrowds are terms that are useful, but they never seen to come to mind when I need to communicate them.
For instance if you're facing port and I ask you to take two steps forward, you could either take two steps to port or two steps towards the bow. If I tell you to take two steps to port then the direction is unambiguous.
At the intersection of Main and Oak, go two blocks east. The building will be on the north side of the street.
Those instructions are the same no matter what direction the person is facing or direction they came from.
However, on a ship, the absolute positions of NSEW (relative to the Earth) aren't as useful as the ship may be heading any direction (and may be going forward or reverse). So a different absolute coordinate system is use - fore, aft, port, starboard.
So "move forward" doesn't say much, whereas "toward the bow" e.g. does.
Same for left/right compared to port/startboard.
"Stage left" as if I'm standing looking towards the stage from the audience, or looking from the stage towards the audience?
I left the cheese on the left side
Agreed, developers are great at finding new ways to write questionable code no matter what languages or programming paradigms claim about their respective abilities to reduce code errors and smells. I know because I'm one of them.
Wanted to echo this point.
I'm a huge FP proponent, and I do believe that liberal use FP paradigms can result in code that's in many ways measurably more expressive and less complex than code that make liberal use of imperative looping, in-place mutation, and inheritance, etc. When I'm coding on my own for side projects, I always strive to use the most elegant and expressive FP constructs I can think of.
However, at the end of the day, most of the coding we do is part of a social activity. And in a social setting, familiarity matters.
Writing slightly less expressive, more complex code in a way that most people who read it will be already familiar with can result in more productivity in aggregate than writing code that's super expressive and simple but requires everyone reading it to first learn the paradigms/patterns that enabled the code to be written that way, especially in a fast growing startup where people are constantly onboarding into the codebase.
Optimizing too much for familiarity can also become detrimental though, in that it results in stagnation and closes the doors to paradigms/patterns that can have an outsized impact on overall productivity compared to their cost in learning/education.
The real challenge is striking a good balance between familiarity and expressiveness in the choice of paradigms/patterns to use.
This was often forgotten about OOP as well, as Alan Kay himself reminds us...
"The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be."
The issue is not that FP is hard to learn. The issue is that most people start off from a much lower baseline knowledge-wise than they will when learning a new imperative and/or OOP language, framework, etc.
I come from the OOP school and have done 10s-100s(?) of C/C++ projects, which I like, btw. But when I discovered FP it was an eye opener, the breadth of things that became possible/tractable is great, even my OOP programs became much better because of it.
I think that the OOP curriculum is actually a regression, and would recommend anyone who is just starting out to try out FP first.
It is certainly possible I’m not looking in the right place for the right thing, but it should probably be an area of improvement for those communities in the future IMO.
It appears to exclusively work with the usual FP understanding of computation as reduction (simplification of an expression), avoiding all named mutable state. This is (in my opinion) the most important and fundamental difference between functional and imperative, and is something you miss if you try to learn FP in, say, JavaScript.
It also doesn't seem to assume any prior experience with programming. The last chapter even has a section specifically directed at students who aren't computer scientists or software developers.
The authors of this book are some of the major folks behind the Racket programming language. Mattias Felleisen, Matthew Flatt, Robert Findler, Shriram Krishnamurthi.
You might recognize Shriram Krishnamurthi's name in the credits to PG's guide to Bel too [1]
I didn't really use this book much directly - when I took that class in 2012, I mostly learned via the lectures, lecture slides, and doing via the assignments. The book is probably sufficient and you can probably talk to other people about it via Racket subreddit, IRC, etc
[1]: https://student.cs.uwaterloo.ca/~cs135/cc/textbook/ [2]: https://sep.yimg.com/ty/cdn/paulgraham/bellanguage.txt?t=159...
As with anything in life, just don't give up. Everything sucks till it's finished :D
I think FP advocates (I'd count myself as one, sometimes) like to gloss over this aspect and instead blame the student for being too stupid, or for having been indoctrinated in the dark side too much.
FP I believe would be better received without the smugness surrounding "you're too dumb or too much of a dinosaur to understand". Be more like Dave Farley.
* ML - includes SML, OCaml, F#, Scala, and Rust
* Lisp - includes Common Lisp, Racket, Scheme, Clojure
* Actor Model - includes Erlang, Elixir, and Pony, as well as other languages that have actor model systems at the library level
You'll find endless sources of opinions on why the Haskell family is so hard to use well. My personal opinion is that most languages create abstractions with concrete types of problems in mind. Haskell created abstractions with other types of abstractions in mind. If you ask the question "what is a monad used for?", the average Haskell user isn't going to respond in any form about making side effects safer, because that's just one thing that they do...they're going to respond with other abstractions. And after 45 minutes of explanations of what they are, they still haven't yet gotten to the explanation of what you can do with it. And then when you finally understand what you can actually do with it, you have to confront the fact that they made an incredibly easy thing hard, just in case you might want to use it a different way.
I cant recommend this rant enough - https://existentialtype.wordpress.com/2011/05/01/of-course-m....
We've been using F# with great success for a few projects now. Our stack is super simple.
On the backend we use F# (on .NET with ASP.NET Core) + Postgres.
On the frontend we use F# (via Fable with React and the Feliz Bindings).
https://zaid-ajaj.github.io/Feliz/
And we have a huge shared library. Works like a charm.
Elixir is also really great IMO. The pipeline composition is a really nice model. If adoption grows more the tooling should step up, because it's lacking a little. The language server can't do things like rename a function, there isn't a complete TreeSitter parser either. I also have this fear with actor model that I'm inadvertantly leaving some process dangle somewhere, which in my experience is not unjustified.
Scala and ocaml suffer from the same reputation. SML didn't really make it out of the academia. F# is easier, but still has a bad rap from C#ers. Rust is making incredible efforts to be accessible, but the learning curve is still steep.
One could say haskell itself is also an offspring of the ML family.
In that family, the article author correctly identified that Elm is probably among the most accessible. Choosing a specific application domain enabled the language creators to cut a lot of complexity, and to use a simple state machine as a runtime.
> If you ask the question "what is a monad used for?"
You'll get the same kind of answers as you'd get you'd get when asking Java programers what this "class" concept is about. The functor-applicative-monad stack is at the heart of Haskell's flavor of functional programming. You can write small programs without it, of course, but it's going to be the same experience as writing Java with a single "main" class.
As far as I know, they both have a reputation for being practical, pragmatic languages that enable imperative, OOP, and FP styles.
> F# is easier, but still has a bad rap from C#ers.
Yes, and C gets a bad rap from Assembly programmers.
Rust's primary influence was ML. The compiler was originally written in OCaml. In fact, Graydon Hoare has commented about how he preferred OCaml's module-based polymorphism, but the Haskell advocates would never shut up about type classes so he eventually relented on that one single idea and implemented traits. But he held on to OCaml's ideas for almost everything else, at least until the point where he resigned his BDFL position.
Haskell has some ML influence, but it received that influence via Miranda. Miranda deliberately diverged from ML in the operator emphasis, and execution (lazy vs eager). They both might have been considered at arm's length with the ML family right up until the point (1990ish) that Haskell decided to eliminate all unpure IO and adopt Monadic IO. There is almost no resemblance anymore, and the functional programming community is very cleanly divided: you're either in the ML camp or you're in the Haskell camp, unless you're one of the few Lisp weirdos sitting in the corner singing hippy songs.
The actor model does have a lot of roots in functional programming (as well as logic programming), but you're right that it is also related to modern OO programming insofar as most modern OO languages inherited a ton from Smalltalk, which was a similar message-passing model. However, unlike Smalltalk, the actor-oriented languages rely on immutable state and pure state transition functions. That makes them functional languages at least in some respects.
It wasn't until I discovered Erlang in 2012 (thanks, Seven Languages in Seven Weeks) that I finally found the motivation, aided in no small part by the fact that it's a very simple language and it's designed for server programming, where I've always been happiest.
I still haven't graduated into category theory or type theory. I still don't know the difference between a monad and a monoid. But functional programming really speaks to me, because I have an old, tired brain and I need pure functions wherever I can employ them to keep things straight.
Now whenever I see mutated variables and class attributes, or random side effects besides reading/writing to a database, it kinda makes me cringe and think it's a "oh here we go" into a rabbit hole just to understand what the code is doing. 9/10 the code does what it's supposed to, but the mutations and side effects makes understanding and extending it so much harder.
A monoid is just a collection of things that can be associatively "added". Think addition with integers, or append with lists.
Members of the dreaded monad can be sequenced, or composed, while taking into account their context. For instance: if I want to get a value from stdin, then use that value safely; or make a network request and then use the result safely; or run a function that can fail, and use it or short-circuit as needed.
Functional programming is hard because in a lot of cases, especially ones beginners encounter, the imperative solution is simpler. Purity and types are things I think you only truly appreciate when you’re writing large or complicated programs. I wasn’t able to really grasp FP (beyond using things like map() in Python) until I took a compilers course which used OCaml, and the ability to pass around and destruct these very complicated immutable trees was a very natural problem to tackle in the domain
If it's something really new and different, like Haskell is to an old school imperative programmer, I think the opposite approach is best. Treat it like a topic in math, start from zero, work out small problems to exercise the basic concepts, then start putting them together.
Immutable data structures are another new and shocking thing, but less complicated than fancy typed FP is. Start with seeing how you can "update" the first element of a linked list by making a new first element and linking it to the existing tail. Then see how AVL or red-black trees let you do something similar with tree nodes in log(N) time, so you can use those instead of hash tables without a monstrous slowdown. That's probably all you need, but the next thing after is probably Chris Okasaki's book Purely Functional Data Structures. It is pretty readable once you've seen some basics.
Which compilers course is it?
Additional roadblocks are that the syntax is strange if you're not familiar with ML-derived languages, the type system is fairly complex and you need to understand quite a bit of it to make progress, laziness can cause performance problems (lost parallelism, excessive memory use) if you're not careful, and the tooling isn't always as user friendly as it could be.
That said, I'm glad I learned Haskell, and though I don't know everything about the language, these days I'd feel pretty comfortable using it for anything I'd use any other decently-performing garbage collected high-level language for.
For example, the main Haskell graph library (fgl) has two type parameters, named 'a' and 'b'. Instead of meaningless letters, why not 'NodeLabel' and 'EdgeLabel'? Now it's obvious what they mean!
There's another general principle that the length of the names of variables should scale with the size of the context in which they're meaningful. So, a 10,000 line program might use a four-or-five syllable name, whereas a one-line function that takes two arbitrary arguments can just call them "x" and "y".
I do agree that points-free style and terse code can be problematic. Haskell lets you use very high levels of abstraction, but if the next person to look at your code doesn't understand those abstractions it'll tend to look like gibberish. I think my own preference tends to be not to go out of my way to make the program extra terse unless there's some compelling reason, like avoiding a lot of repetition.
I think plain old low-level C code is (sometimes) easy to read just because it has a lot of visual cues like "for" loops that your brain can pattern match on to tell you what the general control flow is. In higher level code, you often end up with fewer visual cues. Which is good in the sense that it eliminates redundancy, but bad in the sense that it can take longer to understand what unfamiliar code actually does.
Something that seems to happen in Haskell more than other languages I've used is that sometimes you can kind of forget how the lower layers of abstraction actually work, and it sort of just does what you want as if by magic. For example, the Lens library let's you update data structures using a syntax similar to imperative languages, but behind the scenes it's actually constructing a new data structure from the root on down to the "edited" leaf, and (unlike in imperative languages generally) if something goes wrong you can just throw up your hands and error out, and the old data structure is still there. It's like the convenience of imperative languages, but with transactions practically for free. But the type signatures that the Lens library uses are fantastically complicated. Though I sort of understand what the Lens library does, I really have no idea how it works internally. And I've decided that that's actually fine.
I used it a lot in the past, and less nowadays (Rust), but I occasionally have to teach my coworkers on basic FP principles so they can use and write good Python code. Simple descriptions make the learning more palatable and doesn’t scare people (“you could use a Monoid here” -> “Try using a class like this, with an `empty` and `combine` function, here”).
It usually takes about 6 months of daily practice to learn basic FP skills and 1-2 years to go from beginner to “intermediate” level. Occasionally, you might encounter a FP grandmaster who will melt your brain in less than 2 minutes of conversation, some things never change.
Once you have the bug, you start to pick-up code smell in anything that isn't functional. You see code and ask, "Why are we touching that?, Why are we holding onto this?" Being able to think in a functional style encourages you to throw away as much code as possible and hang-on to as few crufty elements of state as are minimally required.
It also makes you very anti-OOPS and hesitant to define "classes," since these violate the first principles of functional.
myFun : String -> Result String Int
myFun a =
do b <- fn1 a
c <- fn2 b
return c
[0] https://idris2.readthedocs.io/en/latest/tutorial/interfaces....
"If we have a fancy value and a function that takes a normal value but returns a fancy value, how do we feed that fancy value into the function?"
I'll put it in Rust terms:
let x: Option<u8> = Some(5);
fn foo(x: u8) -> Option<u8> {...}
Despite being an IBM Fellow (the company's highest rank with complete freedom to work on whatever you want), John was having trouble getting any traction for his ideas. I certainly didn't grok it at the time. I couldn't see the utility over the procedural PL/AS and imperative assembler we were already using to create the mainframe's higher-level language compilers.
I've since become a big believer in the functional style, sadly after John's passing. It's certainly not the solution for everything. Even the lambda calculus requires that you feed it a starting series of "magical" integers to work. But functional is a useful way of thinking about programming, especially for library functions.
I would say the key difference between the functional style and imperative/procedural is not the presence of recursion but the lack of state. A functional function cannot have any internal state store, nor rely on anyone else having one. In other words, all of its arguments and values must be fully defined by parameters. This is a super-critical concept in debugging because it helps bulletproof your function.
Having said that, no real working program can be fully bulletproofed with the functional style because we need to hold onto state in real programs. (Is the user logged in?, etc.) We cannot pass these values in every single time and have a practical program.
I think merging these concepts of functional when you can and state when you must is the easiest approach. Certainly there are many functions in every program which are functional in style in that they do not contain or rely on any state, and those are good jumping-off points for starting to understand the functional style.
The change from AngularJS to Lit (or React), is an example of this kind of functional refactoring. AngularJS had two-way databinding (state!) and it attempted to pass that state upward when things changed. This made horrible spaghetti and impractical large apps.
Lit and React are only top-down. Yes, each component has state but only at the top. It gets passed-in as a parameter to other things, but they can't change it in return. This is much more modular and debuggable.
Efficiency depends on how your js runtime is optimized. Elm works fine, for instance.
In other words, one can't really optimize for the top level (ease of use) and the bottom level (speed) at the same time.
Turing demonstrated that there are only two differences between any computers that have ever existed: how fast they are, and how difficult they are to program.
With that our code would be 100% pure, and all the side-effect part would live outside our JSX files, similar to how Haskell does.
> In computer science, an operation, function or expression is said to have a side effect if it modifies some state variable value(s) outside its local environment, that is to say has an observable effect besides returning a value (the intended effect) to the invoker of the operation.
I put quotes around runtime because it's not a runtime like an interpreter in dynamic languages, or a library that you can call from inside your pure program. This runtime just calls your main() function, which return a complex value (it's a chain of lambdas, to spoil it) that is interpreted by this runtime or "imperative shell".
I understand this sounds unintuitive and might seem very confusing, but this is what makes useful pure programs possible. These couple presentations show how this type of boundary between imperative and functional code works with simpler type of program [1]. The only difference is that Haskell's "imperative shell" is lower-level than the presenter's. It only deals with IO, etc, whereas the presenter's imperative shell also has some domain code.
[1] https://www.destroyallsoftware.com/talks/boundaries / https://www.destroyallsoftware.com/screencasts/catalog/funct...
But, higher order functions and type parameters are still a bit mind bending for me. I didn't have to write much code for those, but when I did it was really hard. I'm sure I'd need at least couple days of reading and playing around to somewhat grasp how to implement them again.
That being said I think once you get the basics of FP programming learning about FP type systems is worth it. Especially FP effect systems (i.e. IO monad).
It's mostly the type theory that is hard here but people talk about functional programming because of association thanks in large part to haskell.
Mon[a|oi]ds are everywhere, but in some languages you just don't know about it. Which of course lets you not have to learn those concepts knowingly, but they're still there and you still could be a better developer for at least knowing they're there and they're behind some of the repeating patterns you see (string concatenation / promises / etc.)
I believe "not worrying about them at first" is actually a good path to learn these patterns. Using them first, some time later having a thought "oh hey, the API of how JSON decoders are written is kinda similar to how I'm generating random values, that's cool" and only later on learning that the it is in fact monads/monoids/functors/whatever and that they have laws and you can exploit that (eg. [repeated squaring for anything that's a semigroup](https://blog.jle.im/entry/shuffling-things-up.html)) and whatnot.
People building their apps on a discouraged "leaked" implementation detail (JS native/kernel modules) got cut off from using it.
The reason for disallowing that implementation detail wasn't "hey, somebody is using it, let's teach them a lesson" but, as far as I've heard, improvements to dead code elimination.
You can (and people, me included, do) use Elm in production peacefully. Huge apps, nontrivial JavaScript interop needed. You can do all that without depending on JS native modules.
I feel like whether you use a (discouraged) implementation detail of the language is a good indicator of whether you'll have a bad time later on when that implementation detail changes ¯\_(ツ)_/¯
Sure, I agree that learning FP will make you a better development, but spending all these learning resources in order to build a product is not very convenient.
I've been shipping code with mediocre languages all my life. Even now, I routinely pick node.js over Haskell or Rust just because the complexity of the solution I need to build is not high enough to justify writing amazing bug-free code. Sure, you may get more bugs and less help from the compiler, but there is more material online and I can easily find a cheap developer to throw at the project.
I've been developing for 15 years. After 5 years of C++, PHP, JS, I decided to jump into Haskell for my side projects. I can't say the learning curve was as hard as you made it out to be: there are plenty of great resources for learning Haskell and you don't need to go through Elm or Purescript (which weren't even a thing when I learnt Haskell). Actually the differences between the languages may make things more confusing.
In the last couple of years I stopped using Haskell completely, simply because Rust (a language designed to build things, unlike Haskell which is more of a language research project) is functional enough, pleasant enough to use and is developing a nice community. The most useful FP concepts trickled down in other languages.
FP already won and nobody even noticed it.
https://www.haskellforall.com/2012/08/the-category-design-pa...
This is also good:
I like your sense of humor.
Appeal to Wikipedia Fallacy: https://en.wikipedia.org/wiki/Functional_programming
LISP certainly shows up in the discussion. It's even called the first functional programming language!
Functional programming seems to be extremely easy if you have never learned imperative programming first. I have seen beginners grasp FP much faster than OOP and write production ready code only after a few weeks/months of learning whereas beginners need on average multiple years to learn "production ready" design pattern style OOP.
On the other hand I have observed some of the best OOP developers really struggle with FP. It's not that they find FP hard to learn, they find it really hard to unlearn OOP and the thinking that the way things are done in OOP is the holy grail of good software design.
For example, only recently there was this blog post trending on HN (Am I stuck in a local maximum - https://blog.ploeh.dk/2021/08/09/am-i-stuck-in-a-local-maxim...), which was triggered by a "blue tick" OOP programmer (tastapod on Twitter - inventor of BDD) making false statements about FP because he seemingly struggled learning it and wasn't able to work out how to program without mutations. He came to the conclusion that all functional programmers actually use mutations by default and immutable data structures are not common in FP at all. It was a completely unfounded assertion and clearly one made from frustration by someone who was so hard wired into OOP programming that they couldn't adapt to the FP way of thinking. It was a prime example of an "old dog" (citing the original article) finding FP harder than the new guys.
Once you learn how to use them, their utility and benefit (they explicitly limit the scope of the changes in the loop and reduce mental overhead) become clear. But a lot of people never get past "why can't I have a for loop" and don't get there.
It also seemed to map harder onto the real world than IP, where instructions about piecemeal alterations seem to be the way of the world.
It really clicked for me when I started to think in terms of transformations, where each function performs a transformation of the input on the way to some output. This allows you to be 'pure' most of the way and to limit input and output to the top layers of the program, where they should be (should in my opinion).
I’ve worked on systems that run through a chain of background workers. Each job had a complete list of operations (one per worker) to preform. When each worker finished, it posted the job back to the general queue with the new state and one less operation to preform.
All programs are eventually sequenced. You can’t work on data that doesn’t exist yet.
I’m pretty sure I don’t lack the ability to understand what your talking about. I am sure I don’t know what the words you are using mean.
They are especially uneasy about the concept of lazy evaluation. It goes against everything they know about programming - that you write a line of code, it's executed, and you move on to the next line of code. With lazy evaluation (as in Haskell) it's an uphill battle getting them comfortable with the idea of writing a line of code that will be executed at some unspecified point in the future. For many students, this can be a mind-bending realization.
IF you can compress all your javascript into one line of code or as close to it as much as possible then you are absolutely doing functional programming.
That is essentially what functional programming is, how to program so everything goes on one line. You can think of it as expression based programming, or how to compress your entire program into a single expression!
Now, when you see multiline functional code, what's actually happening is that the programmer is giving parts of his expression a name and placing it on another line so that the code is more readable or the programmer could be generalizing logic in the expression for reuse in other places. Example:
addOneTime8PlusTwoMinusThreeTimes300 = (x) => (x + 1 * 8) + 2 - 3 * 300
addOneTimes8 = function(x) => x + 1 * 8
plusTwoMinusThreeTimes300 = function(x) => 2 - 3 * 300
addOneTime8PlusTwoMinusThreeTimes300 = (x) => plusTwoMinusThreeTimes300(addOneTimes8(x))
Now that being said there's a lot of this going on in functional programming:
addOneTime8PlusTwoMinusThreeTimes300 = function(x) => Times300(MinusThree(PlusTwo(Times8(addOne(x)))))
addOneTime8PlusTwoMinusThreeTimes300 = function(x) => ( addOne | Times8 | PlusTwo | MinusThree )(x)
f | y = function(x) => f(y(x))
( f | a | b | x )(x)
This type of thing is called function composition!
A monad is just a special type of composition Not only do I want to compose all the functions but at each step of the composition I want to do an extra thing! So let's say I want to log the output
So I define
f | y = (x) => {
result = y(x)
print(result)
return f(result)
}
( f | a | b | x )(x)
That is essentially one type of monad. A monad is a way to compose functions such that they do an extra thing! And this intuition probably takes you 85% of the way there on how to use monads in haskell. Monads in haskell just have some extra rules but the intuition is 100% the same thing.
Now you will note that I cheated for f | y. I wrote the code on multiple lines! That is exactly what "sequential" code is!
It is the fundamental property of reality that is at odds with functional programming. Haskell is trying to get rid code that requires you to write things on multiple lines! It is trying to abstract all of that away with a bunch of crazy abstractions so all your code can fit beautifully onto a single line! It is in fact impossible to write the multi line code I wrote above in haskell. What haskell does is present to you the IO monad as an API so you print things through composition and you never have to write "sequential" lines of code.
Turns out when you do single line coding a whole class of errors disappears and your code is also far more modular. It's hard to convince you of the benefits with just words. If you want to know more, you have to walk the path, I can only show you the way.
One more thing. When code is written this way the compiler can do much more tricks with it. Because state is immutable the compiler can execute code in a different order to achieve the same result as your intention. You don't have to often think about this when programming in haskell but it does allow you to do certain tricks.
# cat data.csv | process-1 | process-2 | ... | process-n > final-output
Imagine you have this pipeline that already works for data.csv. But now you have data2.csv which has some difference (e.g., some values are null, while the original data.csv had no null values).
Monads are an approach to making the existing pipeline work (with minimal changes) while still being able to handle both data.csv and data2.csv. The minimal changes follow a strict rule as follows (this is not a valid shell command anymore):
# wrap(cat data.csv) ] process-1 ] process-2 ] ... ] process-n > final-output
In other words, only two kinds of changes are allowed:
- You can bring in a wrap function, that modifies the entries of the given csv data.
- You can bring in a new kind of pipe ']' instead of '|'
The idea being, the wrap function takes in original data stream, and for each "unit" (a line in the csv file, called a value) produces a new kind of data-unit (called monadic-value). Then your new pipe ']' has some additional functionality that is aware of the new kind of data-unit and is able to, e.g., process the null values, while leaving the non-null values unchanged.
Note, you didn't have to modify any of the process-1 through process-n commands.
BTW, the null value handling monad is called the 'maybe monad' (and of course there are other kinds of monads).
If you make the existing pipeline work in this way, you essentially created a monad to solve your problem (monad here is the new mechanism consisting of the new value, and the two new changes, the wrap function, and the new pipe).
edit: There may be a need to also modify the '>' mechanism. But I think that is not essential to the idea of a monad, since you could replace ">" with "] process-n+1 >" (i.e., you created a new outermost function 'process-n+1' that simply converts the monadic-values back to regular values).
edit 2: If instead of handling null-values, the purpose is to "create side-effects" e.g., at every pipe invocation, dump all/some contents of the data into a log file, then the kind of monad you end up creating would be something like an "I/O monad".
Try this instead, simply put monads are used to provide an easier to use API to some black box abstraction. Example monads can include a "List" or a "Class"
Why does Haskell etc need this? because its hard for them to make an easier to use API to access the internals of some abstraction due to the strict type system unless they use the monad pattern. In comparison in untyped FP everything is transparent while OOP allows you to create your own API within the abstraction itself.
Funnily enough from this thread you can see all sorts of wrong ideas about Monads that beginner Haskellers have
Mainly because we are monkeys to whom a reasonably accurate reductive analysis can be applied if we label the axes "asshole" and "idiot".
It was a good blog post by a reasonably intelligent monkey who seemed to score low on the "idiot" axis. But some other monkey pegged the "asshole" axis by preventing us from highlighting the text so we could right-click and search to answer the primary question, "What is it?"
Namely, what is functional programming? Even after besting the right-click hurdle, no one easy and succinct answer is available.
The meta-analysis of this blog post therefore circles back around to perform a second-phase adjustment of the author's "idiot" score once we realize that he has done exactly what he is complaining about; he is a bad teacher because he did not answer the "What is it?" question.
He seems like a reasonably nice person, so I think we have to leave his "asshole" score alone in the second phase.
The collective effort, however, scores high on both the "idiot" and "asshole" axes, and this is the core of the "bad teacher" problem.
I spent most of my long university career angry about this problem. Why is is that monkeys can't teach?
Partly, it is because they are arrogant assholes who don't want to weaken their position by making it easy for others to access their expertise.
And partly it is because they are monkeys who think more highly of themselves than is justified, and therefore they cover their ignorance with jargon, hand-waving, and obfuscation.
Other than extinction, I'm not sure how to solve the problem, but I'm sure that the first step is by starting every lesson by answering the "What is it?" question succinctly and concretely, and, more importantly, realizing that if you can't do that, you should shut the fuck up and go away.
For this reason alone I think this is why fellow students tend to be much better at tutoring/helping each other instead of their professors.
https://thevaluable.dev/expert-blind-spot-software-developme...
https://nscblog.com/continual-learning/understanding-underst...
Dropping the judgements, assuming positive intent and trying to see the situation might help.
They describe pure functions and categories of endofunctors, while I have tasks like "invoke this stateful external API if that stateful external API returns specific values".
Personally I think it is a matter of what teacher one happens to bump into.
I was quite lucky to have such set of teachers for logic programming (Tarsis World and Prolog), and Functional Programming (Lisp, Miranda, Caml Light).
It felt no different form other programming classes.
In fact, it had higher success grades than thermodynamics, electromagnetism physics or the most feared of all assignments, data structures and algorithms.
So dammed hard depends very much on the learning path.
Paradigms are modes of thinking. You can't just pick up a new paradigm on the fly when you've spent your entire life in another one. Some individuals are exceptions, but most of us aren't so lucky as to have such a natural aptitude for changing our minds on the fly.
In order to be introduced to a new paradigm you have a couple options:
1. Sink or swim. In Haskell, this is the monad tutorial. Why the hell do people start their instruction here? Did they start here? If they did, did they succeed from that point or did they have to find another path and just forgot that this was a really stupid way to start?
2. Baby steps. "See Spot. See Spot run. Run Spot, run!" Learning Italian (previously Spanish), this is literally the level I'm at (actually, a bit better, but still highly constrained by my limited vocabulary). In Haskell, this is:
double :: Int -> Int
double x = 2 * x
quadruple :: Int -> Int
quadruple x = double (double x)
square_function f = f . f
quadruple = square_function double
By analogy (hah!), we don't start C language learners with implementing a generic swap or sort function. That would be way beyond their initial capability, relying no too many ideas that they have no foundation for (that said, it's a shorter path to that in C than monads in Haskell).
So why do people think that learning a totally novel (to them) paradigm like functional programming, especially in the uber-FP language Haskell, can be done by starting at the deep end without studying its fundamentals?
Just to give you a sense of the issue in typed FP, and I will take Haskell as the prime example, the average Haskeller will not be able what a Monad is beyond the standard definition in category theory. I tried it once, got the whole Slack channel explode as everyone tries to tell me "I will just get it". This is the result of my study. https://medium.com/glassblade/pragmatic-monad-understanding-...
IMO the main problem is that typed FP langs don't try to connect their abstractions to the mainstream languages when these abstractions clearly have some counterpart or an intuitive close cousin in typed OOP. Worse, these abstractions usually are gaping holes in the language design but the typed FP community thinks its a feature.
Sorry for the cheap pun, but I think the main difficulty is getting along with recursion.
And corecursion, of course ;).
This seems like a lot more work than just learning how to think functionally.
Those that had programmed before kept trying to make Haskell behave how they thought a programming language 'should' behave, while those who had never programmed just looked at Haskell and went "I guess this is what programming is" and just rolled with it.
The only problem was that once they moved onto the intermediate course, taught in Java, they where completely lost again.
The trajectory for students is Python to C to SML (15-110 to 15-122 to 15-150). Some people take the C and SML courses in the same semester.
I was really happy to have been introduced to many different paradigms over the course of a few years. But to both your points, once we moved on to Java I had a really bad time.
It seems like a very convoluted way to do a simple thing ( sometimes).
the fact that FP works over a model of computation (aka an abstraction over the underlying hardware limits) makes it a bit harder to visualize its behavior in the bounded real-world environment ( imho ).
The more i progress the more i want to develop with "visible wires", and reduce the amount of abstraction to its most basic components..
The traditional FP model of computation doesn't have a call stack. It proceeds by reduction: replace one subexpression with another, in the program text itself, and repeat until you can't do it anymore.
So if I've got a fibonacci function in two clauses, `fib 0 a b = b` and `fib n a b = fib (n - 1) b (a + b)`, then a reduction may look like this:
fib 3 0 1 -- initial program
= fib 2 1 (0 + 1) -- substitute second clause of `fib`
= fib 2 1 1 -- evaluate `+`
= fib 1 1 (1 + 1)
= fib 1 1 2
= fib 0 2 (1 + 2)
= fib 0 2 3
= 3 -- substitute first clause of `fib`
This isn't just an analogy; this is the way computation works in an FP. To an FP system, the call stack is the invisible, behind-the-scenes optimization.
Also, mutual recursion is a bit more general than loops :)
Multiple recursion, like in `fib n = fib (n - 1) + fib (n - 2)`, is more general than loops... in a world without higher-order functions. You can make such a function tail-recursive by doing the continuation-passing transform, which basically just makes the stack explicit. (You'd then want to "defunctionalize the continuation" [0] to clean up.)
(I've actually done this in Java! Writing the multiply-recursive solution is sometimes a lot easier to do (and verify); transforming it into an iterative solution mechanically exposes a lot more hidden details, but you still have your recursive solution you can test and compare against.)
[0] http://www.pathsensitive.com/2019/07/the-best-refactoring-yo...
Otherwise there’s no law of computer science making the two very different. Some language compilers implement recursion naively, that’s mostly it.
The call stack itself is literally just a stack provided by the OS with restrictions meant to protect the other programs in the os.
The problem is that the idea of recursion as slow or resource hungry has become a kind of circular just-so story among imperative programmers and compiler developers who don't want to write recursion optimizations (or don't even know how).
The argument goes that recursion is slow, so don't use recursion, and if you raise the idea of optimizing recursion, the compiler dev dismisses it, because why would you write recursion support, recursion is slow.
Now it's fair to say in an earlier era, this was somewhat justifiable, but the logic has been passed down far longer than it's actually been true of the technology available. I remember reading "recursion bad" in C books in the 90s, and it's still so pervasive that even though TCO has been in the JS standard since ES6 years ago, most browsers still can't be bothered to support it.
In the case of recursion, the "time" of computation may have a direct consequence, in itself, on the occupied memory (in the case of non-tail optimized calls)
Another one is the use of higher-level functions and function composition.
Another one is the use of monads (at least in pure functional languages like Haskell).
You see things like functors/monads all the time even in regular imperative languages. For example in .NET we have Linq, Nullable types, etc... They aren't as well defined as in Haskell but still an incredibly useful pattern.
You also have higher ordered functions and whatnot in imperative languages now. In fact, in JavaScript and .NET they are used all over the place.
While .NET allows for composition with delegates, it’s not very smart, and results in unnecessary type information erasure; for example in Linq if you take a projection of a fixed-size list, then that length information is lost if you then add another step after the projection. (Linq internally has some ugly, but inconsistent implementation hacks; for example, IList<T> length information may be preserved but IReadOnlyCollection<T> length information will not.
That's my point, you find these patterns all over in common programming languages and even though they are hard to learn it's worth it.
In fact I think this was the easiest way for me to learn monads - to learn them in the context of things I already understood.
I think this SO post covers it well: https://stackoverflow.com/questions/44965/what-is-a-monad
Yes, I know List and Maybe are also technically monads in Haskell. But you don't need to think of them as monads or even understand monads to do anything useful with them. Applying monad operators to lists and maybes is only occasionally useful.
In most impure functional languages (LISP, ML) you rarely model any data types as monads explicitly. As soon as you have side-effects, they lose their appeal.
Also ST and State have no side effects. The only monad in Haskell that has side effects is IO (unless you were to sneak in an unsafePerformIO but that is an entirely different discussion). So they aren't there to work around the limitations of purity, because they are pure.
Namely?
> Also ST and State have no side effects.
I didn't say they do. I said they are useful only because Haskell doesn't allow side-effects. In ML or Lisp you'd just update a local variable instead.
ST & State are far more useful than updating a local variable. If you are going to choose the simplest use case for them ever and then say that is the only thing they can do it is not a productive discussion. The state monad is a lot like a context object that allows state to flow through function composition (for functions that have no knowledge of such state). For example you could compose a bunch of functions and have the results of each function call stored without changing any of the functions to know about the operation. It's a method of abstraction and encapsulation.
It's not just monads that are useful either, it is the full hierarchy of Monoids, functors, applicatives, monads, etc...
meh, recursion is easy. Recursion is mainstream. It _began_ as a way to fake a loop without mutation, but I struggle now to remember a mainstream language without it.
The hardest part is all the new patterns you have to grok, it's like learning programming again from a blank slate, because it kinda is. It doesn't get any easier when Haskell decides it needs 7 new syntaxes and someone rushes to compose them all in one line.
I understand recursion just fine. I can look at a problem and think "this problem space is correctly represented by a tree/graph" and can meaningfully translate that into a program that shoves the data into a tree/graph-like data structure, and my natural inclination when computing on a tree/graph-like data structure is to use recursion.
On the other hand, I understand that not all problems lend themselves to recursive solutions. I can look at a problem and think "this problem space is correctly represented by a matrix" and the reality is that linear algebra problems are often iterative, not recursive, in nature. It is certainly possible to do the simplex algorithm recursively with immutable data, but it's just so much straightforward to do it iteratively on a mutable matrix.
The other thing is that most of my day job is just hooking one API up to another API to achieve my business's desired outcome. I don't need "interesting" algorithmic solutions for 95% of what I do. I just... extract data from one API and plug the data into another. You don't need recursion for that, and the "list of steps to do in order and (possibly) loop over that" paradigm with whichever seasonings you wish to add to it is just fine. (OOP is the flavor of the ~~day~~century at my company)
edit: I forgot the bit that ties it all together. I am absolutely no good at functional programming. I've tried to sit down and learn Haskell or Lisp a dozen times and always failed. I've bought books. I'm still no good. It just takes me ten times as much code that's completely unreadable to do something that would be simple and straightforward in C++.
all "objects" are immutable (or by value), as are the functions arguments and return values.
But within a function you can still use variables and for loops.
Often I think, I could have solved the problem so much faster if I had just stuck to Python or Javascript, or in the language the library was written in. In the end, you end up swimming upstream. This includes the effort of handing it over to someone else.
Elm tries hard to provide an impedance-mismatch free environment for UI development, until you reach a point where you want to do something just slightly different; then it becomes painful. Sure you can integrate Elm with JS using ports or even React with Elm, but creating a polished UX experience is a different level of effort.
Is there a better name for the domain the author is talking about? "Type-driven functional programming"?
They are required when taking the original meaning of functional programming though (nowadays often called "pure functional programming" to differentiate it).
The rise of effect handling systems like monads in functional programming languages since the oughties was driven by the real-world need to, well, print 'Hello World' to the screen. And different practitioners have different opinions on how far to take effect management. Haskell is just one extreme, but there are several FP languages on the spectrum.
Lisp is not a FP language anymore than C.
Most Lisps are multiparadigm, covering at least the functional and procedural paradigms. Going to Common Lisp you can add OO to the mix. It is a severely constrained notion of FP that manages to exclude the first functional programming language from its definition.
It's also worth mentioning that the Common Lisp standard does not require implementations to implement tail call elimination. The pervasive use of a functional style in CL would therefore give rise to programs with performance characteristics that could, at least in principle, vary greatly between implementations. (I'm aware that most popular CL implementations do optimize away tail calls in practice.)
Common Lisp doesn't require implementations to provide tail call support, so functional programs that express iteration using recursion may be severely limited in the inputs that they can handle; they would typically be ported to Common Lisp by a rewrite using iteration.
[0] https://nrinaudo.github.io/scala-best-practices/definitions/...
To answer it to my own satisfaction, I eventually arrived at a form of predicate logic so I could directly experience how cumbersome it was to try to express everything that way. I liken it to trying to speak in a language that lacks a definite article: doable, but way more verbose.
What you're describing are regular functions. If those functions hold a state, they're not functional; they're procedural. For example, a function that holds onto a counter and increments it by some value that is passed in cannot be functional because the counter exists in a hidden state, unknown to anyone else and unpredictable until runtime. This is what Backus was trying to fix.
A functional version of that same function would need two parameters, one for the amount to increment by and a second one for the counter's current value. Often, a function can be rewritten in the functional style and thereby eliminate state (at least from that function).
Whew!
> when your program is (tail-)recursive you can carry the "current" value along without ever really mutating it
main :: IO ()
main = loop (0 :: Int) where
loop n = do
putStrLn $ "The counter is currently " <> show n <> ". Increase it by how much?"
increase <- readLn
loop $ n + increase
The lambda calculus is a way to define math in the functional style. Guess what? It needs state passed in to work. The names and orders of the integers must be passed in and held in a "state" in order to do anything useful. Since functional programming is centered around the concepts from the lambda calculus, this implies that all functional programs must have some "state" somewhere in order to start or do useful work.
Only components or individual functions can be functional in style, the goal being to have as many of those as practical so that you don't touch state often and hopefully, only in a few places and using methods which set the state (rather than letting individual components "reach in" and set it themselves).
However, I often write performance-critical CPU-bound code, often impossible to do with FP. To write fast code for modern processors, programmers have to think about RAM layout and L1D cache misses. With these immutable data structures, FP code ain’t exceptionally fast because for optimal performance it’s essential to reuse the L1D-cached lines of virtual address space. Also, better to avoid any heap allocations on hot paths. Much easier to achieve these two things with imperative code, with their mutable variables, native stack and for loops.
Using FP introduces real advantage in terms of taste and simplicity, meaning that "advanced" concepts are not nearly as prevalent as someone who just learned them might hope. The rule of 3 is helpful and under-applied. Programmers new to FP get eager to use cool tech as opposed to leverage improved taste.
FP can be utilized in many languages but in ones that don't guide your hand toward it—your Haskell, your OCaml, your Elm—it's easy to have it "mix" with other styles. It is not the case that combining FP and non-FP styles immediately make sense or work. It is the case that the strengths can be combined if done thoughtfully.
All of these points generalize, though. As with any programming work, taste is important. It takes a while to develop and often needs to be developed within the context of a team. Tasteless FP is an awful, awful waste of time, energy, money.
Someone who likes to throw all the jargon at your is a hobbyist, a proselytizer, or a fan. Not terrible, but not necessarily someone who can yet manage all of the necessary tradeoffs and balances. We take that sense of taste somewhat for granted in "mainstream" programming styles.
The two books I read and can recommend are Programming Haskell by Graham Hutton & Get programming with Haskell by Will Kurt.
Another source that helped me a lot is the video series Category Theory by Bartosz Milewski [0]. It requires basically no background on category theory and defines all the concepts along the way. Bartosz also uses Haskell code as examples. You only need to know the syntax up to type class to understand.
As for the awful experience people get from learning Haskell, I think it depends a lot on people's expectations.
Most people only learn one language from school or early career, and then migrate their experience from one language to another. You don't learn the concept of "stack" twice, you only "learn" how to express them with a different syntax. So people can "learn" new languages with significantly lower effort than they learned the first language.
And when they find Haskell or other ML-family functional programming languages, they can't find the corresponding concepts in their known language. This time people actually need to learn new concepts as they learned their first language.
The experience is very similar to learning algorithms or OOP design patterns. And I don't think it's harder to learn Haskell up to the level mentioned by the author than e.g. reading the gang-of-four book.
[0]: https://youtube.com/playlist?list=PLbgaMIhjbmEnaH_LTkxLI7FMa...
Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.When you can use either technology on a per-function or class basis, it really helps with adoption. Languages like C#8+ almost feel like cheating when it comes to mixing ideologies. Some developers think this sucks, but I view these complaints similar to those who would complain that someone might try to use a hammer to install a screw.
The general theme in my mind is to use the imperative code to construct the walls of the functional matrix wherein I author the actual business logic as pure functions.
FP for me is usually in the form of SQL, LINQ or switch expressions. I personally have never wanted to go all the way with FP. I see how you can do it in theory, but I question the practical engineering value of actually pursuing this. The most important & scary parts of our product are within this "hosted" functional scope.
I took Martin Odersky's Scala MOOC sometime around 2012-2014 and it was easy. It just made sense and the IDE experience was nice. I would recommend it to anyone who uses C# or Java and wants to learn FP. Although after 8 years I don't know if it's still as good, or if something better has come along. Either way, the course is still available: https://www.coursera.org/learn/scala-functional-programming
Next I worked through SICP and that was the best CS book I've read, although significantly more difficult than the Scala course.
https://www.youtube.com/watch?v=Vgu82wiiZ90
Observations:
1> The syntax is strange, I'm not sure it needs to be, a list of parameters then the return type, with none of the usual commas and parens, it's like someone heard a critique of lisp, and went too far the other way. It's almost as bad as Forth
2> lazy evaluation as a feature? ok, I guess. Is there somewhere else that over-eager evaluation happens? (like in Metamine, where you can do a super fancy equals, and if any of the terms change, the whole thing is re-evaluated, like cells in a spreadsheet) That would balance it out
3> Everything is an expression - this is the worst part of C ever, and it's being called a feature
4> Invariants as good - this is the worst part of python, and it's being called a feature
5> procedural code is banned completely - yeah, this likely breaks a lot of brains, but I'm willing to let it slide if it works in the end
6> definitions of functions being conditional, that one is also a brain breaker... in the normal world, you define a function once, and forever... the 3 different styles of spreading out/pattern matching the inputs is a bit of a hill to climb
7> having to worry about tail recursion instead of regular recursion, it's the metaphorical equivalent of manual memory management, it ads mental complexity that the programmer really shouldn't have to deal with.
Aside/Tangent - Perhaps the difficulties being encountered are why John McCarthy wanted to keep S expressions hidden under the hood in his unrealized programming language.
Regarding 3/"everything is an expression": C does explicitly not have this property, it has statements and expressions. The typical example to illustrate this point is the lack of a true "if expression" in C. [1] Given the following C code:
int x;
if (a > 5) {
x = 1;
} else {
x = 2;
}
let x = if a < 5 { return 1 } else { return 2 };
Regarding 4: Do you mean immutability instead of invariants? I'm not quite sure what you are referring to in the context of Python.
Regarding 7: This was mentioned somewhere else as well, but in a functional programming language you don't have to worry about the compiler recognizing and optimizing tail recursion. The only exception are pathological (extremely and obviously inefficient) function definitions, for example functions that try sum an infinite list of integers and don't terminate or functions that reverse a long list.
[1] Terminology: "if" in C is a statement, and the ternary operator (a?b:c) is an expression similar to "if", but there is no true "if" expression.
Re: 4 - Immutable strings is the python oddity to which I referred
Re: 7 - The video (#4) in the series seemed to indicate that it was almost a requirement that any recursion be a tail recursion.
In my defense, I'd say the return keyword - combined with the different semicolon rules - is one of the more confusing and subtle syntactic differences when learning Rust from a C/C++ background. After a few weeks of programming C++ I always use return more often than necessary.
However, I believe that even after the learning curve has been scaled, the task of writing code in functional languages is objectively more difficult in at least one way: Coding a function in the expression-oriented syntax of functional languages has a higher cognitive load than coding that same computation as a sequence of statements. You simply have to hold more things in your head.
These concepts aren't hard to understand and are actually incredibly useful. At some point though things get so abstract it's really difficult to understand what's going on. I've seen Haskell libraries that do things I can't explain for reasons I don't understand and the extremely detailed README with diagrams and arrows and everything somehow made it even more complex.
A lot of that comes down to extremely advanced use of Haskell's type system. Advanced type systems are valuable, and they're easier to add to FP languages, but I wouldn't consider them to be a fundamentally FP concept. (We see very expressive type systems being included in imperative languages like Rust and Swift these days.)
I think it's useful to keep in mind that Haskell is used a lot for research. You don't see Java used as a research sandbox in the same way, so you're not likely to run into run into these kinds of libraries in Java. (Though, as an aside, I consider anything that heavily uses reflection in Java to be nearly as inscrutable.) It's actually impressive just how many of these research-level libraries are practically useful in some way, even if the bar to understanding them is higher.
I don't think it's a universal truth that libraries making advanced use of type systems must be difficult to understand. Taking the `lens` library as an example (and I think you were alluding to it, anyway), there's a fairly long-running thread of research around what a "lens" even is, and how they fit into a broader domain of "optics". The `lens` library was created based on one particular representation of lenses, and the community tried to find ways to unify the various kinds of optics in a way that made sense to them. I think the more recent understanding of "profunctor optics" might give us a more accessible route to understanding optics in general, separate from the whole hierarchy of distinct optics you can derive from it.
Interesting. That is exactly the opposite of the usual sales pitch for FP - that it simplifies things because you have to keep track of less.
I don't know what fp's you've been using but the most common modern ones have basically all adopted the pipe operator, which make the representation of data transformation as a "sequence of statements" painfully obvious.
"Take this data"
"Transform it with this function"
"Get this integer list out from that"
"Multiply those integers by two"
"Sum them up"
The pipe operator helps if you have a clean sequence of steps and each function returns only the needed input for the next step. The code becomes just as unwieldy if an argument isn't in the right position, so you have to splice in a lambda, or you have to split cases on the result, in which case you have to do something monadic if you want to avoid deeper nesting. All of which looks like unnecessary friction when I know perfectly well in my head what the computation needs to do after each step.
Also, when you have the pipe operator in your toolbox you now have a decision between two styles of coding that are basically inside-out from each other.
I disagree. This is pretty clear even though it does exactlywhat you claim is terrible, and it is far, far clearer than the corresponding imperative code:
https://twitter.com/DNAutics/status/1421465090856005633?s=19
Clarity in important when you're doing code reviews or fixing a bug, in anger.
As that guy was a big executive whose time was interchangeable with money, he might have had better luck treating FP as a topic in math that he was having trouble with, and hiring someone for one-on-one tutoring, either in person or online. It might have gotten him through the various stumbling blocks quicker.
My observations on why people think functional programming is hard:
1. They find it hard because they're not learning functional programming, instead, they're learning functional programming, advanced statically typed systems, type classes, and monadic programming altogether. Languages like Haskell fit into this category. It seems the Author took this path (Elm, PureScript, Haskell).
2. The ecosystem is simply not there. Say I just learned Python and there are many meaningful projects I can build to continuously improve my skill. Say I just learned SML, there are not many exciting things to be done.
3. The ecosystem favors power users. Clojure the language is not hard to learn. The ecosystem is not bad at all, both quality and quantity. But:
3.1 I agree that the idea of "libraries over frameworks" seen in Clojure community is better, but when there's less constraint, the responsibility falls on users' shoulders. Working with libraries feels like assemble my own PC. This is also why many people find Vue is easier to learn than React, and Intellij IDEA is easier than Emacs.
3.2 The ideas are novel. It may not be harder to learn, but it takes time. Rails programmer knows what to expect in a Web framework, be it Django or Spring Boot, so when they learn them they just learn like the last 20% of them. But for Clojure I find myself keep learning things like Fulcro and Pathom, the ideas are exciting but it's not ready for the mass. Just imagine how hard it would be to educating "the Algebraic Effects" to React users in 2013. In fact, the React ecosystem has been influenced by ClojureScript/Om, but the React community did pushed hard for the education part. At that time I feel like I came across those buzzword "functional" "reactive" "immutable" "undirectional" in the community every single day - it became such a cliche so that people start absorbing them.
Otherwise, functional programming without these problems is easy to learn. Rails programmers can be productive with Phoenix in a matter of days - many of them didn't even bother to learn Elixir or functional programming, they just picked them up along the way.
I found this rather bias. Is it true for those who has tried FP / elixir? I have never used it, I don't think my company will ever consider changing the codebase at all. However java / python as backend and typescript (react) as front end seems to scale soo easy for us here.
Don't get me wrong. I like Haskell. But yeah, I learned Haskell because I used to sit at a desk across from Brian O'Sullivan (who later wrote "Real World Haskell".) And even having Brian around to answer questions, it still required stretching of the brainpan. But at least I had a guide.
1. Complexity caused by the change of paradigm: This specifically hits programmers with a lot of experience in OOP languages. For getting into the FP-mindset your brain needs a bit of time to get rewired. You need to switch thinking about mutating objects to thinking in mapping streams of data. Given that many OOP languages are adopting functional ideas like map and fold/reduce, nowadays a lot of developers already have a bit of experience in thinking in the FP-way, and this will get better with time.
2. Complexity caused by the tools: There's a reason the author in the above post started out with Elm instead of Haskell :-) Writing a few recursive functions in haskell is still pretty simple. Where it get's very complex is when you want to build real world applications. To build a simple web app you need to 10s of decision on what tools and libraries to use. Here's a couple questions you'll need to find an answer to when building a haskell app:
- What GHC and what language extensions do I need? How do I install it? - What package manager do I use? Cabal, nix, stack? - What web server library? - What database library do I need? Do I need an ORM, are there even ORMs in haskell even though there are no objects? - What HTML template library to use? - How do I compose it all together? What is a monad stack?
When you have only very few experience in haskell it's really hard to not get stuck here. The quality of documentation of most haskell tools also doesn't very much help here.
I believe that the value haskell can bring is signifcant, and by fixing the tooling situation we can a lot more people to adopt haskell in the future.
With IHP we're trying to fix the tooling situation and build a haskell-based web framework that is as easy to use a rails or laravel :) To combine the benefits of purely-functional programming with the RAD approach of rails, laravel and django. It's now already the most active haskell web framework and we have many people starting their haskell journey with IHP.
We have live reloading in dev mode, a JSX-inspired template language and many code generators to quickly get started with shipping real world apps.
Check out this video of building a simple blog app in IHP: https://www.youtube.com/watch?v=UbDtS_mUMpI To get started (supports macOS, linux, windows) check out the IHP Website: https://ihp.digitallyinduced.com/
That's kind of the point of programming languages, otherwise you'd be writing 100% assembly.
And these abstractions are not hiding the model of the machine? For example the machine has a small set of fixed-size registers, but your language probably has an unlimited amount of arbitrarily sized, lexically-scoped local variables.
Or, one could just swallow ones pride, learn TypeScript, and have the benefit of working in a sane language that transpiles effortlessly into Javascript.
(Not a bad word about Javascript wizards or its inventor. Given the constraints the results are nothing but amazing. I just wish we didn't have to suffer an untyped language for so long before someone came up with the obvious solution that is TypeScript.)
I think a lot of programmers, including me, have a gift for the former kind of thinking but not the latter. So we tend to struggle with abstractions like monads, because we're looking for some literal ground truth about them that doesn't really exist.
Learning Haskell to the point where I could use it for practical work was one of the best things I ever did for my programming brain. I'd recommend it to anybody.
Most FP languages will let you write a recursive form that won't blow the stack. In C, you'd need to used closed form or a loop if you value your stack.
And if you're using memoization, you could do that in C.
And, even a naive implementation wouldn't blow your stack, it would just melt your CPU into a puddle of slag.
Or am I missing something here?
fib(0) -> 0;
fib(1) -> 1;
fib(N) when is_integer(N), N >= 2
-> fib(N, 2, fib(0) + fib(1), fib(1)).
fib(N, N, Current, _Last) -> Current;
fib(N, I, Current, Last) ->
fib(N, I + 1, Current + Last, Current).
fib(N) when ... -> fib(N - 2, fib(0) + fib(1), fib(1)).
Then you can check for N = 0 to return Current, and do N - 1 in the iteration. One less value to pass in each recursion should make it faster, and it's less to think about.
My solution (from another conversation on this very page) is:
fib 0 a b = b
fib n a b = fib (n - 1) b (a + b)
The most trouble I had with Haskell was pronouncing statements after reading them, which prevented me from composing the ideas in my head because the syntax doesn't give you a lot of hints. It's like trying to parse calculus without names for the letters in the greek alphabet. Most people are proficient in their own written languages, and they are capable of mapping ideas to abstractions, but if there is not a way to express those ideas lexically in their language, most are going to stumble.
The incomplete and error prone explanations by novices that the author refers to also create an artificial conceptual barrier, but mostly because there is always the hint and implication (or perhaps just personal anxiety) that you are not writing real functional programs and that your naivety is accumulating a hidden sunk cost that you will eventually be exposed for and made ashamed of because you aren't a category theorist, which is a consequence of this mostly aesthetic idea that functional code is about competing to advance knowledge in a science and not just making stuff you enjoy or people like. The clostest thing I found to a kind of function-punk where you could just bang away on it because it was fun was the excellent Learnyouahaskellforgreatgood, but it focused on the notation less than the underlying ontology - e.g. the instrument - and what I think I might need is more of a category-punk that provides the equivalent to composition with a few concepts, and then add the notation to those.
Not sure there's another attempt in me, but anyway, provocative article about something so important, and that I think ultimately FP will really advance the way people think about the world, if only in another generation or two.
fp and math go abstract fast, people can't see any tangible machine or pragmatic process
fp-ists enjoyed that and pushed it to 11, making it even weirder