A Secret Weapon for Startups – Functional Programming?
ramanan.com
ramanan.com
For scalability, I disagree with the author. Being functional does not necessarily make you scale better. The one hot backend paradigm that is easiest to scale is to write stateless, containerized microservices. Functional programming will nudge you in the right direction of writing stateless code but isn't strictly necessary.
It's kind of disappointing to hear functional programming cargo-culting from someone who admittedly doesn't really program much anymore. The software engineering industry already has enough resume driven development perpetuated by software engineers hyping things up to other software engineers
Microservices force you to reduce your service interfaces to lowest common denominator. You simply can't build a really powerful system out of microservices with a reasonable amount of code, because there's no way to send complex structures across service boundaries; you end up spending more effort serializing and deserializing than actually doing business logic.
Functional programming lets you achieve the same isolation benefits as microservices without the overhead.
>Functional programming lets you achieve the same isolation benefits as microservices without the overhead.
Isolation benefits, sure. Performance benefits, no, because you still run into https://en.wikipedia.org/wiki/Amdahl%27s_law
That's the whole problem I'm talking about. Like I said, it forces you to reduce all these interfaces to lowest common denominator.
> Besides, while I’m not a microservices enthusiast, isolation isn’t the main goal of microservices, it’s being able to deploy, scale, and operate services independently from other teams.
If you're just talking about each team having its own service, that's not what's usually meant by "microservices".
I don’t know what you mean by “lowest common denominator” or how “being forced to write good code” is inherently bad. There are lots of efficiencies to writing good code (easy to test, reuse, refactor, etc), not just writing microservices.
> If you're just talking about each team having its own service, that's not what's usually meant by "microservices".
That may not be the definition but it’s the principally cited advantage.
Honest question, have you ever actually worked with decently written micro-services?
You don't send complicated objects, you send relatively simple objects. And you don't do it by hand, grpc and thrift have existed for a long time. I'd argue that part of good decoupling, in functional or micro-services programming, is making sure you only communicate simple objects around.
As decently as is possible, yes.
> You don't send complicated objects, you send relatively simple objects.
Right, which limits how powerful the overall system you can build is. That was my whole point.
> And you don't do it by hand, grpc and thrift have existed for a long time.
Sure, but they have pretty anaemic type systems and a lot of stuff is just inherently high-effort to serialize (see the recent post about defunctionalising the continuation).
> I'd argue that part of good decoupling, in functional or micro-services programming, is making sure you only communicate simple objects around.
Functional programming gives you more tools to interleave complicated constructions while still enforcing that all your concerns are treated orthogonally. For example you can form a free monad structure that contains commands that will be interpreted by one particular service, but interleaved with other computations that will be implemented by other services or directly. The types ensure that everything's properly decoupled, but you can still represent all the complexity that's there in the actual problem. You can't do that with microservices.
What experience are you basing this on? It sounds like pure assumption, and doesn't at all match my experience of working at either a Scala or Clojure company, or running my businesses on Haskell.
But perhaps I'm incorrect, hence my original use of the qualifier "probably"
But I would tell them this is the simple and boring code I would run in production but probably not the smart code they want to see in the interview. Unfortunately smart code is often full of bugs and it costs more to deploy and maintain.
FWIW, I would have written your function as
f = bitraverse id id
Ah, it looks like you can even just use `bisequence`, which is equivalent to `bitraverse id id`. So you didn't actually need to implement anything!
solve (theInt, theString) =
do
a <- theInt
b <- theString
return (a,b)In someways functional programming has kind of ruined me as a developer: I would probably never take a job where I had to write in an imperative language full-time.
uncurry $ liftA2 (,)
bisequencefor {} yield seems to do the trick
scala> val a = Some("A")
a: Some[String] = Some(A)
scala> val b = Some(1)
b: Some[Int] = Some(1)
scala> val b2 = None
b2: None.type = None
scala> for { x <- a; y <- b } yield (x,y)
res1: Option[(String, Int)] = Some((A,1))
scala> for { x <- a; y <- b2 } yield (x,y)
res2: Option[(String, Nothing)] = None
scala> a zip b
res3: Iterable[(String, Int)] = List((A,1))
scala> a zip b2
res4: Iterable[(String, Nothing)] = List()I'm imagining maybe ITA Software said something like, "We're going develop a new node in the legacy IBM mainframe airline reservation system network, from scratch." And the responses were something like, "One, that sounds impossible; two, why would I want to get into legacy mainframe systems." Then ITA said, "We think it can be done with Lisp." "Where do I sign?"
Everybody knows they exist, everybody knows they are better in certain respects, most companies prefer standard languages where it is easy to recruit standard-level talent for standard wages.
As a full-time Scala developer (and long time Java guy) I've seen first-hand the benefits and drawbacks of both worlds: The highly populated world of Java libs and developers vs. the relatively niche Scala (but growing FAST!) community (relative to Java/Node.js/etc).
The learning curve on Scala (just like haskell) is quite high for most "standard" developers. However even given that we usually don't have much trouble getting people on-boarded and productive in Scala.
Java & Scala (thanks to the JVM) offer fantastic options for resource util. You can build very lightweight stateless microservices that run on a box with 1 or 2 cores. You can also build services that utilize hundreds of cores and terrabytes of RAM
With the added benefit of easier recovery when the main programmer leaves and take with him/her all his/her knowledge about the code. I've heard about this plenty of times when in a relatively small company, using [name your preferred somewhat exclusive language here] was preferred because someone made that decision alone, a long time ago, and the end result being almost unmaintainable code and longer than average time to get going or even find replacements.
Indeed, this is a reductionist view, like saying the best engineers are those that use a specific tool, not those that have access to a broad body of engineering knowledge and real-world experience.
That's because there are higher level abstractions like monads, bind, type classes etc
These take more time to learn but once you know they make modelling some problems much easier.
I do think a lot problems in programming come from businesses wanting everything to be easy to understand and to require little training and education.
As opposed to highly trained people who know their craft.
This as a business decision is fair enough. But you can't then complain your software is buggy or it takes to long to build something because you haven't built higher level abstractions. You made that decision.
I suspect you can hold off imitators in an early stage company for a few extra months by giving the impression that there is a high cost/high risk aspect to your product, and it creates a foundation of quality reasoning about features.
However, I have also witnessed a culture clash a few times, where to build a business with investors, you need to scale up with commodity developer talent, and you lose the core team when you try to integrate them. Anecdotally, functional languages also tend to attract personalities who can be difficult to focus on products, which are to them just side effects of discovery. A company that depends on "geniuses," to run it is probably not a very sound business.
If I wanted to build a company positioned for linear growth over a 20 year horizon that was services heavy, in a specific niche like utilities or security, I would choose a functional language for the platform. If I wanted to position a company for exponential growth or integration/acquisition into another product family, I'd pick something with less friction around its cultural edges.
It's more likely that this aura of cultured mystery is a self-defense against the hard truth that's been recognized by businesses for decades: programming language choice doesn't matter that much towards success OR implementation quality, with some exceptions.
When someone wraps so much of their self-worth in a programming concept, it would be painful indeed if it turned out that said concept is remarkable only in their bubble.
There are upsides and downsides to the mysteriousness piece. The upside is rapid decisions based on group deference to demonstrable expertise and competence. The downside is a bottleneck where you can't delegate competence, so everything needs the guru blessing. Ultimately, this figure only gets a veto, and you get the global veto anti-pattern in the org. It's a basis for alignment that doesn't scale as well as dynamic collaboration.
To me, the quality of thinking part has an effect on the culture is expressed as the cost/benefit and scaling aspects of fast hierarchies vs. the optionality of dynamic collaboration. (viewed negatively as command/control vs. political solutions.)
It usually happens on established products that have been around for a few years.
There are really few languages that can sink a project and that too only happens they're used in a domain that's uncharacteristic for them. E.g: writing a web app in classic VB. Windows GUI app in VB would be fine.
In a code base with no tests, I'd rather start with something had a powerful type system.
But you are right that you make a mess in any language.
(Aside from functional programming, Ruby on Rails comes to mind even though it is a language and a framework.)
Obviously if you want to iterate quickly, "move fast and break thinks", something like Python is more appropriate than C++. Not because Python can be said to be "more powerful" in any absolute sense, but because the trade-offs in the language design (speed of development above low-level control) makes it more appropriate.
Perhaps power = ability to be more concise which translates into "more things done per unit of time compared to a less powerful language."
He wrote that article in 2001, eighteen years ago, and people haven't stopped bringing it up on HN since. So where are these startups crushing their opposition because they're using a more powerful language?
However using reasonable tools is important. The wrong ones can slow down companies and make business fail beyond the skills of the marketing departments. In my experience I didn't see much difference between Elixir, Java, Perl, PHP, Python, Ruby. Properly handled any of them could be ok even if I've got preferences. JavaScript... too much cognitive overhead because of constant ecosystem changes and fragmentation but maybe it's slowing down now.
Focusing on programming languages is the mother of all local optimizations. Although what we do is obsessing, not merely focusing.
While perhaps functional languages are "powerful" in terms of performance[1], but "weak" because they have poor tooling/ecosystems, not even starting on the difficulty of recruiting, etc.
[1] Though I personally think the author is exaggerating the benefits, while conveniently skipping over the massive ((neg))((ati)(ves))))) and incomprehensible code large functional code bases often produce
It would make sense if you managed to sort out the precondition first, i.e. find sufficiently capable and affordable programmers able to use the language, make sure the language ecosystem is as capable of supporting your field as more popular languages. In other words, it rarely makes sense because "all else" is usually not equal.
The same could be said of Elixir + Phoenix in 2019. In a lot of ways, Elixir macros and protocols give at least as much power in building DSLs as Ruby-style metaprogramming, but there's no magic and every single module reveals which macros it uses at the top.
It's not quite the stark trade-off Rails offered vs old school Java / PHP development in the mid to late 2000s, though. In that case there was a tremendous productivity advantage for using Rails to make a web app. Now, Elixir offers moderate advantages, that grow with project size but it's competing against already well-developed Rails and Laravel ecosystems which offer excellent productivity in their own right, plus an existing gem/library for "everything".
I love what Elixir's immutability does to make testing and handling larger projects easier and I'm glad to have learned it, but in all honesty I never would have if not for having been focused on a free to use chat app in 2016-2017.
Modern systems can remove lots of complexity and enable fast prototyping, which we are all aware of. But it comes with a cost, and that cost (e.g. having to deal with scaling much, much sooner) hits at a sensitive time in startup growth.
A native compiler backend for PureScript (via C++) https://github.com/pure-c/purec
They bring in a really good senior developer as a CTO. This person finally has the chance to set their mark and for once build the perfect setup, no legacy systems to take into consideration, no outdated corporate directives limiting languages or architecture. This time it's going to be perfect.
Other aspects tend to be forgotten:
- The startup might not require a very sophisticated tech stack. Most of what needs to be built might be lots of html forms, user handling and a few other thing that can be done in any language.
- They and up with a pretty advanced stack of best-for-the-job tools and languages that forces the company to compete with lots of other trendy startup for a very small pool of highly paid highly qualified developers.
- Some tooling and functionality might have to be built from scratch compared to more widespread languages.
A healthy question to ask before picking a more specialized stack is "What do we really gain compared to building this on Ruby/Rails, Django/Phython or Symfony/PHP?"
I'm going to level with you, the choice of language is a combination of luck, personality and circumstance. Nothing else. There is no hidden knowledge, font of programming youth, etc, etc,
Its like choosing a building material, when you want to make a house. Most people choose bricks. When used properly you can make many a wonderful, functional and innovative house.
In short you can do all of the things you want with any Turing complete language.
As for scaling, its mostly never down to the language, its almost always system design. Crucially its knowing when, where and how to store state. Does it need to be shared?, does it need to be consistent?, how can you partition? what stuff can you push out to static storage? None of those questions are answered by a language.
The stated scale (10million requests in a day) is roughly 120 requests a second. Most systems _should_ be able to handle that without costing the earth hosting wise. Hell, the API I look after can do 5000 concurrent requests, ~8000 requests a seconds using about 2.5 cores. That's in python without any real optimisation. Its not because python (or my skill) is awesome, its because CPUs are really really powerful now.
A skilled programmer could easily get 2-5X more performance, but it'd cost at least 5 years of hosting to get them in and optimise it.
TL;DR:
Its not the language, its the business plan and design that matters.
I still love JS, but I'm not so sure anymore that it's for me.
The major difference of the industry compared to 90s is everyone stop getting hyped by new ideas. In 90s there were companies keep re-writing from C to C++ then to Java recklessly.
Now everyone is 'smarter' - stop worrying and love Java, forever.
And they have a famous quote for justifying being lazy: there is no sliver bullet.
Anecdotally, I have been able to ship much more quickly and reliably with Haskell than I ever could with Ruby/Rails.
If developer is more comfortable with haskell they will likely be faster in that than python. The added advantage it will also likely to be more correct.
I make a living from untangling poorly written code, and these would mean more work for me.
Oh please use an obscure or low traction programming language as well, preferably an archaic one.
Why post here at all if you’re just here to troll and start flame wars?
I am being sarcastic against posts contrarian to common practice only for the sake of being contrarian, this being one.
If a current good practice is to write tests, there will be a post against writing tests. If a current good practice is to write documentation, there will be a post against writing documentation. If OOP then functional.
In my consulting job I do have to untangle code written using contrarian practices, obscure languages, and other temporary fads.
Not that Haskell is not a good language, or that functional programming is a bad approach, but rather that neither are a secret weapon against anything else but specific problems these are good at solving.
Good luck to those companies scaling to more than a handful of devs - they could also hire people to migrate them the right language for the task at hand.
What the hell is that? You want people to not write tests or make documentation? (Totally ignoring the fact function code is easier to test and document anyway).
I'd hate to work with your code. There's probably no point me even trying to find out what you mean because I suspect its some totally ridicilous argument that only seeks to annoy whoever reads it. Nice bait.
Assuming a school day runs from 9 am to 3 pm, that an average of about 460 requests per second.