Comment by samuell | Hacker News Reader

Comment by samuell | Hacker News Reader

The core of the FBP principles are the holy grail of true componentized function architecture.

Instead of losing yourself in ever more complex syntax convolutions as has happened in a lot of functional programming, you make the components (even long running stateful ones) self contained with ports for data input and output as the sole means of communicating with them, over buffered channels to allow asynchronous computation, and most importantly keep the network definition separate.

Just this idea in itself is just brilliant. Hats off to Mr Morrison for that!

It allows to decouple complex software into reusable components, without clever FP syntax.

(Though, FP is perfect for implementing the components themselves. It just doesn't really scale all too well for whole program architecture, in my experience).

One point to note: The visual component of many fbp systems is completely optional, as is the idea of using novel DSLs. You can as well define your networks and components in pure code. See GoFlow https://github.com/trustmaster/goflow) and my own little experiment FlowBase (https://flowbase.org) for examples of that, in Go.

I successfully built a rather complex little app to convert from Semantic web RDF format to (semantic) mediawiki XML dump format, in two weeks straight, of linear development time: for each component (of ca 7), implement, test, go to the next component (See: https://github.com/rdfio/rdf2smw)

The same implementation in procedural PHP took months, and still doesn't have all bugs and strange behaviours filed out.

> you make the components self contained with ports for data input and output as the sole means of communicating with them, over buffered channels to allow asynchronous computation

This concept sounds exactly like actor systems like Erlang/OTP and Akka, only with a different set of terminology.

The submitted site and your comment don't mention those anywhere.

Are there appreciable differences between actor systems and FBP?

Actor systems differ in their means of communication. While actor systems use the actors as the main units that you connect, with fire and forget messaging to each actor's inbox, FBP proposes communication between named ports over channels with bounded buffers, very much like the CSP implementation in Go (just that you have long running components as in actor systems - something that can easily be implemented in Go with structs with go channels in fields for the ports, as shown in the FlowBase library I mentioned in another comment).

The FBP model provides implicit backpressure, because of the bounded buffers on the channels. The actor model on the other hand is more loosely coupled and flexible.

This actually means FBP systems are slightly more optimal for efficient within-one-node, in-memory parallellisation, whereas actor systems shine more for distributed systems.

They shine on different scales, in other words.

I wrote a post many years ago on this, that seems to have aligned with the experience of a number of people, based on the reactions in the comments: https://rillabs.com/posts/flowbased-vs-erlang-message-passin...

Akka Streams adds named ports and defined backpressure, as well as a graphical-ish DSL for connecting everything.[0]

[0]: https://doc.akka.io/docs/akka/current/stream/stream-graphs.h...

That is nice to see! I was evaluating Akka a bit some years ago.

AkkaHttp is written on top of Streams. There are many more things written using it but Http is a good example to learn from.

It is worth noting that Akka Streams is an implementation of the Reactive Streams [0] spec, using Akka

[0]: https://www.reactive-streams.org

> The FBP model provides implicit backpressure, because of the bounded buffers on the channels. The actor model on the other hand is more loosely coupled and flexible.

Note that this is why people recommend using GenServer's "call" instead of "cast" in Elixir/Erlang by default - it applies a sort of backpressure because call waits for the process to return a result rather than being fire and forget.

It's not exactly the same thing, or as configurable as Go channels, but it is an option.

Elixir has several tools built around solving this problem; GenStage being the foundation library and Flow and Broadway at a higher level.

On a related note, there's also work on an FBP implementation in Elixir: https://github.com/pcmarks/ElixirFBP

What's curious is that CSP started with named ports as well, but evolved to become anonymous.

> Programs in the original CSP were written as a parallel composition of a fixed number of sequential processes communicating with each other strictly through synchronous message-passing. In contrast to later versions of CSP, each process was assigned an explicit name, and the source or destination of a message was defined by specifying the name of the intended sending or receiving process

I wonder what gives? Why this return to an older form?

FBP is most analogous to the unit record machines of yore:

https://en.m.wikipedia.org/wiki/Unit_record_equipment

What you are programming is the graph of how these machines connect and flow data(keypunch cards) through each other, while the machines themselves and their function are abstracted out. Data does not stay "at rest", it's presumed to reach a terminating point where it exits the graph. A buildup of unprocessed data in a machine's inbox results in an overflow.

It's a very useful model for making a debuggable asynchronous system since it imposes static constraints everywhere that you can map to your real hardware resources, versus the emphasis on dynamism seen in the actor model(actors hold private data, modify their state, create new actors - all explicitly disallowed in FBP).

Having studied and implemented FBP systems in the past, one major takeaway I've gleaned is that most automation problems start off as a linear sequence of processes, and so the branching-graph of FBP looks unwieldy and superfluous. But this is deceptive; you probably don't want to have a huge number of branches in the design, but you will want them as an optimization step or a way to combine inputs.

So it's useful to design with FBP in mind but with a linear interface as the entry point.

Another aspect of this is that FBP graphs are static but you may have a need to reconfigure them frequently; that is, you may want to have a graph compilation step drawn from a source language, rather than manually wiring it up.

A way of making that graph compilation more than a syntax is to include a formal constraint solver: Excel, for example, flows the data after determining a solution for how cells relate to each other. The power of the spreadsheet paradigm really lies in these combinations of concepts.

Lastly, there isn't really magic in the algorithmic/implementation aspects of FBP. It grew out of 1960's mainframe types of problems, and so it can be implemented in a low level way with static pools of memory and pieces of assembly code. But it remains conceptually just as relevant to today's massive distributed systems.

Visual aspect of FBP is also brilliant and helps to navigate/comprehend inherently parallel execution flow. The same complexity network expressed in linear text would require more effort to grasp. And it allows you to leverage your brain's visual capacity which is quite powerful - humans are visual creatures (https://www.seyens.com/humans-are-visual-creatures). For example in Excel the update dependency network is completely hidden. The maintenance of complex Excel models is really hard.

For same reasons we start looking at FBP and functional reactive programing to simplify the design and maintenance of complex interactive UIs. End up implementing Kelp (https://kelp.app) with visual FBP editor and reactive framework (https://kefirjs.github.io/kefir/).

Sounds like XProc 3.0 https://xproc.org/

Your flowbase.org domain redirects to a different domain. I thought that perhaps you had a typo in the URL or forgot to renew your registration of the domain, but upon a closer look I think it’s probably just a misconfiguration in your webserver config causing it to redirect to another one of your projects.

My guess is that you forgot to put the definition of the domain sans www in your webserver config.

If my guess is correct then this link should work:

https://www.flowbase.org

Edit: doesn’t work either. Perhaps you don’t have a TLS cert for that domain? In which case maybe this link will work instead:

http://www.flowbase.org

Edit 2: Without https it works. And like the person responding to this comment said it redirects to the GH repo.

The link's trying to point you here: https://github.com/flowbase/flowbase

Thank you!

Ouch, sorry, and thanks! Wrote on mobile on the go and didn't manage to double check the link.

FP decouples the AST (symbolic functions with input ports and an output port) from the evaluation context, which might be async, or sync, backpressured, stateful, exceptional, incremental/reactive ... mix-n-match whatever behaviors you want, all for the same abstract AST

I actually didn't want to make this too much of an FP-bashing comment, as I find great use of many FP concepts, daily.

I have yet to see an FP concept for composition though, that is as simple and generic in its implementation, as the FBP principles.

I have sometimes thought FBP networks provide roughly the same function (pun not intended) as a monad, although I never seem to fully grasp what a monad is, so I can't tell for sure :o)

Yes, dataflow with dynamic topologies (self-adjusting computation) forms a monad. If the "flowchart" is merely static, that forms an applicative. https://blog.janestreet.com/breaking-down-frp/ In my opinion, understanding the monad is essential to understanding how to fully separate an AST from its evaluation behavior. That is basically what a monad is, the monad is the computational structure that does that.

Thanks, that clarifies a lot!

I think most FP languages don't decouple AST from evaluation context. E.g. I cannot perform abstract interpretation on a function's AST, nor rewrite a function's AST for incremental computation, nor perform explicit partial evaluation during function composition by composing the ASTs. I only can access opaque `Input -> Output`.

Also, FP algorithms developed for one evaluation context cannot easily be ported to another. They are implicitly entangled with assumptions. For example, I cannot evaluate a lazy algorithm in an eager evaluation context without paying a huge price. Adding back-pressure for list processing where a program was not designed for it would easily result in deadlock. Adding exceptions without adding unwind (or bracket, try/finally, etc.) to the existing program expressions will easily result in buggy code.

I think your assertion might be true for some specific integrations of FP nodes into FBP. Is this what you mean?

Today-era functional languages (so basically haskell and scala) don't do that at the "host PL" level, but you can model it in-band, and modeling embedded PLs is what most of the cool FP research for the last 5 years is all about. Free monad, tagless-final interpreters, etc. You make a DSL with abstract capabilities and then code your application in that. Any reified behaviors end up encoded into the type. I hear Facebook is using algebraic effects to model and track data access.

FP is relatively convenient for metaprogramming, yes. GADTs and tagless-final encodings (or as I think of them, Church-encoded ASTs) are especially useful for this.

But it wouldn't be a big difference if the host language was heavily imperative, so long as it favors immutable data structures. The FP aspect for this exploration of DSLs is much more social and cultural than technical.

I think that if we really want to decouple AST from evaluation context, the main tool we'd need is to deconflate module 'import' into separate steps to 'load' a module's AST and 'integrate' definitions, allowing for intermediate processing. This requires some host language changes, elimination of module-global state, etc..

The flowbase.org link is redirecting me to a polish website, not sure what’s going on there

Sorry about this, I was typing in a hurry. The links should redirect to https://github.com/flowbase/flowbase but seems to be misconfigured for https. Looking into it.

Reply on news.ycombinator.com