A General Theory of Reactivity
github.com
github.com
Even using the (quite unclear) hijacked definition of "reactive", there is absolutely nothing tying the goal of responsive programs to concepts such as asynchronous APIs, promises and observables. As an example, Erlang (and later Go) has been used to write "reactive" software with neither a hint nor a mention of callbacks, promises or observables (at least for the purposes outlined in this text).
Those concepts are mere implementation tricks designed to circumvent blocking in environments that either don't have support for multi-threading at all (which makes blocking impossible if concurrency is to be maintained), or those where threads are necessarily kernel threads (where blocking entails a significant overhead). In fact, the whole notion of non-blocking is entirely accidental to the discussion of responsive/concurrent applications. Where lightweight threads are available (and, therefore, blocking is free), those design patterns are completely unnecessary for writing responsive software, and, in fact, make it quite cumbersome.
Callbacks, promises and observables obliterate stack traces, make concurrency opaque, require explicit back-pressure as well as a complete "shadow" implementation of error handling and control structures beyond those provided by the language. They would all be considered anti-patterns[2] if they weren't a necessary last-resort workaround to the problem of expensive blocking (which happens to be not too hard to fix by other means).
[1]: http://en.wikipedia.org/wiki/Reactive_programming
[2]: At least in non-pure languages. They do serve a purpose in referentially-transparent languages that don't have the concept of a stack or a thread, at least not in their common definition.
React is a good example of a framework that lies outside of this: it is a simple solution for the problem that doesn't involve callbacks, promises, or even observables. My own project, Glitch, solves these problems by managing time:
http://research.microsoft.com/en-us/people/smcdirm/managedti...
It doesn't block, it doesn't use functional abstractions either, Glitch simply replays code until all writes have seen all reads (it also doesn't requiring lifting of time-dependent values in the same way that React doesn't). Also, see:
http://research.microsoft.com/pubs/230746/control.pdf
Which argues that we might have given up on control flow too soon.
Still this isn't a bad essay for what it is, even if it isn't very general.
I talk about this toward the end of GToR.
Let me just clarify one point: lightweight[1] threads are not an abstraction; threads are. Lightweight threads are simply threads whose implementation is such that blocking has no (or negligible) overhead. When you have threads, you're better off using blocking queues and blocking futures, which are the "pull" duals of the "push" mechanisms described in the article.
[1]: I don't like the term green threads because traditionally it's been used to describe threads that do not support parallelism.
Functional programming and async I/O in single-threaded languages are closely connected. Functional programming has no built-in notion of time; conceptually, the time a pure function takes to run must not have any effect on correctness, or replacing a function call with its value would have side-effects. To allow time to pass in a way that matters for correctness, you need callbacks, and monadic types are a form of callback-based programming. In a single-threaded program, the goal is for functions to execute as quickly as possible so that the user doesn't notice pauses, so the solutions end up being similar.
So these are not implementation tricks; they're fundamental to the theory of how programs execute. Erlang is built on callbacks as well since each event results in another function call.
Goroutines and blocking function calls are fundamentally different. Blocking function calls need to return at the right time or it's a bug.
Are you getting hung up on the fact that GTOR talks through some specific apis for reactivity? You seem very much opposed to letting someone mention callbacks, prormises or observability? But GTOR is itself about mentioning these things so we can get past them and think of reactivity more generally. Yet you seem mired in your hatred for these concepts, in a way that prevents you from seeing that the article is about building effectful processing- having code or data whose effect is the running of other code, in a general fashion. And your inability to see that generalness, your forcing some weird diatribe against non-greenthreaded systems as all missing the point, is there rooted itself in confusing an implementation with intent, for surely blocking or no, concurrent or non, code and data will create reactions amid others.
I suggest getting more into the categories about. Sometimes binding happens before event, sometimes event before looking. Sometimes event is singular, sometimes it's over a duration, sometimes it's ongoing- edge, level triggering and continuous. Sometimes event has concurrency sometimes it's over time.
None of these don't make sense in Erlang or any other runtime, even if you might not see them expressed as first class constructs with apis, even if the reactivity were more baked in, as Erlangs send is.
Most of all, I have a very hard time understanding what your clash really is. I feel very much I'm being sold something, but the first paragraph doesn't give me a clear idea of what it is I'm being asked to accept evidence for.
There is nothing about these that make them related to "general reactivity", mostly because there is no such thing as reactivity. They are simply techniques of performing actions in referentially transparent languages. In imperative languages, the most general, theoretically pure, fundamental construct of "reactivity" (whatever that means), is the thread (Erlang's send is not special, and not required).
There is absolutely nothing more fundamental about the referentially-transparent (lambda calculus) way of doing things than the imperative way of doing things, and since the article discusses imperative languages, the LC way of doing things is potentially harmful as there is nothing to enforce referential transparency.
And those things GTOR does use in square 1 are fine. Your heavyhanded rejection of them is colosssaly sad and twisted. If a mental construct suits a problem and helps you think of it, that's what is important. The programmatic platform follows our ability to think, it does not define it, yet you keep insisting the cart must come first, that the platform has to be a certain way or else or else or else. FUD. FUD FUD FUD.
It's using the references that everyone on the planet but those of your little cultish view know to explain general ways of thinking about things. Your unwillingness to see parity between the general ideas put forward because it's not your favored implementation reads grossly to me. Observers, contracts, data-flow are real, graspable concepts and if they closely model how you want to think about a problem then they are most definitely good concepts, and it's up to the platform to have good ways to express natural thought. I still have an enormously hard time decyphering the mad mess of shibboleths you toss out, activating your elitist trope of true beleievers (and I'm not a complete neophyte to your clique myself), but I am 100% sure, as I stated in my first paragraph, that all systems enact a coupling/reactivity. Control, whether it's explicit or implicit in the flow, is still the chief thing entailed in mentally modeling and putting into practice: this is unrejectable.
In particular, your bringing up of the referential transparency doubles down on my prime criticism in the previous: you bring up points that you claim contend, but you provide no support, no structure where anyone but those already attuned to your brand of thought can get onboard with you. Your claims are uncontestable, because they are indecypherable. They claim a split, but don't deign to point out what that split is. That, culturally, is a big indicator to me.
Your case rests around whether or not reactivity is a general thing or not, and I think it's quite clear we can find these concepts embodied in any system.
What is "reactivity"? This document defines it as "the process of receiving external stimuli and propagating events". Ok, then -- I wouldn't call it a very rigorous definition, but I can roughly grasp what the author means (more or less; I think). Now, please show me why callbacks, promises or observables are more "general" approaches to "reactivity" than, say, threads, blocking queues and blocking futures? I chose those because they happen to be the duals of the aforementioned constructs. If A is a dual of B, how can you say A is the "general theory" while B is a "cultish view"? This is what is very much rejectable.
Now that we've firmly established that A (callbacks, promises, observables) is no more general than B (threads, futures and queues) -- yet the document focuses on one while completely ignoring the other -- I express my opinion (that I can support[1]), which is that in imperative languages, constructs B are far superior to their no-more-general constructs A. That, however, unlike my previous statement (about the document confusing goals with implementations, which was a statement of fact), is just a statement of opinion.
> Observers, contracts, data-flow are real, graspable concepts and if they closely model how you want to think about a problem then they are most definitely good concepts, and it's up to the platform to have good ways to express natural thought.
Sure they are, but this document mentions some antipatterns to implementing these abstractions (which is why I said it confuses implementation with abstraction). Queues are definitely a dual of observables, and in my (and many others') opinion, they are a far superior way of implementing dataflows in imperative languages (they provide implicit backpressure, and they make concurrency clear). Same goes for futures vs callbacks, or threads vs monadic composition (threads preserve stack traces, control flow and exceptions, while monads/promises don't -- they require a shadow implementation of all those basic constructs).
I model many problems with a dataflow abstraction, but when I want to implement it, I reach for a blocking queue (aka, channel) rather than observables. Don't confuse abstraction with implementation.
[1]: https://www.usenix.org/legacy/events/hotos03/tech/full_paper...
I agree with grand parent on this one. Since it "complects" what is to be done with how to do it, it's by definition not as general as it could be.
[1]: No filtered exception handling? Just got halfway-decent `for..of` loops?
In systems, there is a big push for (and against) "event-based programming" models with respect to concurrent and distributed systems; e.g. see:
https://www.usenix.org/legacy/events/hotos03/tech/full_paper...
where the definitions are even more different.
http://lampwww.epfl.ch/~mcdirmid/papers/mcdirmid06superglue....
Early on, I realized that pull was the only safe/sane solution, so in all my languages (barring early versions of Superglue), change propagation merely dirties execution that could have read the changed value, which is queued to be re-executed and will pull the new value if it still depends on it. That is the style that I use for Glitch, and it has turned out to be very robust.
Push is much harder. Actually, I'm not even sure how those systems even work properly; too many things to coordinate.
As a someone who does JS for my day job I'm really enjoying playing with Clojure in my evenings and weekends (like now, it's Saturday night 00:30 here in Berlin). A small part of me dreams about doing it full time ;).
In Clojure, a sequence (seq) is a list datastructure. It can be traditional Lisp list, an array, or some other datastructure who's items are arrayed in an ordered sequence. You can generally treat a sequence like you can treat and array or a list in other languages accessing any element by its index, (except that it's immutable, of course.) Clojure doesn't traverse the sequence via 'next' unless the particular sequence type (ie a Lisp style list) is implemented to do it that way.
A lazy sequence is a special kind of sequence which is implemented with a sort of 'cons' structure where the 'cdr' is a function to be called rather than a pointer. To clarify, in a traditional Lisp, a list is implement with a series of pairs of pointers called 'cons' cells. The first pointer is called the 'car' and it pointed to the content at that point in the list and the second pointer is called 'cdr' and it points to the next pair. A lazy sequence is similar except the that the pointer to the next pair is rather a function which returns the next pair. This means you can have a sequence who's actual data is not yet fulfilled. When you access an element in a lazy sequence, that element and every element before will be generated (and cached). Unless you access an element in that sequence, that element will never be generated.
This lets you have things like infinite datastructures or do something like wrap a stream in a sequence and loop over it like it was an array.
BTW, if your day job is with JS, maybe you can squeeze in a a ClojureScript app on a small project? I've done this.