Visual Dataflow Implemented in Lisp [pdf] | Hacker News Reader

Visual Dataflow Implemented in Lisp [pdf]

web.onetel.com

31 pointsisp9 comments

Full Metal Jacket:

- Home: http://web.onetel.com/~hibou/fmj/FMJ.html

- Tutorial: http://web.onetel.com/~hibou/fmj/tutorials/TOC.html

Can we download it and kick the tires?

Not yet. Please be patient. I still have to add the ability to define new data structures and types before it's a complete language. I'm currently working on that. Then I need to make the code more robust. I also have to eat my own dog food for a while - if I don't use it, who else will?

After all that, I'll sort out the .303 release. This will allow free (as in beer) non-commercial use.

Simple arithmetic and function definition are simple enough to follow, but the iteration and conditionals seem clunky.

Also, I think there's a typo on this page because the images doesn't seem to match the equation (or I'm not understanding the mapping between code and graphics): http://web.onetel.com/~hibou/fmj/tutorials/Functions.html

Pretty much. Iteration and conditionals are the bane of a making a DAG elegant. I think the inverse is fairly true too, data flow is much better visualized in a DAG and much more difficult to see in an imperative line by line approach.

Funny in image compositing software the iteration is abstracted away as time ticks over image sequences that update the input nodes thus pushing new outputs. It's there without being there explicitely.

Right, and the iteration over pixels is hidden in plugins themselves. Complex filters though are difficult or impossible to build since there is no granularity and in fact gathering or scattering of arbitrary memory locations is very awkward.

Anyone who has used Shake or Nuke knows that working with essentially a domain specific interface is an enormous benefit.

I think there is a lot of value in learning from compositing systems as well as Houdini when writing software - namely being able to freeze data in place and iterate on one data transformation at a time.

Houdini especially since he deals with any kind of data, it's almost the Lisp of computer graphics. Some tutorials are mixing geometry, analog signals (audio), discrete signals (user inputs), through Houdini's builtin operators. Never seen such versatility. I always say this should be taught in programming classes (both for UX, ergonomy, programming paradigm).

And the virtual + freeze is also very present in audio production software. All this things that are close to memoization, laziness are present there.

Clunky compared to what? Other visual dataflow languages (Prograph and LabVIEW) use additional constructs for iteration and conditionals, complicating their syntax. The original plan (as described in the papers) required several different emitters and collectors. Allowing edges to loop back for iteration would complicate the editor. TAnother option is tail call elimination, which forces you to think recursively about something which is naturally iterative. If you can think of something better, I'd be interested to know what it is. Remember, there is no flow of control.

Text-based languages have a different problem: indicating which parts of your code can be run in concurrently. They evolved to run on single processors, with flow of control specified by the programmer. The result is that programmers of those languages tend to think sequentially. Adding support for SIMD parallelismmight be straightforward, but MIMD parallelism is more problematic. With dataflow programs, you don't have to specify what can run in parallel.

You also get homoiconicity for free: programs are directed graphs, the most general data structure. No parsing or optimization is necessary. Algorithms are reduced to their most basic form (actions and communication), not shoe-horned onto a von-Neumann computer architecture. This does, however, require the dataflow programmer to think in a different way.

The typo has been fixed. Thanks for pointing it out.

Reply on news.ycombinator.com