6 karma · joined November 8, 2015
> the thing that would have helped me most in avoiding some of the missteps might have been a richer test interpreter, where I could have "mocked" a network
This is a really intriguing idea. Debugging distributed can be a huge headache, and I can see a lot of value in having something like that.
> We’re planning on doing several of these articles to show different applications.
Looking forward to future articles and definitely will be following this project!
> Confusingly, the term refers to overall invisibility of the component, it does not refer to visibility of component's internals
Indeed, perhaps "invisible" would have been a better word choice.
1. The posts mention Spark, but were you (the Unison team) also inspired by dataflow / stream processing engines (e.g. Flink, Beam, Timely Dataflow etc.) or any other technologies?
2. Relatedly, are there any plans to add a stream processing API? Or perhaps that would belong in a 3rd-party library?
3. Broadly, where do you think (in your opinion) Unison fits in the ecosystem of tooling? Is Unison specialized at solving big data problems that require distributed execution e.g. something I pull out of my toolbox for a problem I'd otherwise solve with Spark? Or is it a truly general-purpose programming language?
4. With the Tree example, you show the "wrong" way and then the "right" way -- one problem I've experienced in the past is how do you as a new user know the right way. For example, I wrap my entire program in IO then .unsafeRunSync it -- technically following the rules, but got nothing out of doing that. Have you thought about how the APIs could guide the user to the idiomatic solution? Perhaps via types?
By the way, I really loved the FP in Scala book, it was my first step into FP -- thank you and Runar for writing it
1. Explicitly marking non-local _data_ vs. in-memory. A very cool idea -- languages and libraries I've used all seem to try to make this transparent (unsuccessfully, as indicated by the random serialization errors you get)
2. Making the noise around serialization and RPC transparent to the user
3. Bring the code to the data, not the data to the code
4. Immutability, FP, and I like the syntax -- no coincidence in its similarities to Haskell and Scala, I'm sure :)
These are all pain points in a lot of existing "big data systems" I've used, where they either solve the problem half-baked or don't address at all. I'm excited to see where this project goes!