Iteratee-based IO in Haskell at Tsuru Capital | Hacker News Reader

Iteratee-based IO in Haskell at Tsuru Capital

blog.kfish.org

50 pointsdons11 comments

I work at Tsuru Capital. (Apparently we have a contributors file for the iteratees library, and I never added myself.)

Feel free to ask questions here. I'll try my best to respond to them. We're preparing for ICFP tomorrow, and my train back from Yokohama was pretty late, so I probably won't be able to answer right away.

is iteratee IO something where you need a special library for each input source? For example, if I want to read data from MongoDB via the iteratee paradigm, do I need a special MongoDB-Iteratee library, or can I just compose an existing MongoDB library with an iteratee library somehow?

To gain all of the benefits, you need the code to be written in iteratee style. You can take existing code as a starting point, but you'll often find it has to be changed quite a bit to get all of the benefits. It's usually a bit more involved than simply composing existing functions into iteratees.

I wrote the persistent-mongoDB library which exposes an enumerator interface. It was actually fairly straightforward to hook up MongoDB to persistent's enumerator interface because MongoDB's driver already exposes a cursor which is essentially the same thing.

It is extra work to use persistent's enumerator interface though.

It looks like you guys are using the iteratee library; why did you choose that over the enumerator one? I've been trying to learn enumerator, mostly because it's what Snap is built on, but I'd love to hear about why one is better than the other. I know that in the enumerator library, stream converters (Enumeratees) make my brain hurt; are they any easier in iteratee?

Enumerator didn't exist at the time the project was started.

(I'm the author of enumerator)

I originally wrote/released the enumerator package because iteratees seemed really interesting, but I didn't like the function names or docs. That's it. The sole advantage enumerator has is that it's prettier.

Compare:

  mapStream :: (Monad m, ListLike (s el) el, ListLike (s el') el', NullPoint (s el), LooseMap s el el') => (el -> el') -> Enumeratee (s el) (s el') m a

  Map the stream: another iteratee transformer Given the stream of elements of the
  type el and the function (el->el'), build a nested stream of elements of the type
  el' and apply the given iteratee to it.

with:

  map :: Monad m => (ao -> ai) -> Enumeratee ao ai m b

  map f applies f to each input element and feeds the
  resulting outputs to the inner iteratee.

I figured a simplified package would be useful as sort of a tutorial before people upgraded to "full" iteratees. I never planned on it becoming heavily used as itself.

I guess the biggest problem I have trying to figure out Enumeratees is that I want them to transform Enumerators into other Enumerators, but they seem to do something different, and I'm not exactly sure what. Enumeratees are actually Iteratees, but they have a strange "b" that's actually a Step. It boggles my mind somehow, probably because I'm trying to work out how that's equivalent to an Enumerator converter.

After wrestling with this since thursday, I wrote a function that's something like (wrapEnum :: Enumerator ai m b -> IO (Enumerator ao m b)), so I can convert an enumerator of one type into an enumerator of another type (I'm not looking at the code right now; that signature is obviously lacking, but it's the idea, anyhow). I think that's probably horribly wrong, but I just can't wrap my head around the type and behaviour of Enumeratees, even after reading the really pretty and simple ones that come with the snap-core package.

I think the idea is awesome, and I'm very happy about being able to use other people's Enumeratees like the various compression ones and ideas that I can express as a map or a fold, but I'm definitely missing something.

The extra complexity is for dealing with extra data. Say you're mapping ByteString->Text. The iteratee reads one char, then says it's done. What do you do with the extra bytes and text?

Making them return a Step allows both types of extra input to be stored, and possibly consumed later by other iteratees.

You can use (=$) or ($=) to modify how extra input is handled (by discarding it at certain points). Your wrapEnum is probably a special case of ($=).

Not related to iteratees, can you describe the use of Haskell at Tsuru Capital? Disantagens, advantages, etc. Here I live people look weird if they suggest Haskell(including myself), so great is .net and java monopolium.

whgats it like livin g in japan

Reply on news.ycombinator.com