Getting Closer to Real Time With Hadoop

gigaom.com

14 points·jhammerb··5 comments

I've been playing with Hadoop for the past couple months, and recently have been using it quite extensively. It's an amazing piece of software and I hope to start contributing to it.

If anyone wants to talk start-up ideas involving large data-processing, my e-mail is in my profile.

Anyone have real world experience to share with HBase or Hypertable? Or Cassandra, for that matter. None of these projects strike me as "mature" software but I'd love to learn otherwise.

Yeah, they're all pretty early adopter, although worth it if you're feeling enough pain from data size and/or volume. A couple recent articles on Cassandra:

http://news.ycombinator.com/item?id=813528 http://news.ycombinator.com/item?id=813919

(I work for Rackspace on the Cassandra project, and we are in the early stages of deploying Cassandra internally too.)

All today I have been obsessing over Map-Reduce on GPUs. Here is a link (http://graphics.cs.ucdavis.edu/~lefohn/work/dissertation/lef...) to get things started. Still not sure if Java is a good front end to openCL. (Would appreciate thoughtful responses.)

This article isn't really about map-reduce, but anyway.

Map-Reduce on GPUs

It depends whether your code is suited to GPUs. If you're map-reducing a bunch of single precision floating point data it may be a good idea.

Still not sure if Java is a good front end to OpenCL.

It doesn't matter IMO; you're still writing the code that does the actual work in OpenCL. Think of it like a different flavor of JNI.

Reply on news.ycombinator.com