Yahoo open-sources Kafka Manager for Kafka cluster management | Hacker News Reader

Yahoo open-sources Kafka Manager for Kafka cluster management

yahooeng.tumblr.com

92 pointsfangjin12 comments

Nothing that couldn't be done before with the CLI, but the visualization is nice for unbalanced leadership/assignments.

It's also really nice not to have to generate my own assignments.

Maybe I'll contribute a tool to do other helpful tasks such as adding replication to a topic when developers create topics with no replication.

I got an initial

[error] k.m.ApiError - error : Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [1000 ms]

But then I waited for a bit (Updating internal state...?) and hit refresh and it worked. Also it didn't just pick up my cluster from zookeeper, had to click "Add cluster" and enter zookeeper deets before it showed. This is probably expected behaviour, just a heads up to others.

Yes, the initial request into play framework initializes the whole application, which is why you get the time out on first request.

The zkhost configuration you performed was for the kafka manager application itself, since it stores state in Zookeeper. Once started, you did the right thing by adding a cluster.

This headline makes me giggle. I can't stop imagining a bored bureaucrat managing a group of depressed writers trapped in endless labyrinths of rules.

Yeah… It always seemed like a weird name for a messaging system. Was there any explanation why?

Perhaps it's meant to convey that, unlike the story, the system can deliver "A Message From the Emperor".

I could see the project leads finding the following quote relevant to their software's messaging-focused design, in the wry Unix tradition that calls services daemons and so on:

"They were given the choice between becoming kings or the couriers of kings. In the manner of children, they all wanted to be couriers. As a result, there are only couriers. They gallop through the world shouting to each other messages that, since there are no kings, have become meaningless. Gladly would they put an end to their miserable existence, but they dare not, because of their oaths of service."

Looks like it could be a useful tool. Is this version expected to work with the release 0.8.2.0 (besides the beta version listed in the dropdown?)

I have a single cluster of two brokers (no topics yet) but it does not show up in the manager after creating a cluster pointing to my zk... suggestions? I am using kafka 0.8.2.0 release version.

Oops... The reason the cluster was not displaying was because I left the node name off the zookeeper url. Originally I just had hostname:2181 but should have been hostname:2181/kafka. So far seems to be working well.

Great stuff. Can't wait to see how well this plays with Kafka running under Mesos.

Kafka manager uses Zookeeper for all information, doesn't know what container is being used to run Kafka.

Just dockerized it: https://github.com/srangwal/kafkamanager-docker

Reply on news.ycombinator.com