Hacker News Reader

Comment by vacri | Hacker News Reader

Dear Elastic:

For the love of internet-god, please stop your constant moving of stuff around. Now we have new logos.

Last year's fun, I was using logstash-shipper to ship logs. Early on, the package got pulled completely - sucks to be you if it's in your deployment script or in your documentation. Then it had a name change. Then it moved to one domain. Then it moved to another domain. Then it got switched out for Beats.

Not everyone finds setting up and maintaining an ELK stack so fascinating that they want to keep up to date with exactly where everything is this month. While you can do other things with ELK, the primary use-case is logging. Logging is supposed to be reliable and 'just work'. Every time I see the elastic website, something else has changed, and everyone is pushing the new stuff.

ELK is cool and all, but it's frustrating to follow when you just poke your nose in every few months.

Love,

- Vacri

View on HN

Agreed. I stopped using Logstash for about a year and used it for a bit about a month ago. Awful experience. Awful documentation. Deprecated shit everywhere. Inconsistent stackoverflow information and TWO external websites too help make logstash actually functional. Oh, and since Logstash is a Java based application - would it hurt to give some java stacktrace log parsing configs?

Also, their shitty Debian repo management resulted in a bug that caused my company to lose $30,000.

The world needs more ELK hate.

It's not Java, it's Ruby. Which is even worse, because distribution tarball with logstash weighs ridiculous 71MB (?!?) and requires JVM to run (?!?) (or at least nobody talks about it being runnable with MRI).

> Also, their shitty Debian repo management resulted in a bug that caused my company to lose $30,000.

Well, this is not their fault that much. If you had put any thought about using repositories, you wouldn't use random packages from random sources over which you have no control and no trust with regard to package retention policy or packages quality.

Or maybe you would happily install also MongoDB from Mongo's site?

Elasticsearch is Java, and the logstash tarball includes it and kibana from memory, so you can run it as an all-in-one where logstash launches it's own Elasticsearch.

While I don't have the tiniest font in my terminal, I still couldn't read the entire Elasticsearch process line in htop, even when I'd stretched the terminal all the way across three monitors! The middle one was an ultrawide! I really wish Java would stop using arguments instead of storing config somewhere...

> Elasticsearch is Java, and the logstash tarball includes it and kibana from memory,

Not really. It's just logstash, along with some plugins (and what the heck are Maven bindings doing there?).

ElasticSearch is another 29MB compressed, which is fine for a database-like thing, and Kibana 4.x takes 30MB compressed (150MB uncompressed, of which not the biggest part is Node.js copy).

Got me on the ruby part. Meant to say JVM. My point was more to the fact that their site's incredibly short on examples.

FWIW, I inherited a thrice-removed infrastructure in a latency-sensitive environment. It was fun ;)

Logstash was installed on a couple machines the night before I was going to load some configs from my machine. I didn't load anything yet, except that logstash-web was running at 2s uptime. For some reason, (with upstart's help), the logstash installation included a broken webservice with a bad config. It got stuck in a JVM-birthing restart loop that was enough to cause significant impact. It happens.

(Funny enough, same place used mongodb. We ran into an issue once where database cleanups failed once mongodb reached around 50% of the disk. Local copy cleanups are fun.)

it's JRuby: best of both worlds LOL

My favorite thing about logstash was when the XML parser would choke and logstash would just quietly stop processing events.. I wish it was engineered as well as riemann.

My favourite thing about logstash-forwarder was that it was apparently an actual intentional design decision to shut the service down if none of the log files it was watching moved in 24 hours. I'm still trying to figure that one out - a quiescent service... shuts itself down... by default... to what end? Ended up having to cronjob it to restart every day. If you wanted it not to do that, from memory you had to rebuild it from scratch.

I've never actually come across that in a service before, one that shuts itself down due to inactivity. I mean, they must be out there, but I never would have guessed it'd be intentionally designed into a log shipper.

As a replacement for Logstash, check out Flowgger: https://github.com/jedisct1/flowgger and Heka https://github.com/mozilla-services/heka

Their renaming of ElasticSearch to Elastic really irked me. If only for the simple reason that searching for questions related to them on Google is harder now.

They could have gone with "Elastico" or something easier to trademark. I think that's why they keep saying "Elastic Stack" instead of just "Elastic".

Can you rally trademark Spanish words?

Probably. Trademarks are contextual. Two companies can both trademark the word "Apple". One in the context of computers, another in music distribution. So long as they don't get into each other's business, it's OK. Hmm... Yeah, Apple Records wasn't too happy with iTunes.

I maintain some Ansible roles, as well as some well-worn examples of ELK in production... And this marks the fourth time I'll have to basically rework _everything_ due to Elastic changing up the core architecture, naming, logos, etc.

I'm also less-than-thrilled with the performance for larger deployments. Having to babysit my log aggregation infrastructure (average about 50-100 events/sec across a few dozen hosts) is not very thrilling.

Have you had a chance to look at Fluentd? (Disclaimer: I'm a maintainer) Elasticsearch is far the most popular use case for Fluentd (much kudos to both the community and Elastic), and it's been pretty stable for awhile now.

No, I haven't yet, though I have heard the name here and there. I'm not going to be looking at logs again for a few months, but thanks for the pointer.

Then remember about Fluentd. I was using it for filling ElasticSearch with logs when Kibana was still written in PHP by a separate developer and was meant as a somewhat specialized frontend for logstash, not a generic one for ElasticSearch. I'm using it still for shipping logs (again to ElasticSearch) and metrics collected by monitoring, and I have even built a proof-of-concept of a stream processing engine for monitoring out of it.

You could go really far with Fluentd using or writing only few plugins.

I gave it a try today. Wanted to send process stats to graphite. Installed it on Ubuntu Trusty, installed the graphite plugin and the process watch plugin and... The td-agent couldn't start watch metrics plugin was incompatible with the 0.12 version of fluentd. So I resorted to collectl.

Sorry to hear this. Can you give me the exact name of the plugin? I'd make sure to look at it.

Does ELK = Elasticsearch + Logstash + Kibana?

Yep. Elasticsearch is often used in other projects, but logstash and kibana are generally used in tandem with each other and Elasticsearch.

Reply on news.ycombinator.com