Hacker News Reader

Comment by Flimm | Hacker News Reader

I would say that we are moving similarly, and I like the way Benji Weber put it:

- From continuous integration to continuous deployment

- To project ownership (although there is some resistance here on the part of some software devs, who don't feel it's their responsibility)

- Monitoring first: we have a strong desire to increase this but we're a bit at a loss as to what a good light-weight solution is here

The only exception is mob programming. Instead, we still have developers code on their own, and we switch to pair programming when troubleshooting a tricky issue.

View on HN

I think it's totally awesome that some groups are doing that. I just wish we had a set of these practices which were compatible with the realities of shipping bigcorp and gov contracts.

- Continuous deployment: sure, you're continuously deploying, because it takes a year and 23 signatures to get a release greenlighted into production.

- Project ownership: you can own any part which wasn't nailed down in the contract before the first line of code was written, meaning almost no parts at all.

- Monitoring first: you can come monitor as much as you want, if you fly over to another country to sit behind a screen logged onto the private network on which the app runs.

We do continuous integration in our dev environment, and ofcourse not all production environments are that walled off, but I've seen examples of such extremes many times, and it seems endemic to large-scale organizations.

You could pay for a NewRelic account, or invest the time in getting statsd+graphite or Riemann infrastructure set up. There are other systems too; it's a growth area.

"Monitoring-First Programming" to me means that monitoring and capturing errors in production is very valuable indeed. It's not a reason to abandon regular testing activities, but it is basically the same kind of thing as testing.

Links: http://newrelic.com/

http://www.kinvey.com/blog/89/how-to-set-up-metric-collectio...

http://riemann.io/

I believe it's a mistake to put Riemann in the same sentence as statsd/graphite. It really does not try and solve the same problems, it's more focused on alerting than trend analysis. In fact it recommends pushing data out of Riemann and into Graphite to perform long term trend analysis.

That said, if you have the time to invest in grokking Riemann (learn Clojure first), it would be a fantastic tool to base the rest of your monitoring infrastructure upon.

You are right, sorry if it wasn't clear - I had in mind Riemann as a replacement for the statsd part of that duo. That's how I've seen it used but my knowledge and experience are limited.

> Monitoring first: we have a strong desire to increase this but we're a bit at a loss as to what a good light-weight solution is here

Prometheus is designed with this in mind. Instrumentation is easy to add and takes care of things like concurrency and state maintenance for you, so you can sprinkle it anywhere safely. http://www.slideshare.net/brianbrazil/python-ireland-monitor... goes into this a bit more.

> Monitoring first: we have a strong desire to increase this but we're a bit at a loss as to what a good light-weight solution is here

In my experience, a toolchain consisting of statsd => collectd => graphite. It will, assuming you have never used it before, consume about man-week to get it installed and tuned to your needs, but with benefits lasting for years.

Is it better than the paid tools out there? Probably not, but it's all open source, relatively stable, and has low cognitive overhead.

Reply on news.ycombinator.com