room271

> Actually non-blocking logic becomes really easy, as you can always shove an immutable value into an atomic reference (note - I'm not saying "wait free", which still takes a lot of work :))

Why the atomic reference here? I know that provides CAS but if we're talking about a single writer aren't you okay to just replace things anyway?

I'm genuinely curious about this, and I don't know anyone who works at Amazon, but if they really do have such a bad work environment and low tenure then how are they managing to consistently produce such good products (I'm thinking primarily of AWS here)?

This is a fascinating article. Some innovative stuff to deal with the difficult problem of how to innovate without breaking existing customers.

Some highlights:

* when a client first makes requests store their version, so that they don't have to worry about it * decouple features from versions, by using 'gates' (allowing for far more versions than would otherwise be possible)

I really recommend you do whatever it takes to find close friends who you can talk to about this stuff if you don't have that already. Whether that means joining some kind of help group, church, whatever. Other people are a great normalising influence - they correct our negative thinking, but also help normalise shame and other stuff because it normally doesn't feel so bad once it's shared (and because we realise other people are just as screwed up as us, or at least, have problems of their own). Plus friends help us focus on others rather than ourselves.

You're right that HTTP is not suitable for a wide-range of tasks. There are, however, very real alternatives which people use all the time to better meet specific needs.

Wikipedia has a good source of protocols used across the Internet - you'll recognise a lot as they have significant usage:

https://en.wikipedia.org/wiki/Application_protocol

Yeah, I read the title and immediately thought the Guardian had messed up!

In fact, they simply upgraded their version of Play and yielded some performance wins (which the article then explains more fully).

Persistent data structures allow for the appearance of cloning (you have a new thing) but are much more efficient behind the scenes (you don't actually copy everything). So if you find yourself doing a lot of defensive cloning, then Mori or this library would probably be a lot faster for you.

I think people are being harsh in the down-votes here.

I suspect most people (HN people) hold pretty strongly the belief that everything can be quantified. However, within reasonable limits and current stats and science, lots of things in practice cannot - yet at least.

A lot of advertising probably falls under this. To verify effectiveness we'd need, as the article notes, to control for so many other factors that it would often be too difficult.

So we often guess, or leverage past knowledge and apply it to what we hope are similar circumstances.

If you take the point to be that marketing is not a hard-science, than it's true. Some cases sure, but plenty not.

The 'Nic' he quotes in the article is me :) This is seriously the highlight in my career folks!

Although to be fair, he quotes me generously. Later on in that discussion I give up my wisdom and get confused again.

I feel like my original comment was a bit harsh so thanks for replying.

Re: the debate about CA vs CP, I've seen this too, but I think it reflects confusion rather than genuine options. The 'P' of CAP takes effect whenever you have two or more processes which clients can communicate with. So even if you are running within the same datacenter when using ES, provided clients can communicate with more than one ES node, CAP still applies.

Unfortunately, this guy doesn't understand the CAP theorem at all.

Once you are distributed, P is not an optional. Rather, in the case of network failure, consistency or availability is what suffers. A system cannot be both CA and distributed.

So, for example, Elasticsearch is not a 'CA' solution despite the diagram in the article, but is actually closer to PC (although, in practice it is far more subtle than even that as it is not perfectly consistent, and configuration options allow for some trade-offs between availability and consistency in the case of communication errors.

The docs are pretty good and have been for a while:

http://www.elasticsearch.org/guide/

Admittedly they were pretty poor early on, but they've matured with the product.

As a member of the younger generation, I rarely think in imperial and I think the same is true for most of my social group at least.

For example:

- I struggle to visualise feet or inches, we always used metric units at school so metres and cms are what I understand - kilograms for weight is much more natural to me than stones or ounces (really, I have no idea how much an ounce is, although I kind of know a pound is a small bag of sugar-ish) - miles are probably the only imperial unit I'd use, but even still, I find km much easier to reason about, perhaps because I have lived in Europe in the past. - oh, and the trusty pint of course!

While Piketty is quite polite, there are some strong criticisms of the FT team in here. For example:

'The FT journalists evidently did not read carefully the technical research papers and excel files that I have put on-line.'

In the vast chasm between pure monopoly (one supplier) and the perfectly competitive market (which is imaginary), is large room for firms which have competition but are still big enough to abuse their market position.

It's not useful to focus only on pure monopoly. From an economics perspective, there are plenty of other market structures which can be damaging for consumers.

I have to disagree with this strongly. There are significant economies to scale in any streaming service. For example, in terms of costs (streaming services require a lot of hardware). But also in terms of branding and reaching a critical mass in terms of content.

I submitted this link 6 hours before the OP but it didn't gain much traction. I though Hacker News had a way to prevent double submissions?

https://news.ycombinator.com/item?id=7772074

Surely this is a massive competition concern and should be blocked on such grounds?!

It doesn't matter whether a country is rich or poor, but whether people are rich or poor. And the USA has a lot of poor people.

I agree that using Elasticsearch as a primary data store seems risky. The situation is improving though; the v1 release introduces easy backups (for example, to s3) which from my experience work very well.

Prior to this backing up was a messy process.

Yeah, the Dynamo paper is very clear about the rationale behind creating DynamoDB and its usage:

- downtime (lack of availability) is very expensive to them, both directly and also reputationally

- traditional ACID stores don't scale with writes whereas an eventually consistent store can achieve this

So I completely agree with you in short: Amazon have great reasons to pursue approaches based on eventual consistency.

An interesting article and it does seem like people rushed to embrace NoSQL and are now trying to force it into some kind of consistency after the event - not a bad thing, incidentally - there's a lot of interesting work here (Vector clocks, CRDTs, etc.).

One thing that surprised me though was the lack of a key player: Amazon. Their Dynamo paper was hugely significant and as a company they use eventually consistent stores for a whole swathe of products at scale.

Why mention Facebook and Google but omit this other major player, especially are their experiences tell a different story.

You have to applaud this app. The title alone has value.

I mostly agree with this post, but I do wonder whether the difficulty of doing unit testing the OP describes is partially related to their use of RoR and the associated (complex) object-oriented design.

The main benefit of having an actual API layer is decoupling - between your front end clients and you back end service(s). This article seems to forget this major point.

Decoupling has loads of benefits - for example, it gives separate development cycles, and in this case provides a point of scalability too - not all REST dbs cache brilliantly or aggressively enough for all use cases.

The point about using delayed syncing between client and server is separate. I can see how it can help in certain use cases (Google docs doesn't auto-save every single letter you type in for example I suspect, but probably bulks them up into a save every x seconds) but at other times this kind of strategy sounds like it will add unnecessary complication.

This page crashes my browser (FF 28 in Ubuntu). Ooops.

Anyone else experiencing this?

I'm actually surprised at how short the list is. Presumably they use a lot more open-source projects than this?!

There is a new responsive site which is going to be phased in soonish (no definite time frame though). It's much better performance-wise.

Small webshops are also rife with this kind of programming environment. Mostly because their clients have no idea about code quality and the projects are small enough that you can still just about be productive with terrible programming standards.

It is, but it's definitely not a good way to treat new people to the team, especially people on their first job.

Much better to explain the bug/problem and let them fix it.