Elasticsearch: 10,000ft view

This isn't true, just fyi. Lucene actually relies on the OS' filesystem cache heavily. A Lucene index (e.g. an Elasticsearch shard) is composed of many small "segments". Each segment is an immutable file, which allows the OS to aggressively cache them.

But if your index is larger than available memory (very common), the OS will naturally start paging the segments on/off disk as required.

That's the root of the "give no more than 50% of your memory to ES" advice, since the other 50% is needed by the OS FS cache for Lucene segments

Source: I'm an Elasticsearch dev :)