I'm kind of surprised it took this long for someone to do this. It was clear very early on that the JVM was a bad match for what Spark was trying to do.
I attended a talk at Strata a few years back by a Spark committer who was talking about how Spark was stretching JVM memory allocations far past how the JVM was originally designed. Do a couple searches for "spark JVM OOM" and you'll see some discussions about similar things.