Comment by Jeaye | Hacker News Reader

Comment by Jeaye | Hacker News Reader

Parent Full thread

Java, especially JAR files, can be quite light weight. However, JVM environments, and development with Java and Clojure, can be very heavy _and_ slow.

For Clojure, starting `lein repl`, takes 16 seconds on my 2012 Macbook and 9 seconds on my similarly-aged Dell laptop, both with SSDs and i7 quads.

Regarding memory usage examples, the base memory usage of a Google App Engine instance running the move trivial Hello World Java program takes around 140MB. Given that the default F1 instance has a soft memory limit of 128MB, it becomes clear that the JVM is working against you in both cost effectiveness (the price to spin up new instances as your existing ones are already above the soft limit) and latency (since spinning up instances is slow). Add Clojure on top and the problem certainly doesn't get any better. As an added annoyance, which is specific to App Engine but a result of using the JVM, it's impossible to specify JAVA_OPTS, so any of the -X flags, without switching to the Flexible environment.

As a result of both of the above, choosing Clojure for developing on App Engine, as my specific example, has had the serious downfall of slow development tools and memory issues out of the gate on my instances, causing me to pay more for a beefier instance class. The REPL is really hard to beat, but the combination of JVM and Clojure are the biggest pain in the ass, with this stack.

Yes, the startup is slow. But nothing afterwards is. The JVM gets bad press because of startup time while in reality that hardly matters.

In production, what matters is the ridiculous memory usage. Both Java and App Engine are to blame about this, but the Python and Go folks aren't running into the same issue.

If memory usage matters to you, have you tried telling the JVM not to use all the memory? Have you tried any of the tuning options? Can't say I've ever used app engine, but the pure java applications I've worked on did in fact use a fair bit of memory, much like windows and OSX will aggressively use spare memory for caching. Then I use -Xmx to tell it not to use all the memory and now it's much better. For a server, I'd say that this is both expected and the correct behaviour.

I have, and gave up after a month of touching every couple of days trying to find the sweet spot. Someone more familiar with JVM internals would have probably succeeded - but somehow C Python Go and even OCaml runtimes don't need this level of tuning in my experience.

JVM deployments tend to assume nothing else happens on the same machine, in my experience.

You last point is probably true. In every application I have ever deployed I only ever set the Xmx and J-Server settings, I never faced any problems. You need to give the GC some breathing room though.

> As an added annoyance, which is specific to App Engine but a result of using the JVM, it's impossible to specify JAVA_OPTS, so any of the -X flags, without switching to the Flexible environment

I haven't used App Engine either but I suspect that the flexible environment is more expensive.

I use -Xmx and then the process crashes because it runs out of memory. It's very wasteful with memory and I wish jep 169 was implemented to mitigate it. It's unfortunate that it was written in 2012 and remains a draft.

Well, I don't know what you consider "ridiculous", but I run a web/application server with significant complexity (written in Clojure) in a 2GB RAM VM, with the JVM taking up about ~800MB. I'd say that's pretty reasonable and it meets my production/business needs.

Considering that RAM is the primary cost in hosted environments, I'd say that 800MB is anything but reasonable. In fact, as a guy coming originally from a non-Java background, that seems downright opulent. Perhaps that's fine if it's the only thing running on that machine, but is that very realistic? A database, some daemons, cron jobs... It seems uncouth to me that the business logic should take up as much RAM as the data itself.

I run a fairly complex Clojure app and it uses 1.5GB of RAM all told. Factor in cron jobs, caching, DB flushes and other periodic spikes on the machine and whoops! we're over 2GB used. To prevent thrashing and OOMEs, I sized up to a 4GB VM, doubling my monthly costs (there are several of these boxes).

Now yes, I can go through and swap out libraries to slim the beast down. But that would mean rewriting large chunks of it, since so few Clojure libraries are interoperable and most simply wrap "enterprisey" (and heavy) Java libraries in sexier syntax. And if I'm rewriting it, I might as well avoid the whole mess and pick an ecosystem with a more streamlined standard library.

It's tough. I adore Clojure, but the combination of Clojure+JVM has made deployment and management less fun and more expensive than necessary. The JVM is awesome, but, just as it's not the hog so many claim it is, it's also not sleek.

Reply on news.ycombinator.com