Hacker News Reader

Comment by nostrademons | Hacker News Reader

Wonder what this means for system software and application development.

There's a factor of 10-40x speedup by going from an interpreted language like Python/Ruby/PHP to a tight compiled one like C++/Rust/Ocaml. 2-4x going from a good JIT like V8 or Hotspot (or Go's runtime, though technically not a JIT). Probably another 10-100x by cutting out bloated middleware like most web frameworks or the contents of your node_modules.

All this was irrelevant when you could get your 2-4x speedup by waiting 18 months, and your 10x speedup by waiting 5 years. It's very relevant when your 2x now takes 20 years and 10x takes a lifetime. Maybe this is why Rust gets so much attention recently.

View on HN

I run a production Rust web service. The speedup for this service over using slightly stripped Rails was only about 5x. As you said, you can gain like 50-100x performance improvements from not using the default Rails JSON serialization and skipping ActiveRecord.

After that, you're lucky to gain 5x performance from re-writing the whole thing in Rust. Most of the hot spots of serving web applications using Ruby are already written as native extensions.

I think Rust is fantastic. I'm writing a tinyrb like "Ruby" VM in Rust at the moment. But... it's just not worth the hassle for plugging web services together. Maybe if you're at Google scale and already have web services in C++ it'd be a good choice.

Let's not forget that 5x is actually a big difference, especially in terms of the end user... You can also avoid much more near-the-edge performance issues/instability.

As you alude to it may not be worth a 10x increase in developer effort to try and do a bunch of rewriting when you could do something (as you said) like just replacing the bloatiest, slowest parts of rails and get large performance gains.

Have you written anything about this? I'd love to read more. As a person that is neutral/dislike on ruby and full-on hates rails, I'd love to hear from someone who's experienced running something as new as Rust in production alongside a rails application, especially from someone who likes ruby.

Right, but you can also get that same 5x increase from writing something using a higher level language running on the JVM or BEAM. You don't need to go full on "systems programming" to have something that outperforms Ruby.

MRI Ruby and BEAM are both bytecode VMs. There's no simple 5x increase in performance there. BEAM is register based, and YARV is stack based, so BEAM tends to be a little faster.

The JVM is a completely different beast. Thanks to improvements in the JVM for hosting dynamic languages, JRuby is able to offer some serious performance advantages over MRI Ruby, and I'm really excited about TruffleRuby.

Just to confirm why this is getting downvoted -- I'm guessing the MRI vs BEAM statement is super wrong? I can't say I know much about erlang's VM but being "bytecode VMs" I don't think MRI the same as BEAM, nevermind the primary paradigms they service being completely different.

I assume JRuby is faster than MRI Ruby (at least for parallelizable tasks), so I can't imagine the second statement being wrong per say... Also I'd never heard of TruffleRuby, and call me prejudiced, but I don't want to be anywhere near anything developed by Oracle, whether it's oracle labs or not (I hadn't heard about GraalVM before).

Speaking of fast ruby... I'm surprised no efforts to make ruby faster/swap out MRI haven't panned out. In python land I know there are efforts like pypy and pypy+stm tackling the slowness and GIL problems while maintaining axiomatic use of the language.

I'm not sure why this is downvoted so hard because it's true. BEAM has massive advantages over MRI Ruby/Cython for concurrency and parallelism but not really much in sheer throughput.

For example, Sinatra + Sequel on MRI has a 50% lead in throughput over Phoenix + Elixir on BEAM here: https://www.techempower.com/benchmarks/#section=data-r15&hw=...

Bytecode VMs all have the same fundamental problem with instruction dispatch overhead, regardless of what language or paradigm they're supporting, which is why JIT is so important even though it's so much additional complexity.

JRuby is now 2-3x faster in general as the JVM improved support for hosting dynamic languages, and it's in heavy production use at places like TalkDesk, so in that sense it is the "faster Ruby".

There's also Topaz, which is Ruby built using the underlying framework of PyPy, but performance is disappointing AFAIK.

5x is an amazing speedup! How much latency did that strip out for end users?

Almost nothing in the context of the long-running analytical queries the web layer has to wait on the DB for.

How much did this save you in energy and hosting fees?

Not at a large enough scale for either to be relevant.

But if you are talking about a stripped down Rails without ActiveRecord, we are looking at something on par with a Sinatra service. That is pretty lean already. Compared to Rails you already have a big performance improvement.

5x overhead for a Ruby webservice on that level is decent. I wouldn't have expected much more.

I find the fact that anyone can speak dismissively about a 5x speedup disheartening. Has anyone ever done a study on how much CO2 we are emitting in the name of "developer productivity"?

It's probably less than you think. Humans - just by virtue of existence - produce a huge amount of CO2, both through the air they breathe, the meat they eat, the automobiles they get to work in, the heavy machinery used to build those roads & buildings, the manufactured goods they consume, etc. And the CO2 cost of a developer isn't just that one developer's emissions; it's also those of all the support staff needed, from managers/admins/HR at work to the food service workers that serve them meals out to the doctors/lawyers/therapists and other service providers they visit to the parents that raised them.

It's almost certain one developer generates more CO2 than any reasonable number of servers that run their code. Anything that reduces manpower costs is a net positive for emissions. Besides, when the equation changes (say, when the software enters maintenance mode but the servers stay up), they'll be a strong economic incentive to spend the developer time to rewrite it more efficiently.

It's almost certain one developer generates more CO2 than any reasonable number of servers that run their code.

I'm not so sure. Let's do a back-of-the-envelope estimate.

Assume a single really hefty server that consumes 1 kilowatt. Over one year, this is about 10,000 kw hr. 1 kw hr of electricity produced by a coal fired plant generates about 1 kg of CO2 (https://carbonpositivelife.com/co2-per-kwh-of-electricity/). Thus that big server running for a year produces about 10 metric tons of CO2.

An average American lifestyle (all in, total country production divided by population, https://www.theguardian.com/environment/datablog/2009/sep/02...) involves the production about about 20 tons of CO2 per year. So if you write code that full-time on more than 2 really big servers per year, your code might be producing more CO2 than the rest of your lifestyle.

I'm guessing that most of the errors in this are probably overestimating the code's CO2 (probably not coal fired, probably less than 1 kw, a year is less than 10,000 hours), so more realistically maybe it's 4-8 servers to be break-even? Still, I think it's fair to say that there are some participants in this forum whose running code probably generates more CO2 than the rest of their lifestyle.

Something like 2% of the US power consumption goes to data centers.

But think about this... how much CO2 would you spend getting developers to work every day so they can reduce CO2 somewhere else? I suspect the biggest users of DC power, in terms of code, are already written in languages like C++.

Not having used Rust, I'm wondering how it makes writing a VM easier or harder?

On the other hand, software often follows the Pareto principle: 80% of the time is spent executing 20% of the code [^1]. Optimizing/migrating just that 20% would yield most of the benefits of a complete rewrite. (Good luck if that 20% resides in your framework of choice, though).

[^1]: https://www.codeproject.com/Articles/49023/The-impact-of-the...

The evils of premature optimization are well known but from my experience, execution often isn't the bottleneck, memory is.

Which mean you can get a lot of benefits by optimizing your data structures. Making them smaller is a good start. Unfortunately, changing data structures can have far reaching implications, sometimes justifying a complete rewrite.

For example, going from string based hashes to raw C structures can yield massive improvements, but you need to rewrite most of the code that access it, even parts that are rarely used.

Another performance killer are deep copies. This is also hard to optimize out because if you switch to a more efficient reference, you need to mess up data for the rest of the program in every part that uses it.

Price for compute is still going steadily down for web services. Which I think is more relevant than individual core/CPU performance. CPUs are still getting cheaper, blades are getting more compact. Virtualization, containers and function-as-a-service are improving utilization. Modern cloud service providers have massive economies of scale.

Yep, we are seeing computers evolve from hybrid media consumption/creation/business tools into a bifurcated model where IaaS providers rent out the majority of the world's compute power, and average consumers own a (powerful) thin client for UI/display/consumption.

There will be still sizable markets (real-time industrial/automotive/medical/etc., gaming, some business apps) where individuals will still want to own and upgrade their own beefy on-prem hardware, but in the consumer realm I can't think of many new applications that have taxed my 2012 MBP, or even my older Dell/Win7 laptop. AWS/Azure/GCP shoulder the burden for me.

That only helps for embarrassingly parallel tasks, like rendering hundreds of distinct web pages per second (which in many cases might just as well have been partially precalculated to trade space for time).

As was pointed out in the chart, we’re already hitting Amdahl’s law with half a dozen cores.

I believe what you're seeing is market competition, not reduction in hardware costs.

The total cost to offer compute has actually gone up significantly over the last year or so due to memory prices shooting up. I believe the cost of processors has gone up too, but not as drastically.

It's all about implementation. I got a ~100x speedup by going from bad Fortran to average Go (notice, both compiled). And a ~10x speedup by going from bad Go to excellent Python (compiled -> basically interpreted).

So yes, compiled languages can give you performance benefit, but it's not guaranteed, you need to work for it.

What is your app?

There's such a thing as being penny-wise and pound-foolish, though.

Imagine if I told you I had a desktop application that was slow, and I'd achieved a 2-4x speedup in common tasks by rewriting the whole thing in a different language. But then you discovered that I'd written it in such a way that it had to hit the (spinning-platters, just to drive the analogy home) disk to do even the most basic things. You would, I hope, tell me that I'd wasted a ton of effort optimizing entirely the wrong thing, and that my 2-4x speedup from choosing a new language would be blown away by the likely multiple-orders-of-magnitude speedups I could get from better memory management and I/O patterns.

In the web world, most discussion of backend languages is like this. Sure, you could get your impressive-sounding speedup from switching to a "faster" language. But the time spent executing application code is so utterly dwarfed by the time you spend idle while waiting on a database, or by the time it takes to send things over the network to the client, that "switch to a faster application language" should probably be the 1000th item on the list of the first 1000 things you do to try to improve performance.

Maybe, maybe not.

What does the profiler say. This is always the first question in optimization, if the profiler comes up with anything you can fix, that fix is always your biggest bang for your buck. If the profiler doesn't really say anything we are in a tough spot. There are things that are worth doing anyway, you have to be careful, in most cases even if the change is really faster it probably will not be fast enough.

Back to your example, maybe we cannot fix the database, but even if the database query is 10 seconds, going from 10.5 seconds to 10.001 seconds response is an improvement.

Your scale is an important question. Facebook can save tens of thousands per month with an optimization so small no user will notice it - between not having to buy as many servers, and not having to pay as much power to run the CPU for those extra cycles. (Facebook will not give you the actual numbers, but they can tell you they have measured and you can read between the lines to guess how much it must be given they have a few people employed in small optimizations)

In web apps, the first thing you do is start looking at DB queries. How many, how complex, etc. Reducing the number of queries and making those queries as simple/fast as possible accounts for at least the first quarter of my hypothetical "first 1000 things you do to try to improve performance" list.

Also, regarding scale: the number of entities operating at AmaFaceGoog scale is small. The number of entities operating even within an order of magnitude or two of them is small. The odds are very much against any advice relevant only at that scale being relevant to the average Hacker News reader.

The idea that a webapp should be a frontend to a RDBMS is itself an artifact of the "time to market is way more important than computational efficiency" culture of the mid-00s.

You can get 1000x speedups by ditching the database and serving out of RAM. Sites like Hacker News, PlentyOfFish, Mailinator and Google (back when the webserver was written by just Craig Silverstein rather than a team of hundreds) serve thousands of queries per second off a single box. In most cases the actual access patterns of apps don't map particularly well to either an RDBMS or a key-value store, so if you're willing to put in the time to develop a custom datastore, there are still large efficiencies available.

I think the death of conventional Dennard scaling and Moore's law is part of why interpreted super-dynamic languages like Ruby have lost their luster. We can no longer just ride CPU performance increases to get increasing application performance, so these languages' implicit slowness becomes a liability that won't just go away magically.

My optimistic hope is that the death of easy CPU performance gains will lead to a round of serious evolutionary improvements in software architecture. Rust is probably a sign of that happening.

> interpreted language like Python/Ruby/PHP

There is no such thing as an "interpreted language." An interpreter is a class of programming language implementation. The languages you listed do not even have interpreters as their standard implementations; they have bytecode virtual machines. There are alternative compiler implementations for Python and PHP.

Everybody says this - believe me, I was one of them when I was younger - but if you've ever tried to write a compiler for Python or Ruby, you'll understand why they're often called "interpreted languages".

(Interpretation vs. compilation is a continuum, anyway; even a compiled language like C uses a runtime for several operations like malloc or strings, while modern JITs like V8 or PyPy can compile a trace of methods and then fall back on an interpreter for uncommon cases. Nevertheless, there's still an important distinction there: in a "compiled language" like C or Rust common syntactic forms like addition, property access, or function calls can semantically map to machine instructions or memory locations fairly easily, while in an "interpreted language" like Python or Javascript, even a property access or arithmetic operation may invoke an arbitrary, not-predictable-at-compile-time piece of code, and hence require runtime dispatch.)

> but if you've ever tried to write a compiler for Python or Ruby, you'll understand why they're often called "interpreted languages".

Why, I happen to have hacked on Lisp implementations. Including Thinlisp, which is a compiler for a non-garbage collected real time subset of Common Lisp. And what you say is still wrong.

An interpreter is not the same as a bytecode VM, and is not the same as native code compilation. And none of those things are a property of programming languages. The only programming languages where that might remotely be true are purely string-based ones like TRAC and maybe some of the term rewriting ones.

> even a property access or arithmetic operation may invoke an arbitrary, not-predictable-at-compile-time piece of code, and hence require runtime dispatch.)

If that's the criterion, then vtables make C++ "interpreted."

> If that's the criterion, then vtables make C++ "interpreted."

So do function calls; you don't know until the .o is linked where a read call will go. Unix C library? Or some local override of read?

And, speaking of vtables, modern shared lib calls give obj->virtual(args) a run for its money in terms of overhead. It's worse because a vtable's structure is determined at compile time, so it's just positional referencing; a shared lib call has the referencing through a table plus string name lookup to figure out the offsets at run-time (at least the first time through).

There's a reason why C++ requires the "virtual" keyword on methods that may be overridden...

What that means for software development? I hope the lazy excuses to NOT write performant software will die out and software gets fast again. remove the hundreds layers of libraries and frameworks and build good fast software again. we will find other ways to manage complexity and software reuse.

Death of Moore's Law really is the best that could happen to us...

This assumes compute costs comprise a sizable part of the budget of a software company. If a company can save a couple of percent per year by doing a complete rewrite in Rust or C++, the return doesn’t make the cost worthwhile.

Reply on news.ycombinator.com