Comment by baybal2 | Hacker News Reader

Comment by baybal2 | Hacker News Reader

I remember hearing about it many years ago, and am surprised to find it alive, and well now.

I always felt that the web people were never about performance, since there were many "pure C" webdev attempts before without much success.

Nginx can very realistically handle 1-2M requests per minute on commodity hardware, and no customisation, and that's from the disk.

Were somebody really serious about web performance, I think going from millions of requests per minute, to millions of requests per second is 100% possible.

I worked on this problem around 6 years ago, when I had a task of squeezing HTTP, and network perf on some API servers close to hardware limits. The task was mostly about gluing DPDK to popular software: nginx, memcached, postgres.

I am very enthusiastic to see Libwebsockets getting glib support. Glib is a one of a kind piece of software in the C ecosystem with which you can adopt modern programming methods, and in general approach it as you do it in a big "platform" like environment like NODEJS. Glib is really undeserving neglected, and overlooked.

On a 5 year old Intel Xeon, nginx was able to serve a 1 kilobyte static file over TLS at 1.2M requests per second. It's definitely possible already.

Coding big web backends in C is extremely expensive. I'm not sure it's worth the tradeoff for most use cases

I am not talking about whole backends, but task specific individual APIs, especially for performance sensitive tasks.

You can get within 20% the performance for 1/10th the cost by using a modern fast language (rust, d, nim, go), and within 50% for even cheaper by using c# or java.

I really doubt.

Of my short encounter with "modern" webdev, I found that running out of RAM was far faster than running out of CPU, even with every trick possible thrown to increase GC aggressiveness.

RAM is by far more discriminately priced on all these new "cloud" hostings, and has the most unpredictable performance change with size. Even on real hardware, going for high RAM servers is quite expensive.

While CPU, or I/O saturation naturally throttles itself, RAM exhaustion is rarely pretty, and hard to proof your software against. Most disconcerting about this is that your RUST, GO, or the TRUE ENTERPRISE JAVA®, don't really use that RAM at all. Most of "modern languages" RAM content is just zeroes, and empty buffers.

How is Rust cost more memory than C again?

Also as shown by TechEmpower benchmark, in "webdev" field, C/C++ doesn't have proper/official driver to Postgres with pipelined support, thus even lose to Java in the fortunes benchmark.

Nowadays, they are using a fork of libpq with batch API that has not been merged for 6 years in order to compete. So, lacking of good library support in "webdev" field will put C/C++ at extreme disadvantages compare to other "webdev"-friendly languages.

You'd be hard pressed to run into memory issues with nim or go, to list the easier languages to write.

You don't know what you are talking about. Rust doesn't have garbage collection and has a memory footprint very close to C.

Your GP didn't just mention Rust, they also mentioned C# and Java, so when your parent refers to GC the more charitable interpretation is that they were using C# or Java and responding to that part of the message.

Nim's ARC is one of the most lightweight memory managers.

You can make efficient systems with Java too.

Quote from 10 years ago:

> The system [LMAX] is built on the JVM platform and centers on a Business Logic Processor that can handle 6 million orders per second on a single thread. The Business Logic Processor runs entirely in-memory using event sourcing.

https://martinfowler.com/articles/lmax.html (2011)

Is there an example of a task like that? Specifically, ones that don’t talk to a database/files or another service, in which case the talker’s performance becomes irrelevant, unless it is really crawling. Most code “we” write is scheduling queries, glueing datasets together and jsoning the results into a socket through some stream library. IO takes 98% anyway, 2% rerouting and checks. Personally I’m fluent in a range of languages, but wouldn’t ever think of writing networking in C or a similar low-level environment. A mountain of work and skill for something expressable in just a few lines of python/perl/js/ts/lua/sql, zero economy. (Okay maybe an nginx plugin in a critical case when multiplying in instance costs doesn’t help.)

That's the crux of the issue. Most of webdev is just managing a huge amount of very simple pieces of code, where I/O from somewhere, to somewhere dominates.

One particular issue I remember when talking with Alibaba engineers when I worked on a subcontractor for a custom DC project was "1 second kill"

That's a phenomenon when some super good deal is posted onto the front-page of Taobao, and they get squished when people from all over China smashing F5 click on the deal.

Purchasing looks like a lock, and write task from the DB side, and the whole of Taobao.com was tied onto a single point of failure MySQL cluster in 2016 abused to the maximum.

They went to Computer Science people which only said that there is no way around locking, and a single DB write origin.

External contractors wrote a super-duper performant "database gateway" which organised, and queued purchase reservations to the stock database at around 50hz.

People confuse performance (CPU) and throughput (IO). The reason CPU usage is undervalued is that it's simply not the bottleneck typically. Very few people actually have the challenge of having millions of requests per minute. The people that do, can scale horizontally by just throwing load balancers and cheap vms at the problem. That gets you both bandwidth, memory, and CPU. Scaling horizontally is very competitive with whatever cleverness engineers can provide to reduce the need for that. Engineers cost a lot more to scale than VMs.

We all love to optimize but there is a point of diminishing returns. Scaling a cheap vm with CPU credits (i.e. it's not even supposed to have double digit CPU usage other than for short bursts) costs nothing compared to the salary involved with e.g. making something like that 2x or 3x faster. And the yields are terrible too, even if you succeed.

Say you are running 12 vms costing about 20/month. That should get you some CPU and modest amount of memory and would be something appropriate for a web server. Twelve of these means you have fail-over, multiple availability zones, and possibly even regions. Going from 12 to 6 would be a nice cost saving of about 120/month. Except now you have less bandwidth to go around and maybe a bit less resilience. That's a decent but not amazing freelance rate per hour. If you spend a week on implementing this, the return on investment (i.e. your time) would be 40 months (assuming an unreasonable 40 hour work week), or about 3 years a and a bit before you earn back the expense of your time. Now say that instead of spending that time, you simply scale to 24 servers: i.e. you double your cost from 240 to 480 per month. Or about four hours of your hypothetical hourly rate. Or about half a day. So a week of your time still adds up to nearly a year of simply running at 2x the capacity.

If you are any good, your rate might be higher and the tradeoff is even worse. Not even worth having meetings about. Using C for this stuff means hiring more expensive C developers and making a bad deal even worse. The smart way to get performance is to have those C developers work on the OSS infrastructure we all love to use. It will trickle down and get us decent performance elsewhere. Nodejs is actually built on lot of C/C++ libraries and benefits from a lot of cumulative optimizations that have gone into these libraries. That's why it is so competitive in this space. There are a gazillion other languages to consider with similarly good enough performance and throughput. C is a last resort when performance wins over security and stability concerns. Sometimes it does, but mostly it doesn't make sense from a cost point of view.

According to TechEmpower benchmark, C# and Java perform better in some tasks than C/C++ [1]

Even in categories where C/C++ are more performant, other languages are not that far behind.

If at all possible, we should not use memory unsafe languages for anything, especially something exposed as a server. No mater how careful you are, and with all the tooling available, majority of exploits in popular software happen due to memory unsafe languages.

[1] https://www.techempower.com/benchmarks/#section=data-r20&hw=...

Memory safety issues are exaggerated. It's definitely harder to keep big, complex software memory sanitised, but it's not something completely insurmountable.

You use C judiciously on the most performance demanding tasks, while trying to bring the overall task itself closer to some simple algorithm, on which you can later throw heavy verification, like formal verification, valgrind it to death, fuzzing etc.

The current wave of "new age" computer languages like Koltin, Go, Rust have a very noisy activist userbase which tend to extol some very simple, obvious things as ultimate virtues.

Memory safety issues are bugs. Do you know any programmer that does not occasionally create bugs? Don't forget tight schedules, low budgets, ...

Also rust is just what you propose that - a programming language integrated with heavy verification of safeness built-in. Because occasionally someone writes c code without using all available tools to verify the code it is better to have it built in.

> Do you know any programmer that does not occasionally create bugs? Don't forget tight schedules, low budgets, ...

Do not run programming in C on tight schedules, and low budgets. It needs tact, and understanding.

“Have you simply considered not having bugs” isn’t a useful strategy though.

Memory safety is not an issue if you actually learn to take advantage of the C toolchain. I've caught memory leaks and buffer overflows to great effect just by using Valgrind and ASAN. And for most applications, you can limit the attack surface by only writing C for the performance-sensitive areas and using FFI to call into those routines. As a bonus, it becomes much easier to unit test for logical corner cases.

This just isn’t true in practice. Can you point to a popular c project that’s accomplished this? I bet there are a few tiny ones that make such claims but haven’t received scrutiny.

IIUC, it needs extensive code coverage and it's even difficult for library (the lws case).

It helps 40 years of wasted advocacy how good programmers never do errors with C.

I can´t wait for governments to make security exploits liable, then lets see how much software will still be written in C, or derived languages.

People like you pointed to benchmarks in the past and said the exact opposite. It is sad that any kind of reasoning stops at some benchmarks

Benchmarks are useless if you don't understand what it is you're measuring. A contrived "Hello world" benchmark tells you nothing because your bottleneck is in the system calls, which has nothing to do with what framework you use. If you run a web service that requires a compute or memory intensive task, like a game server physics engine, or multimedia streaming, or anything that requires large-scale postprocessing, you're going to heavily rely on C or C++ based libraries to do the heavy lifting.

Reply on news.ycombinator.com