Hacker News Reader

Comment by muraiki | Hacker News Reader

I think that you express a fallacy. You're saying that nobody can provide thoughtful criticism unless they take up the task themselves. So the next time you criticize politics, are you going to run for office? The next time you criticize a meal, are you going to become a chef? The next time you criticize a car, are you going to become an engineer? Yes, it's one thing to make thoughtless and uninformed criticism. But providing thoughtful and useful criticism does not necessarily require actively creating something from scratch. Otherwise, we would all only be starting new projects and never collaborating!

View on HN

No, he's spot on. The one thing that keeps happening over-and-over again is that people who have never put together a system of any complexity will yell from the sidelines how others should build their stuff.

This goes for operating systems, application software and advanced rocketry alike.

The reason we have so much C code powering the world is that (1) there wasn't anything better when that code was written, (2) time has weeded out most of the bad bugs to the point that the old codebase will be substantially more stable than anything new no matter what language you would write it in.

If someone is willing to throw a few hundred million or so at re-building an operating system from the ground up that is 100% safe and reliable and that does not have more bugs than what is available right now then they are free to do so. They're also free to donate their own time and to try to enlist their buddies. But they're not free to yell at others how those others should do their jobs.

Show don't tell.

> (2) time has weeded out most of the bad bugs to the point that the old codebase will be substantially more stable than anything new no matter what language you would write it in.

This seems to assume that the codebase is stable and in maintainance mode. However, the reality for the vast majority of open-source projects is that they're either being actively developed, adding new code and new bugs, or they're abandoned.

>The reason we have so much C code powering the world is that (1) there wasn't anything better when that code was written...

Right, but the criticism he received for his first post was not primarily about choosing the wrong language or about refusing to rewrite in a different language now. It was primarily about downplaying the role of C in the security issues that were found.

The rest of the debate, at least as I read it, was not so much criticism as it was a generic debate about the pros and cons of various programming languages like we're having one every other day on here and on reddit.

Choosing C back then was clearly justified for a project like curl given all the constraints and considerations. And for a rewrite to be a net positive in terms of security probably takes years, not to mention the huge effort that someone would have to put into it.

> If someone is willing to throw a few hundred million or so at re-building an operating system from the ground up that is 100% safe and reliable and that does not have more bugs than what is available right now then they are free to do so.

But at the same time - we are spending those billions. We keep throwing billions at Apple by buying their new phones, yet they release lists of security issues which this time were basically all:

  - malicious crafted image -> remote code execution

  - maliciously crafted font -> remote code execution

  - maliciously crafted audio -> remote code execution

I'm not sure if I have to be able to write a font parser and rendering subsystem in a memory safe language myself to be allowed to complain that Apple is using my money for something other than doing just that. To me it's completely mind blowing that someone uses a C library to parse complex third party binary data from the internet.

If c lib is mind-blowing, what about kernel and it's extensions?

For now c is the only thing we have that can be used so widely. Some day we may be able to use rust or something else, but that's future, not present.

Rust can speak C's ABI, so you can write kernel extensions in Rust. There's some legwork involved, but it's quite possible.

Interesting.

So you're saying that Rust, at the moment, in theory, can be used in any place where C is required? Ie. writing kext on macOS, loadable kernel module on Linux or things like extension for SQLite are all possible?

In that case I wonder if there are any theoretical blockers for things like (using no_std and) mapping internally used C structures (let's say BSD kernel's rbtree) into typed, first-class Rust citizens - to avoid std overhead and reuse what's already available? In other words - is there anything there in C macros that can't effectively be expressed in Rust for example?

That's correct.

There are no theoretical blockers; tooling could make it a lot easier though. I was hand-porting chunks of the Ruby interpreter, which uses C macros extensively, and it's doable, but not fun.

Any lang that easily calls C and can be called by C, or can be compiled to C should be fine to use today.

Re: kernel - it's a lot of pretty simple subsystems. Each system is usually simpler (easier to verify and polish) than a lot of common libs doing really complex things (ssl, video codecs, ttf, ...). Complexity + raw pointers quickly add up to guaranteed problems.

Rust uses llvm. Use it for new development on any platform that llvm supports. A few microcontrollers might be left out.

Exactly, show don't tell. And when I point to the fact that the Rust codebase itself has a lot of "unsafe" places I get the explanation "yes, but it's an old code and we haven't had the time to rewrite it to use the newer features that would allow more safe code."

Think about it: the people who design Rust don't have the time to rewrite their own Rust code to be safe.

When there's no time to rewrite the code of Rust to be demonstrably safe, why should any other project have more time than that, where the programmers know Rust much less or not at all?

Maintaining the successful project is hard and takes a lot of time, even without switching to something new.

Second point: look at the Benchmark game, a fast Rust code example also typically has an "unsafe" code. So it's still not even demonstrated that "you can have it both fast and safe" holds for these examples. Fast as in "as fast as C" and "safe" as in "provably safer than C."

Finally, I believe that the language should simply implement a language-level no-cost check for integer overflow, as the machine languages already have, and not complain that there aren't some special mechanisms in CPUs to do it differently. It's your language's failure if you can't use what already exists. The CPUs already can do it, the high level languages don't provide the high-level access to it. Which doesn't matter for C which is "unsafe" by definition (and where it's natural to call some asm when needed) but it should for Rust which goal is to produce a more provably "safe" code.

Writing unsafe Rust instead of unsafe C is not giving you any real advantage for the existing projects, as long as the state is as it is.

Also, look how Rust uses TLS. Is safely rewriting a crypto library which is useful in real project still a too big task to be taken?

I like the perspectives of what Rust could bring. But I'd like to see a real life case of its success too.

> Writing unsafe Rust instead of unsafe C is not giving you any real advantage for the existing projects, as long as the state is as it is.

Your argument hinges pretty heavily on this. It's incorrect. The reality is that unsafe code in rust is not like unsafe code in C. First, it's bounded to a module. Second, you can grep for it (huge). Third, you still have less UB in unsafe rust than in C.

I've never written code in rust that required unsafe, after many thousands of lines.

> Is safely rewriting a crypto library which is useful in real project still a too big task to be taken?

https://github.com/ctz/rustls

?????????????

> But I'd like to see a real life case of its success too.

Ripgrep is a nice one to point to. There's redox too. Tons of rust code out there.

> It's incorrect.

> I've never written code in rust that required unsafe, after many thousands of lines.

That illustrates the level of understanding of those who argue that curl should be rewritten in rust. Bravo.

Why, specifically?

It's self evident. One personal and very small example (many thousands is obviously less than 10 thousand) doesn't prove anything.

curl has an order of 120000 non-empty lines in c and h files.

The person who made statement of "many thousand" probably made toy applications, compared to something like curl.

More realistic would be to compare something that should have been made in curl: like the "ring" library you mentioned as a Rust answer to "openssl." Which is neither an openssl nor written in Rust.

It is far from providing the functionality of openssl. The author states that:

"ring exposes a Rust API and is written in a hybrid of Rust, C, and assembly language."

"ring is focused on general-purpose cryptography. WebPKI X.509 certificate validation is done in the webpki project, which is built on top of ring. Also, multiple groups are working on implementations of cryptographic protocols like TLS, SSH, and DNSSEC on top of ring."

Actually a wrapper, and actually much smaller than openssl.

Second, "ring" has just some 9000 lines in rs files but 13000 lines of C (as it is defined by the author to be a wrapper, not surprising). So it is still just a wrapper around the C, still orders of magnitude smaller than curl, and still unable to be actually written in Rust.

On that level, I guess the Rust people would call curl the Rust application as soon as a few .rs files would appear in curl code base. Or not.

It's simply has no sense, supporting Rust with misinformation instead of being honest about its current limitations regarding what's actually implemented and what actually doesn't use "unsafe."

It's obvious that those who suggest to others to rewrite the big projects in Rust should first show that they managed to make some much smaller safe code. Especially that they managed to make some meaningful full-Rust and safe libraries.

See, this is much better of a comment than the above, which was mostly an insult, with no substance.

(I don't agree with much of this but that's not the point; my point was trying to raise the level of discourse around here.)

> when I point to the fact that the Rust codebase itself has a lot of "unsafe" places

One of the criteria for being in the standard library is "is this something which needs a lot of unsafe." This is so it can receive extra auditing, etc.

> yes, but it's an old code

This may have been true a while ago, but in my understanding (I'm not on the library team) it isn't really any more.

> why should any other project have more time than that

Because they do not need to write their own unsafe code, since it's already done in the standard library.

Generally speaking, applications in Rust should almost never need unsafe. Libraries may.

> a fast Rust code example also typically has an "unsafe" code.

This is not really true anymore; in fact, recently some programs were contributed that are 100% safe yet are faster than their previous, unsafe-using versions.

> believe that the language should simply implement a language-level no-cost check for integer overflow

This is not viable. Not enough machines have this.

> Writing unsafe Rust instead of unsafe C is not giving you any real advantage for the existing projects, as long as the state is as it is.

Unsafe Rust still has many, many more safety checks than C.

> Also, look how Rust uses TLS. Is safely rewriting a crypto library which is useful in real project still a too big task to be taken?

https://crates.io/crates/ring ?

> This is not viable. Not enough machines have this.

Of course they have. It's a basic set of machine code instructions. Can you specify which doesn't please?

> One of the criteria for being in the standard library is "is this something which needs a lot of unsafe."

Do you honestly think a project like curl has everything it needs "which needs a lot of unsafe" already implemented in the current standard libraries?

Why don't you rewrite the curl in Rust then, it's just using what's already written? Let me know how far you come.

> https://crates.io/crates/ring ?

Does it have all the features that curl needs, or does curl still have to use a Rust wrapper around openssl, inheriting all the potential unsafety problems of openssl anyway? What is worth then, in percentage, having anyway less exposed part of the code in Rust? How many other wrappers curl has to use? Have you tried to check and estimate if it's still worth?

Once again, Rust people should first demonstrate that they can manage to make any relevant library "cleanly" safe, then complete the feature set, and only then expect big non-trivial programs consider using some Rust.

First make the basics safe, then let us use that basics. Once you have save openssl equivalent, even C programs can link to it and be much less exposed to the potential problems than they are now. It's that direction that only has sense.

> Of course they have. It's a basic set of machine code instructions. Can you specify which doesn't please?

That they exist is not the problem. That they exist _and_ don't provide enough performance is the issue. That is, your "no cost" is what I'm asserting here, not that it doesn't exist at all.

(This is not my area of specialty but was brought up when discussing this behavior; it's measured as a 20%-100% hit to speed, which is unacceptable.)

> Do you honestly think a project like curl has everything it needs "which needs a lot of unsafe" already implemented in the current standard libraries?

I don't think a project like curl needs much (if any) unsafe.

> Why don't you rewrite the curl in Rust then, it's just using what's already written? Let me know how far you come.

I did not suggest that curl should be re-written in Rust. I do think a functional equivalent in Rust would be good to have. I do not have the time to write such a thing myself. Luckily, many others have been working on this kind of thing.

> does curl still have to use a Rust wrapper around openssl,

ring is a port of BoringSSL to Rust, piecemeal. It still has some C code today, but at the end, will be all Rust/asm. Given that BoringSSL is a fork of OpenSSL, I would imagine that it would have enough functionality.

> That they exist is not the problem. That they exist _and_ don't provide enough performance is the issue. That is, your "no cost" is what I'm asserting here, not that it doesn't exist at all.

I'm claiming that the machine instructions exist to implement at no cost this high level construct (using C-like syntax):

    if ( overflows( c = a + b ) ) {
    }

What people say to produce slowdowns is if they want to implement

     try {
          a huge bunch of instructions where some are additions
     }
     on integer_overflow_from_wherever do whatever

The former is no cost if implemented, and exists in every CPU. The later doesn't exist, but who says that only later is what has to exist? Why not implementing the no-cost former?

> It still has some C code today

Exactly my point why curl has no reason to be reimplemented in Rust now.

> I'm claiming that

This is different than the claims that were made when we made this decision. Specifically, the cost angle. You can absolutely do this, but we don't turn it on by default due to its expense. See https://danluu.com/integer-overflow/ for example. From that link:

> Summing up, integer overflow checks ought to cost a few percent on typical integer-heavy workloads

That few percent matters. And the extensive discussion (in both directions) on HN: https://news.ycombinator.com/item?id=8765714

It would be reasonable to turn it on, but that's not the decision we made.

> Exactly my point why curl has no reason to be reimplemented in Rust now.

Again, I never said curl should be ported to Rust.

FWIW, the former "no-cost" thing does exist in Rust, in the form of "checked_..." methods on the various integer types, e.g. https://doc.rust-lang.org/std/primitive.i32.html#method.chec... .

Thanks! That's the good approach.

> Second point: look at the Benchmark game, a fast Rust code example also typically has an "unsafe" code.

That's false. From[1], it looks like two submissions use `unsafe`. And one of those is for ffi to gmp.

[1] - http://benchmarksgame.alioth.debian.org/u64q/rust.html

3 spectral-norm, 1 rev-comp

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

2 pi-digits (ffi to gmp)

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

OK. I only looked at the programs linked directly from: http://benchmarksgame.alioth.debian.org/u64q/rust.html

I think it's missing the point to count the programs that aren't the fastest, i.e. only one each of the spectral-norms and pi-digits are "interesting".

And apparently steveklabnik1 thinks it's missing the point to count other than "the latest version of each program".

https://www.reddit.com/r/programming/comments/62cx5d/addendu...

There are many ways to be selective with evidence.

As-a-matter-of-fact all the contributed Rust programs I listed use unsafe.

> Benchmark game, a fast Rust code example also typically has an "unsafe" code

So you say but do not show.

Please show us which of those Rust programs you count as "fast".

Please show us how many of those programs have unsafe code.

I have put together systems of considerable complexity (>100k lines), and am working professional programmer for almost 2 decades. Most of my work does not involve C, but various "safe" languages.

There were better alternatives already available at the beginning of the 70s, and certainly when most of the software was written that we are using today. Just to name a few I am familiar of with:

- Pascal being as old as C, and later on

- Modula 2, Oberon, the Oberon System was completely implemented in Oberon

- Eiffel

and of course, today:

- Rust

- Go

So, even back then, there were more safe alternatives to C around. C gained more popularity, and consequently the tooling gave it some edge, but there would have been alternatives. Of course, when you have a nice debugged C program, that is something worth keeping. And as a consequence, for mostly historic reasons, a lot of our current infrastructure is built on top of C - but that alone does not mean that C was the perfect or even the right tool to build all of this upon.

I wouldn't put it as strongly--I think valid criticism can come from someone who hasn't "put something together"--but at the same time I think a lot of the criticism in this area isn't all that valid. Some of it is, but mostly it seems to me just people parroting "c is unsafe" as though that is itself the end of the discussion.

It's not what I tried to say, but I see your point. I merely pointed out that the people who're saying saying "Why not rewrite curl in X" probably didn't stop to think how challenging it would be.

> The next time you criticize a meal, are you going to become a chef?

Well, they're not criticising the meal here, they're criticising the kitchen it was prepared in. :P

But joking aside, they don't need to rebuild all of curl in <language of choice>, just get it started, and if enough people think it's a worthwhile project, they will contribute.

> But providing thoughtful and useful criticism does not necessarily require actively creating something from scratch.

I agree. However, a lot of these comments are along the lines of "C is unsafe. Go rewrite curl in Rust!"[1] without realising how much effort and time that would take (and I'd not be surprised if some portion of this group never worked on a project in either language). I personally don't find those types of comments very useful.

> Otherwise, we would all only be starting new projects and never collaborating!

This is what the people I'm referring to are essentially asking. Going hybrid would probably introduce subtle bugs, and increase complexity and maintainability down the line.

[1]: I love Rust, and a few of my recent pet projects are being written in Rust (one of which is being ported from C). So, I'm not trying to bash Rust. I'd even encourage anyone to use Rust over C, if they were to start a new project.

It's one thing to criticize somebody over a missing feature or a bug. It doesn't take an engineer to author a valid complaint when a car door handle falls off. It's another thing entirely to criticize an entire methodology or to try to dictate exactly how something should be done.

I don't think this generalises that well. We're learning some general truths that should be known to many people with enough time. You can certainly criticise whole architecture and methodology based on those ideas. For some ideas it took a long time before they became something obvious - we may be at a time when the scales are slowly tipping.

For example imagine telling someone in 2000 that they should be running an automated continuous integration pipeline rather than purely manual testing. Now it's a norm. Then, we were still discussing whether Extreme Programming was a waste of time. But we also didn't have the tools to support the CI properly.

Memory safe compiled languages seem to me like the same thing. We had things like Cyclone, OOC, and other experiments which barely anyone knew. Now we have Rust which is still hard to use, but it's gaining ground. In a decade, I hope to be pointing out to this post when someone complains that you can't criticize an entire methodology. "Remember when people said the same thing about enforced memory safety?"

Curl will not be rewritten of course. But I'm happy people push this idea. The next curl will not be written in C.

> It doesn't take an engineer to author a valid complaint when a car door handle falls off. It's another thing entirely to criticize an entire methodology or to try to dictate exactly how something should be done.

But if a car manufacturer consistently uses a production methodology which causes door handles to keep falling off, again and again, even after they've been re-glued on for the Nth time...

Then I think it's fair game to say that you think there's something wrong with the methodology. People suggesting alternate approaches without putting in the actual work involved may very well be "backseat drivers of the Internet"-type commenters, but that doesn't mean they're not on to something.

If my comment could be construed in any way as an attack on people for suggesting others use different languages such as Rust, it was not intended. I'm a fan of Rust myself and I would recommend it to anyone who wants to start a new project.

My comment was meant to pertain to the suggestion floating around that Daniel ought to undertake a total rewrite of curl in Rust. That's an entirely different ball of wax. It's like telling the car maker to tear down and build a new factory because the door handles keep falling off.

>But if a car manufacturer consistently uses a production methodology which causes door handles to keep falling off, again and again, even after they've been re-glued on for the Nth time...

We know the characteristics of glue, and have switched to better glues. All our assembly jigs work with glue. When the handle falls off it is easy and cheap to glue it back on.

If we switch to screws we need to redesign the parts and the jigs; and reorder the entry assembly line. [Not realized by the engineers at the time is if they use screws dissimilar metals of the screws corrode and result in the handle falling off as well - and it is a much more expensive fix because you have to replace the handle and the part it attaches to as both have nothing to screw into)

If I say: "There are all these bugs in curl, you should rewrite it in rust and it will be better (but I've never done anything similar, nor there is any scienetific proof for what I am saying)" then it's not thoughtful criticism, but pure personal opinion. There is a very low value in that.

On the other hand if there is a project like curl rewritten from C to Rust and its author says: "We have switched from C to Rust in a year and after two years we have less new bugs", there is at least some value.

I think the parent is correct, and the situation is more nuanced than your argument would suggest.

Politics: Are you critiquing the distribution of funds to the school your child attends? You probably have some insight there. Are you instead expressing a strong opinion on a complex subject that you likely don't have an iota of experience with? Well, then you should realize that you're probably not in a position to have such a strong opinion.

Meal: You're the user, so you have experience here. All you're commenting on is how it tastes to you, so that's perfectly valid, no need to become a chef.

Car: Again, you're the user. Are you critiquing a UX problem? Well, then you're in a perfect position to do so. If you're remarking about e.g. the torque to weight ratio then you should probably understand some of the engineering that goes into creating a vehicle.

Your argument is the fallacious one here. You're making an all or nothing comparison and life rarely works that way. If you're going to make a statement like "you should rewrite curl in X language" then yes, you should know what the hell you're talking about because you're making an engineering level statement.

I wasn't making the argument that curl should be rewritten in Rust. It's too late to edit my comment to clarify it, so please see my response here: https://news.ycombinator.com/item?id=13996154

It's not that anyone who has an opinion or criticism has to be an expert or be in the trenches. But in the same token not every tom, dick and harry has valuable opinion or criticism that puts them on par with people doing the work or legitimate experts. There's certainly a fine line to walk between ignoring people or wasting your time trying to filter out a cacophony of opinion and belief from anyone willing to utter it.

No, it is not a fallacy - when it comes to matters of opinion on the feasibility of difficult tasks, experience has some relevance.

Your objection follows a common pattern - if we took every nuanced situation and turned it into a universal black-and-white dichotomy, we would end up with half the arguments on the internet.

God help anyone criticizing Google. You make your own multi billion corporation.

If a person was claiming it should be much bigger and more successful than it is, and that claim is primarily opinion-based, one might take into consideration that person's achievements when evaluating their claim.

Replying to myself, since I can no longer edit what I posted: I'm not advocating the developers of curl rewriting it in Rust! I think that a lot of the debate here is around a few different points, and that we're generally talking past one another. My primary concern was that the initial response seemed to disregard the vulnerabilities that can occur due to unsafe memory management. To the extent that I read people say arguments along the line of "you can only make that criticism if you write a curl replacement yourself", I felt that that was unfair.

Reply on news.ycombinator.com