Chromium project finds that 70% of security defects are memory safety problems

I think the previous poster was referring to SQL injection vulnerabilities from not sanitizing DB query parameters. And that's something you only f* up if you're clueless (like I was back when I used to do that).

>Wow. To equate logic errors with being clueless?

I meant that input sanitization is generally an easier problem to solve than writing safe C. If you can't do the former reliably, I'm certain you wouldn't be able to do the latter.

Input sanitization should generally be handled very close to the interface, once you went through this step you should expect to only deal with safe data. Memory safety covers the entire application, if some non-critical debugging function 10 level deep in the call stack messes up when it computes the size of a message buffer you can have a remote code execution vulnerability.

>Also you assert it is absurd to compare WordPress and openssh In the context of this argument why? They are both widely used software with very high stakes on their reliability and security.

Wordpress is a complicated ecosystem with multiple plugins that each can introduce security problems. Its attack surface is also incredibly large since it's generally public facing and anybody can interact with a lot of the features by default. That coupled with the fact that it's extremely popular make it a very good target for attackers. It's also a relatively fast moving target since the web changes fairly fast and new features have to be implemented regularly.

OpenSSH does mostly one thing and does it well. Its surface of attack is a lot smaller and it's vastly easier to audit and test. Its development is also overseen by the OpenBSD developers, who are famously uncompromising regarding safety. The feature set is very stable and changes very slowly.

>Actually it shows memory management is a don't care variable.

The absence of evidence is not the evidence of absence. How much development effort went into making sure that these issues didn't exist? How many thousands of hours auditing the code and making sure it did what people wanted? In a safer language this time could've been spent auditing other parts of the codebase or implementing other features.

I'd also like to point out that one of the biggest OpenSSH vulnerabilities ever found (in the Debian version of OpenSSH where the maintainer heavily reduced the generated key entropy by mistake) was indirectly caused by a memory issue since the reason for the patch was a false-positive returned by Valgrind regarding use of uninitialized memory.

It's a miracle that people who think like that use computers at all - after all, computers only reduce the time it takes to perform a clerical task or a calculation, they do not eliminate it.

Also memory-safe languages with GC allow for simpler API and design. That in turn gives more time to think on errors in logic.

> Switching from C++ to a language without memory safety issues while being just as fast

Does that actually exist? (I mean in the general case, not just a few carefully chosen benchmarks).

If your type system can handle the case where I can divert code flow along an unsandboxed path by replacing functions from the system libraries even before you can achieve code execution (and long before you certainly though I had code execution), I would like to hear it. But expressed in a less brusque way: yes, encoding state in your type system is generally great, and there is a lot of work being done in languages where you can formally “prove” guarantees about your code. The issue is that your proofs only hold true if the system they run on is correct, you chose exactly the right thing to check, you didn’t just leave out part of the program because you couldn’t fit it into the type system…it’s a nontrivial problem.

I do this all the time in Scala. Define a marker for "required security level", slap it in a free monad or add it to an existing effect stack, then it just naturally gets propagated right the way up, and you can then put the checking where it makes sense (e.g. right at the outermost request handling level).

This is possible, but I certainly would not describe it as “easy”.

Big if true :-)

But here is a great article

https://queue.acm.org/detail.cfm?id=3212479

It's rather easy to dynamically link something that's not polymorphic.

What exactly is the issue with Rust and dynamic linking? Is it that it can't dynamically load Rust interfaces? There's nothing wrong with linking plain old symbols from shared objects in Rust.

Software is getting slower due to poor algorithm choice, not due to the one-off factor of maybe 2 that you pay by switching to a compiled but memory-safe language. If C++ was a sensible choice in 1995, 18 months later you would have got the same performance out of a memory-safe language; we're now 10 cycles of Moore's law down the road, our hardware is 1024x faster and our languages are certainly not 1024x slower (unless you're using a scripting language). It's more about stuff that's accidentally quadratic, and that kind of error is if anything easier to make in C++ than in a more concise language where it's easier to see what you're doing.

C++ developers may be smart people because you have to be smart to do anything in C++ - imagine if the same brainpower that's being spent tracking memory usage and pointer/reference distinctions could be put into your actual application logic instead.

Handling large numbers of connections is more easily done in Java, in my experience, since Java's async support is relatively straightforward.

I would lay money that the language isn't your real bottleneck. Switching languages might save you a factor of 2, using better algorithms or datastructures can save you a factor of 1000 or more. How much profiling do you do?

Look at the networking sourc code for Redis. It meets all your requirements and isn't that big.

> If you are making building blocks (libraries) for other languages, you have to use C or C++ (maybe Rust soon).

Sure (though there are often other ways to achieve the thing you actually want to do). But in that case you're not doing it "for performance".

> When you aim for high performance you don't profile day 1, that's a complete waste of time. You will never transform every of your function in a critical kernel. You profile when you reach a performance bottleneck and you need it.

In that case your performance requirements are not extreme enough to justify using C/C++ for your whole application. Write it in a safer language, when you hit performance issues profile and optimise, and maybe drop into C/C++ for those few "critical kernels" in the unlikely event that it turns out you actually need to.

If using Rust results in fewer bugs for less money, shouldn't we still do it?

The reason SQLite can test so thoroughly is that their requirements never change. I have tried to browse the web with SQLite but so far no luck.

I am talking specifically about the style engine, which is the topic of this sub-thread.

Here is is straight from the devs: https://youtu.be/Y6SSTRr2mFU?t=360

> And that if I just took off my rose-tinted glasses, I'd realize my Rust code is buggy, unsafe, slow, and hard to maintain

I didn't get the impression.

I understood that Firefox talks about their success in rewriting in Rust because it's their language, they control it and are the major sponsor and user.

I don't think Google or MS or any other company heavily involved in crafting programming languages for their own purposes will ever go that route for some of their core software, because they can't control the language and if they tried they would get the blame for trying.

The previous posters were talking about languages available when chromium was started. So while Rust probably would be the language of choice today (see Firefox), it wasn't an option, but Java was already around and mature.

Like a factor of 2 at best. Nice to have, but not the difference between usable and unusable.

Haskell would be a good choice for a reference implementation by the committee that designs HTML and the rest of the web.

From there, benchmarking could reveal if it is adequate for production use. Perhaps some parts could be rewritten in a lower level language.

> I am genuinely curious about this, do you really think Haskell has performance characteristics that are suitable for implementing Chromium? Haskell is wonderful for many uses, but it is difficult for me to imagine it as the basis for a modern web browser.

Short answer: yes. Long answer: profiling and diagnosing performance issues is still kind of a black art, but for large codebases I've seen Haskell rewrites outperform C++ significantly. You need talented Haskell developers, but surely Google should be able to find those. Most of the tasks that I can think of in a web browser seem like things that Haskell is ideally suited to - parsing, data transformation, rule-based logic - what is it that makes you think it would be unsuitable?

Chromium took parts from WebKit, but e.g. the whole JavaScript engine was completely new. Even if they needed to depend on some libraries written in C++, that doesn't justify writing the whole application in that language. And once they forked WebCore as Blink they could have gradually pushed the borderline down.

&mut T does not mean what you think it means

&mut T means that there is only one pointer (the &mut T) that will be used to access some memory. You can have as many other pointers to that memory as you want, as long as you don't invalidate the first one.

This code is safe:

    let mut foo = 13_i32;
    let x: &mut i32 = &mut foo; // First &mut
    {
        let y: &mut i32 = unsafe { std::mem::transmute(x as *mut i32) }; // Second &mut T (copy)
        *y = 42; // Write through second
    }
    dbg!(*x); // Read through first

You can run this under miri here: https://play.rust-lang.org/?version=stable&mode=release&edit...

I think Rust's model with smart pointers and deref could work really well with a colored pointer, short-pause GC like ZGC.

Rust has the capability to do things like execute load barriers on access, so it could still work when objects have been moved around.

The allocator API hands out `*mut u8` so... not really. If you changed the allocator API and everything that uses it and makes assumptions about how it works then sure, you could make it hand out a pointer to a pointer so you could come in later and update the inner pointer after you've done a GC pass. You couldn't do something like handing out a normal `Box` and keeping track of where it lives so you can patch it later though. Rust doesn't have move or copy constructors so as it gets passed around it'll just be `memcpy`'d and you'll lose track of it.

Ah, I see what you’re saying, and I am afraid I am out of my depth because this touches on a lot of maintainer intentions I can’t speak to. You are correct in implementation assumptions at least.

> gc-arena will not collect while the heap is being mutated, which is Rust-friendly but also impractical.

This is where the generator stuff I mentioned comes in- I think the approach is much more practical than it seems.

At the end of the day, any GC language needs to ensure there are no live un-rooted GC pointers at points where collection takes place. So any realistic Rust/GC integration will need to solve the same problem, if it is to retain memory safety.

And the best way (so far) to ensure all references into the heap have disappeared is to make sure they are all derived from a function parameter with an unconstrained lifetime, because then the mutator knows nothing except that it might go away when it returns. (This is "generative lifetimes" from the links.)

The real trick then is to make it look like the mutator is returning, without actually forcing it to exit. A yield point in a generator can do this, even without ever actually yielding. So you can manually insert collection points in the middle of a Rust mutator, and you can have a "managed -> native -> managed" stack where the native frame thinks its managed callee can yield across it, and borrowck will ensure anything derived from that reference parameter is gone.

Of course this is a long way off from Rust today, where generators aren't available on stable (though perhaps you could hack something together with async/await), but does look like it should eventually be possible to have reentrant heap access and memory safety across Rust and a GC language.

Surely they way most gcs achieve this is by asking you to tell the gc when you aquire a reference to an object, and tell it again when you no longer need it. You could do that with a custom implementation of drop in rust.

Ah. I am not a Rust expert but I think the language might actually support this sort of construct (though it might be inefficient to implement), as IIRC if you want something to stay at a fixed place in memory you need to “pin” it there.

Not to say they don’t exist or haven’t been reported, but many of the unsound issues on github for Rust don’t seem to hold water.

They all seem to misbehave on a single nightly build, affect only a very narrow and unsupported target group, be related to a bug in a unsupported release of LLVM, or simply aren’t reproducible. I think the percentage of security bugs in Rust related to memory safety, based off that list, is _effectively_ 0%. I can’t find an issue in the list that seems like it would impact Rust programs that people write today or on targets that people deploy Rust code where memory safety matters.

And there are memory-safety bugs on the CPU level, as we had to learn. However, "safe" languages cut the risk of a memory-safety related bug down by orders of magnitudes, in most cases to practically zero, as the Go example require quite an effort to create the situation. You don't end up in such a situation just by a programmers or logic error, and thats what the memory bugs are about.

I would not even say that Rust (or any other language) is or can be perfectly safe in "safe" code even with a bug-free compiler. It is possible to access unallocated memory in 100% "safe" Rust, because what memory that is allocated and not depends on the programmer's intent. Allocated memory is memory that you intend to use.

If I make an array of 3 integers that are not supposed to be used yet but I accidentaly access one of them, then that is a memory-safety bug because I accessed unallocated memory. The functions that languages provide to "allocate" memory don't define what allocating memory means. They just provide a tool that you can use to keep track of what memory you are using.

What memory is allocated or not is relative and there are multiple levels of allocation. If we zoom out a bit we can even call C a memory-safe language because all memory accesses in C must be to allocated memory and the ones that are not will kill the process (allocated here as in allocated to the process by the operating system).

70% of all failed jokes on HN were best efforts. But not this one.

maybe the parent changed the link? the link appears to be wikipedia :)

And usage of C++ is still growing exponentially, so the number of lines of code is growing faster than our ability to rewrite/replace them. I know, it's a losing battle. We'll never do it. We are stuck with C++ until the end of time because of the sunk cost fallacy. If everyone who currently works on C++ instead switching to rewriting instead of adding, we'd be able to replace everything in 5 years.

If C++ is only kept around because of a few libraries needed to implement the runtime of safe languages, then I would consider that some kind of victory.

Ah, that sucks. I don't think there's a solution to that apart from just adding an "Invalid" field to every enum class and initialising to that, but checking for that (asserting not invalid) adds ugly boilerplate to every usage of every enum.

Ah, I don't think that's a vexing parse issue; it's something else. It's a really great example though, because it illustrates a lot of different issues at once, and I can tell you exactly how you could've avoided it (though a some of the blame does lie with C++). In fact, it's possible you haven't realized what really went wrong yet. What you have (or had, until you used braces) is a heavily-templated version of what simplifies to the following:

  struct Instruction { explicit Instruction(int encoding) { } };
  using T = Instruction &&;
  int encoding = 0;
  auto &&result = T(encoding);

There are a few things that went wrong here, but the final red flag to notice is that you should never call a constructor directly with 1 argument directly, because it's just a different syntax for a C-style cast, which we know is dangerous due to its bypassing of safety checks—and this is true regardless of whether we're dealing with C++ constructs (like classes and constructors and such), which I think might be what you're realizing now. (This is poor C++ design, but it's old and people know to avoid them syntactically just like C-style casts. It might be nice to have a warning for it too.) Rather, when you're passing a single argument, you want to write one of these syntaxes:

  T result(encoding);  // option 1
  auto &&result = static_cast<T>(encoding);  // option 2

With these, you receive an error, e.g.:

  error: invalid static_cast from type 'int' to type 'T' {aka 'Instruction&&'}

I believe this is because the code requires 2 conversions to occur at once, which is an error because (surprise!) it's a generally unsafe thing to do: (a) conversion of int to Instruction, and (b) conversion of Instruction to Instruction&&.

Now you bypassed this by using the uniform initialization syntax (i.e. braces). I'm going to go out on a limb here and say that was another mistake, even though it "solved" your problem here: despite the widespread use, brace initializers are, in my experience, not a good thing, and it's unfortunate that people embraced them (ha) with open arms, and similarly goes with emplace_back() and some other things which I'll address below. The syntactic convenience they provide is just too minor compared to the issues they introduce or obscure. And in this case, they indeed actually introduce a new issue if you use them like 'T result{encoding}': they allow 2 casts to occur at once. I find that incredibly dangerous, and I think it should be at least a warning if not an outright error like before. It seems like a C++ design flaw to me, but in any case, maybe someone should get compiler writers to add a warning for this.

Anyway, let's move on. If you're reading this, you're probably noticing that you ended up with Instruction&& in the first place—that's probably not what you wanted, or at least not what you should've wanted.

And that's where we get to the heart of the issue: your real problem is that you used decltype. If I saw that during code review, I would force you to change it—and using it on declval is just adding more fuel to the ember.

The reality—which unfortunately you do not see people acknowledging—is that decltype, auto, uniform initialization syntax, emplace_back, etc. are all dangerous, and harder to reason about than they look. I think it's unfortunate that the C++ committee encouraged people to use them so much, and I think they're overused to an insane degree. People who love them for their nice syntax don't go out of their way to figure out their pitfalls, but I almost never use any of them unless I absolutely need to. Most problems that they solve (one notable exception being 'auto' with lambdas) were quite elegantly solved in C++03 using typedefs. The only caveat was that you had to give up on the idea of minimizing keystrokes. It's quite a realization when you realize that solves so many of your problems.

Anyway, I'm not trying to blame these on you. Obviously these are blamable on C++, and we could (and should) have more warnings for them. Rather, my main message is that you can avoid these problems (even if they're other people's faults) syntactically—and locally—if you don't try to embrace the absolute "latest and greatest" in C++. IMHO you should only use the newer features if they solve an actual semantic problem for you, not merely because they minimize your typing.

In fact, I think is a huge mistake people make with software in general, and here, C++ in particular. They feel if something is old then it must be bad and you have to do everything in a new way. But if you stick with what works and start caring less about people looking down on you for using "old" syntax just because it's old, you'll find a lot of the old C++03 patterns (typename Pair::first_type, etc.) are actually robust to the problems that the newer ones introduce. People just don't realize this because they're more verbose than they would like, and we're in an era where doing things old style looks bad for no good reason.

Oh also, one last thing: aside from avoiding decltype and using the equivalent of a C-style cast, one more thing that helps you avoid this is to avoid overusing templates. They also obscure what's going on, like here. Not to mention the slow compilation speed and lack of independent compilability. Those are also overused (and I see their appeal) but they're often unnecessary and make code statically difficult to reason about.

How does that help though?

The memory management issues in my C++ programs (recently, real-time audio stuff) are where I explicitly decide not to use modern C++ / automated memory management, and write my own allocators that rely on malloc/free or some variant (aligned_alloc, etc.)

In Rust I suppose I would just use an unsafe block and have the exact same issues.

Note that good unit testing, assertions and sanitizers generally take care of the issue.

I too used to program in C++. Every Monday morning, it was the same routine: as I enter the office, the stench of decaying bodies is overwhelming. Yet I must gather my strength to collect and identify the people killed over the week-end by C++ memory safety issues gone unchecked. Once that's done, I start my daily Scrum standup at 11 and I start coding a bit. First build at 11.30, first segfault at 11.35. Then it's pretty much the same routine, after lunch I read the Valgrind and ASan reports, spotting which one of the hundreds of new safety issues it identified might be an easy fix. I go back home riding my bicycle around 7pm, making sure to avoid the cars trying to crash against me due to segfaults. Sometimes I cry at night thinking about all that.

And then one day I found Rust, and all those problems went away. I can now write fearless code, and I don't have to endure the stench of rotting bodies anymore.

True story.

See also the example of librsvg, a C implementation of SVG that was incrementally translated to Rust by some volunteers over a couple years.

https://people.gnome.org/~federico/blog/exposing-c-and-rust-...

Quite on the contrary, the worst offenders are:

- implicit conversions

- decays from enums to integers

- implicit conversiosn from integers to unexisting enums

- no bounds checking

- implicit conversions between pointers and arrays

- no proper way to ensure a given array length is valid as part of a function parameter

- null terminated strings, that occasionally aren't terminated

- abusing null terminated strings with clever algorithms, e.g. strtok()

- the preprocessor

- const that isn't really const

- variable arguments that require getting the macro type arguments

- UB explored to the last possibility of code optimization

- no safe way to deal with output parameters

- typedef don't introduce strong typing

All of that came from C, not C++.

Because what got C++ adopted in first place was its copy-paste compatibility with C.

C++ was also born at Bell Labs, and due to that, all major C compiler vendors quickly started shipping C++ on their boxes as well.

If you take away copy-paste compatibility you might be better off doing D, C# or whatever safe variant already exists.

Which is what many of us have done, to move to type safe languages, and only use C and C++ at the boundaries, in small pockets of unsafe code.

In fact if you look at mobile OSes, that is the reality for app developers, C and C++ are no longer the full stack languages they were 20 years ago, rather used for the kernel, drivers, compositor and shading languages, but everything else happens in safer languages.

And the SDKs only allow you to write libraries, not full applications.

Naturally are clever developers that subvert the workflow and transform the libraries into the actual application.

Because this would refuse to compile 100% of real-world C++ code, so nobody would use it.

One problem is that C++ can directly include C headers of the operating system, while languages that aren't copy-paste compatible with C have to create some kind of wrapper library for them... this is a major reason for the success of C++, but it also makes improvements of this kind far harder to deploy in practice.

Because you would make many proven algorithms from Knuth or CLRS illegal.

This is never acknowledged by the C++ people: Idiomatic C++ is not suitable for formal proofs, if you don't believe me, ask Xavier Leroy.

These are database kernels used for mission-critical things with extreme workload profiles. It has to survive longevity testing at saturating workloads with continuous fault injection. If there were memory safety issues, I think someone would have noticed by now.

The reality is that there isn’t much opportunity for memory safety issues to occur anyway, the type system and scheduler do most of the heavy lifting. Similarly, concurrency safety isn’t much of an issue because threads barely interact. Most high-performance server software looks this way these days.

Bugs tend to be of the boring logic variety that can happen in any programming language.

Far more exhaustive testing than that, these are database kernels. Formal verification of the design and code where it makes sense. Extensive use of the C++ type infrastructure to verify several aspects of correctness at compile-time that would be impractical in any other systems language. The Linux kernel is bypassed for almost everything, most of the schedulers, I/O memory management, etc is in user space like most modern high-performance databases, which actually reduces the bug surface and increases verifiability. It doesn’t have to just be functionally correct, the performance has to be consistent under diverse and adversarial workloads.

No one can know if it is bug free, that is impractical. But it also isn’t like this is a weekend hobby project either. Most of the bugs that get out are in unimportant peripheral code and integrations.