The sooner we can rewrite our programs in Go and Rust, the more secure we will be. Our shells, coreutils, mail readers and web browsers have to be written in safer languages.
It's harder to find the dependencies for building the latter, but the former has its own version of dependency hell. I have real trouble building both types of projects, though admittedly (especially when the building instructions don't work when followed to the letter) C++ a bit more than the strategy of "everything is just pulled from github, you only have to make sure you've got gigabytes of free space in ~/.cache/, a build environment that was released in the past four to nine days, and have appropriate isolation or simply not care about potentially vulnerable or compromised code being run on your system".
On a rare occasion, I will find a nice and small program using only standard libraries that compiles simply with `cc my.c && ./a.out` or runs simply with `python3 my.py`, demonstrating it doesn't depend on the language to have an easy time building it, but in both categories it's the exception for some reason. I see so much software that needs only standard libraries and runs on literally any python version released in the last decade, but to run it you have to setup some environment or globally install it with setuptools or something.
I hear this a lot, but I can't divine any substance from it. Why is GitHub a less-secure repository medium than SourceForge + random website downloads + various Linux package managers? Maybe this is a red herring and your real complaint is that the Rust ecosystem is less secure than the C/++ ecosystem?
> you only have to make sure you've got gigabytes of free space in ~/.cache/
I cleared my $GOCACHE relatively recently (I thought maybe I had a cache issue, but I was mistaken), but it's currently at 75M while my .cargo directory weighs 704M. If these ever really got too big I would just `go clean -cache` and move on with life. If this is one of the biggest issues with Go/Rust/etc then I think you're arguing my point for me.
> a build environment that was released in the past four to nine days
What does this even mean? You can compile Go or Rust programs on any Linux machine with the build tools. On the contrary, C/C++ dependencies are very tightly coupled to the build environment.
> have appropriate isolation or simply not care about potentially vulnerable or compromised code being run on your system
Not sure about Rust programs, but Go programs absolutely don't run arbitrary code at compile/install time. C programs on the other hand absolutely do run arbitrary code (e.g., CMake scripts, Makefiles, random bash scripts, etc).
> I see so much software that needs only standard libraries and runs on literally any python version released in the last decade, but to run it you have to setup some environment or globally install it with setuptools or something.
Yeah, Python package management is a trashfire; however, this is entirely because it is so tightly coupled to C dependencies (many Python libraries are thin wrappers around various C programs, each with their own bespoke build system). Python package management tries to paper over the universe of C packages and it kind of works as long as you're on a handful of well-supported distributions and your dependencies have been well-vetted and well-maintained.
In Rust every non-trivial library pulls in 10s or even 100s of dependencies.
I don't think anyone can expect that all of these libraries are of good quality but how would one even try to verify that? And you have to verify it every time you update your project.
Then there is the issue of licencing - how to verify that I am not using some library in violation of its licence and what happens if the licence changes down the road and I don't notice it because I am implicitly using 500 dependencies due to my 3 main libraries?
Rust and Go have solved memory safety compared to C and C++ but have introduced dependency hell of yet unknown proportions.
Python and other dynamically typed languages are in a league of their own in that on top of the dependency hell they also do not provide compiler checks that would allow user to see the problem before the exact conditions occur at runtime. They are good for scripting but people keep pumping out full applications and to be honest there is not much difference between giant Python application, giant maze of Excel VBA and giant Node.js heap of code. Of those, Excel VBA is most likely to work for 5 years and across 5 versions of the product yet it is also the most likely one to receive the most negative comments.
But it's also quite a lot easier to audit those dependencies, even automatically (incidentally, GitHub provides dependency scanning for free for many languages).
> Then there is the issue of licencing - how to verify that I am not using some library in violation of its licence and what happens if the licence changes down the road and I don't notice it because I am implicitly using 500 dependencies due to my 3 main libraries?
This is also an automated task. For example, https://github.com/google/go-licenses: "go-licenses analyzes the dependency tree of a Go package/binary. It can output a report on the libraries used and under what license they can be used. It can also collect all of the license documents, copyright notices and source code into a directory in order to comply with license terms on redistribution."
> Rust and Go have solved memory safety compared to C and C++ but have introduced dependency hell of yet unknown proportions.
I mean, it's been a decade and things seem to be going pretty well. Also, I don't think anyone who has actually used these languages seriously has ever characterized their dependency management as "dependency hell"; however, lots of people talk about the "dependency hell" of managing C and C++ dependencies.
> Python and other dynamically typed languages are in a league of their own in that on top of the dependency hell they also do not provide compiler checks that would allow user to see the problem before the exact conditions occur at runtime.
I won't argue with you there.
You're exaggerating here. The most recent project I've been working on pulls in 6 dependencies. The anyhow crate has no dependencies, regex 3 (recursively!), clap and csv each 8. Only handlebars and palette pull in 10s of dependencies, and I can trim a fair few dependencies of palette by opting out of named color support (dropping the phf crate for perfect hash functions).
There are two key differences between large projects in ecosystems that have working package managers and those in C/C++. For starters, the basic working 'package' tends to be smaller. Consider something like LLVM (just LLVM itself, none of its subprojects like Clang). This you'd count as "one project" without any dependencies (at least, no required dependencies that aren't vendored, liked zlib and googletest). But if you were to make it in something like Rust, it would likely consist of a dozen or more packages: the core IR, analysis passes, command-line tools, transformations, codegen (with each target likely being its own package!). With such fragmentation, you'd have dozens of dependencies that are essentially nothing more than the 'project' itself.
But more importantly is that, because dependencies are hard to add, code duplication is the norm; it's only when what you want to do is too impossible to duplicate that you resort to a dependency. For any large C++ project, I can virtually guarantee that there will exist some form of 'standard library++' dependency in the code that will contain at least an intrusive linked list, better hashtable implementation, std::vector-lookalike that avoids heap allocation for small lists, replacement for iostream, helpers for process creation, custom string formatting routines, and something to make dealing with segfaults sane. In a package ecosystem, each of those kinds of things would be a separate dependency. Is it really necessary for everybody who wants to write something dealing with graphs to have write their own graph-based data structure, as opposed to just using the ecosystem-standard graph library (e.g., networkx in Python, or petgraph in Rust)? Or to have everybody that wants to use a hashtable to have to write their own implementation of a notoriously tricky data structure?
I take your point that NIH is a large thing driving this sort of divergence in many languages, with some (such as Common Lisp, and C) being more notorious than others.
But I'd say it's just as common in other languages for programmers to pile on needless dependencies. E.g. how many Rust/Node.js/Java etc. projects are using some "standard library++" to e.g. get better memory behavior for strings, but for whom string allocation overhead is never ever going to be an issue?
It would be an interesting metric to check what % of the code in a given library is commonly used by its consumers across languages. Even if you only used 1% you're implicitly assuming some of the maintenance overhead for 100% of it. E.g. if it won't build on an older toolchain that's now a hassle for someone packaging your program, even if it's in the 99% you don't use.
druid-derive v0.4.0 (proc-macro) (\druid\druid-derive) ()
druid-shell v0.7.0 (\druid\druid-shell) ()
I can't build github.com/restic/rest-server because my golang compiler is too old. Admittedly I'm running something relatively old (but still supported) at the moment but this isn't the first time. Such errors are my experience every time I try to build go/js code. If you're actually using a stable distribution you're simply out of luck and need to find another system to contribute to those projects.
> Why is GitHub a less-secure repository medium than SourceForge + random website downloads + various Linux package managers?
I'd trust GitHub a lot more than SourceForge, but GitHub is where everyone hosts everything these days so I meant it to mean: random repositories from the internet. When trying to build the aforementioned project, I saw a it pull from a ton of domains, but also a lot form GitHub (many different authors).
That's the crux: I may trust github itself and the author of this one repository (in this case, the author is actually a friend of a friend, in other cases it might just be a well-known person like cperciva or sircmpwn), and if my OS is compromised then I'm screwed anyway so the repositories are also considered mostly trusted, but I don't for a minute believe that the author of the code I'm trying to build managed to vet the 600 repositories it's pulling.
Also, six hundred repositories. All this thing does is spawn an http server and handle a few API calls, acting as a dummy storage server (implementing things like 'get me this file' or 'put this file here').
> Go programs absolutely don't run arbitrary code at compile/install time
I didn't actually know that, that's nice. I mean, still if I can't trust the binary then it's of little use to me personally, but any malicious code would presumably have to be called into and build servers will have an easier time serving binaries, so that's a good thing.
> I would argue that letting us figure out how to build and install the dependencies is fairly UNIXy
Crumby build systems force you to figure out how to build and install dependencies (or die trying). Modern build systems allow you to figure out how to build and install dependencies. If the former is "more UNIXy" than the latter, then I strongly contend that "UNIXy" is not a desirable property.
> It’s also unclear to me that centralized package managers are necessarily better for security, though they’re easier to use.
"Centralized" is irrelevant. Go's package manager is decentralized, for example. Moreover, many folks in the C world rely heavily on centralized repositories. Further, I would be shocked if manually managing your dependencies was somehow less error prone (and thus more secure) than having an expert-developed program automatically pull and verify your dependencies.
> Also a lot of more modern tools I’ve tried to build in recent months do not give a crap about cross compilation as a use case.
I mean, C doesn't care about anything, much less cross compilation. It puts the onus on the developer to figure out how to cross compile. Some build system generators (e.g., CMake, Autotools) purport to solve cross compilation, but I've always had problems. Maybe I just don't possess the mental faculties or years of experience required to master these tools, but I think that supports my point. By comparison, cross compilation in Go is trivial (set `CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build` works virtually every time from any platform). I haven't done much Rust cross-compilation, but I would be surprised if it were harder than C/C++.
The annoying factor is gathering the cross compiler and target dependencies, which is fortunately becoming easier with tools such as llvm/clang, and distros such as debian have always made huge efforts to facilitate it.
On the other hand, more modern languages like Rust and Go tend to have a single standard library and compiler that can be used across basically any common operating system, whereas trying to use gcc/libstdc++ or clang/libc++ on Windows can be an ordeal, and using MSVC on Unix is essentially not an option at all. Cross compilation for other architectures might be a bit more work, but the majority of developers don't have to worry about that very much, whereas support across Linux, MacOS, and Windows is a lot more common.
One of my friends who work on AIX machines without direct Internet access does not share the same view, though.
Because C codes tend to have less dependencies and shallow/more "clustered" dependency graph.
To be fair, that's more or less due to dependency management being a 100% pain 0 fun experience.
Unfortunately this does not match my experience :( Rust projects tend to depend on a lot of smaller, single-purposed (or, npm-y) dependencies, for example, Debian's ripgrep package has a X-Cargo-Built-Using saying:
X-Cargo-Built-Using: rust-aho-corasick (= 0.7.10-1), rust-atty (= 0.2.14-2), rust-base64 (= 0.12.1-1), rust-bitflags (= 1.2.1-1), rust-bstr (= 0.2.12-1), rust-bytecount (= 0.6.0-1), rust-byteorder (= 1.3.4-1), rust-cfg-if-0.1 (= 0.1.10-2), rust-clap (= 2.33.3-1), rust-crossbeam-utils (= 0.7.2-2), rust-encoding-rs (= 0.8.22-1), rust-encoding-rs-io (= 0.1.6-2), rust-fnv (= 1.0.6-1), rust-globset (= 0.4.5-1), rust-grep-cli (= 0.1.5-1), rust-grep (= 0.2.7-1), rust-grep-matcher (= 0.1.4-1), rust-grep-pcre2 (= 0.1.4-2), rust-grep-printer (= 0.1.5-1), rust-grep-regex (= 0.1.8-1), rust-grep-searcher (= 0.1.7-1), rust-ignore (= 0.4.16-2), rust-itoa (= 0.4.3-1), rust-lazy-static (= 1.4.0-1), rust-libc (= 0.2.80-1), rust-log (= 0.4.11-2), rust-memchr (= 2.3.3-1), rust-memmap (= 0.7.0-1), rust-num-cpus (= 1.13.0-1), rust-pcre2 (= 0.2.3-1), rust-pcre2-sys (= 0.2.2-1), rust-regex-automata (= 0.1.8-2), rust-regex (= 1.3.7-1), rust-regex-syntax (= 0.6.17-1), rust-ryu (= 1.0.2-1), rust-same-file (= 1.0.6-1), rust-serde (= 1.0.106-1), rust-serde-json (= 1.0.41-1), rust-strsim (= 0.9.3-1), rust-termcolor (= 1.1.0-1), rust-textwrap (= 0.11.0-1), rust-thread-local (= 1.0.1-1), rust-unicode-width (= 0.1.8-1), rust-walkdir (= 2.3.1-1), rustc (= 1.48.0+dfsg1-2)
Building it using cargo with Internet access is a breeze. Figuring out how to `cargo vendor` is not. And the sheer number of the dependencies makes it not practical to manually do stuff.
In short, what cargo actively supports and everyone uses are great, otherwise it's disaster.
It turns out that there are a lot of factors that go into building safety. The material is one vulnerability, sure. But there's also the connection method and strength, the calculated loads, shear forces, wind, earthquakes, egress, and a billion other considerations.
What resulted in better building safety was the development of building codes. Even using flammable building materials, people adapted the way they built so that the end result was much safer than before. If you told a builder you'd never buy a wooden home because wood is "unsafe", they'd laugh at you - and then sell you a bunch of "inflammable" crap you don't need.
This will seem trite, but I think it's just literally easier to figure out how to build wooden buildings that are fire resistant than it is to write memory safe code in C/C++.
I actually don't think securing C/C++ code is that hard. It's certainly a skill you need to learn, but so is writing linked lists and qsort. I think people just aren't applying themselves. But the language seems to catch the flack rather than the programmer.
From the article:
The bug is that there is simply no bounds checking at all; sig and key are
arbitrary-length, attacker-controlled blobs, and cx->u is a fixed-size buffer.
Anyway. I'm definitely not against new languages. But I think before a program is rewritten, it should be for a reason much better than "I didn't want to secure the code".