For a small one time fee of 1000 USD I can copy paste you 50-100 lines of C that provides an implementation of an array without buffer oveflows. Cheaper than switching to rust or you know, learning C properly.
I'd argue that for programs such as NTP, avoiding dynamic allocations on heap is not such a terrible goal.
It then follows that reading things safely is therefore a completely insurmountable problem in C. There is no possible way for me thin wrapper around read that works on bounds checked arrays, and not use "read" cavalierly in my code without thinking. Right?
It only got worse.
I think I recognise your name from some rather aggressive rust advocacy in another thread, so I'll try and break this down in a way that won't trigger you:
- Buffer overflows are trivial to avoid in Rust, AFAIK. I acknowledge this
- Buffer overflows are very easy to code in C, and have occurred many times in the wild. Again, I acknowledge this.
- My argument is - buffer overflows are easily preventable in C if you provide your own thin abstractions. The fact that people don't take these steps doesn't mean that these steps cannot be taken. Take the rust compiler itself: AFAIK it's now implemented in C++, an 'unsafe' language, and YET it manages to provide in abstraction in the form of a language that makes buffer overflows all but impossible. Do you understand now how it's possible create a safe abstractions in an unsafe language?
Who cares? What's "possible" is completely besides the point. What matters what is done. And so, yes, you can create safe abstractions in C. But if you need to interoperate with someone else's C code, you have to deal with how your abstractions interact with someone else's, or someone else's lack of them. The point of using a language like Rust or Go or... pretty much any non-C language with lots of traction is that you don't have to provide "your own thin abstractions." You use the same one as everyone else. The point of Rust or Go isn't to make things easier on you. It's to protect you from every other piece of code you interact with. The standard library. Libraries written in the language. Other people working on the same project as you. Yourself five weeks ago. You all use the SAME safe abstraction. Nobody has to roll their own. There aren't 25 safe abstractions for 20 people working on the project. You don't have to reason out or guess whether or not some library (or even the standard library) is doing the right thing to avoid buffer overrun. You don't need to defend your entire codebase against someone else's code not doing the right thing.
Because I am old enough to remember when C was only relevant to UNIX users and those of us that care about quality code were able to enjoy much better options.
It is not possible, because one seldom works alone, so regardless of whatever laboratory attempts to write safe C code, they all fall down when the team reaches a size of two, or dependencies to binary libraries are required.
C++ is also not free from this. Regardless of the language features we are able to use, there are always a couple of guys that code it like C, thus making an hole under the castle walls.
As for Rust, as any compiler writer knows the implementation language has very little to do with the language being compiled. If LLVM was done in Java, as Chris initially thought of doing, Rust would be using a Java compiler instead.
Regardless of this being wrong, it also doesn't prove your point. The compiler and the built code are two separate executables. Your "C plus abstractions" language still allows "C without abstractions" within the same executable. And that's the dangerous part, because now you're talking about audits to ensure that only your "C plus abstractions" language is the one being used.
Without generics these abstractions are hard to work with and hard to make work across libraries. I am reminded of https://thefeedbackloop.xyz/i-bet-that-almost-works/
> Take the rust compiler itself: AFAIK it's now implemented in C++
The compiler is written in Rust. It uses LLVM, which is in C++, for codegen, but in the future we might be able to turn that off too (https://internals.rust-lang.org/t/possible-alternative-compi...)
Besides, the implementation language of a compiler does not make the compiler an "abstraction" over the language. Sort of, but not really.
The only fair thing to say at present is that Rust's main/most useful/most performant/most whatever implementation is a mixture of Rust and C++.
I really enjoyed these posts by strncat:
https://www.reddit.com/r/programming/comments/5krztf/rust_vs...
> It's scary how unfamiliar C programmers are with the rules of the language they're using... it's very difficult to write correct / secure C code without undefined behavior even when you know the rules.
[...]
> It's not feasible to avoid undefined behavior at scale in C or C++ projects. It's simply infeasible. They are not usable as safe tools without using a very constrained dialect of the languages where nearly all real world code would be treated as invalid, with annotations required to prove things to the compiler and communicate information about APIs to it.
and of course, if lots of people fail at something, it will get proportionally harder with a larger team / larger codebase / different stakeholders demanding xyz, etc etc etc.
not chiming in about the basic disagreements in this thread, but the repeated claim that something that has caused trouble for countless professionals is "easy" just can't be right. if all the qualified people are "unqualified" because they all fail at something "everyone can do," the speaker is the one who's confused, and it's a lexical problem.
I mean people manage to do a lot of dangerous things due to sloppiness - cause car accidents, for example. I don't think it's hard to avoid that, and I don't think those people should be driving. I suppose others think we need wider roads and bumper cars.
This isn't an argument against safer languages, or higher level languages, or languages with built in bounds checking or anything else. This is more an argument against programming in a "close to the metal" language like C so thoughtlessly that buffer overflows are actually a serious issue.
Maybe you just need a pedantic mind. When first learning C, as soon as I figured out that writing "char buffer[BUFFER_LEN];" could cause your program to crash, I immediately set out to write a safe dynamic array implementation I could push chars onto it and I didn't have to worry. And I am by no means an expert C programmer.
Can you give me a similar set of primitives to manipulate memory in a temporally safe manner as well? What is the cost of that abstraction? How does it compare to a runtime's GC?
It will cost a single correctly predicted branch, which is effectively free on modern architectures. Any "safe" language will have to make this conditional branch too (Rust's File::read method will check the size of the slice).
As for the cost of the abstraction of bound checked arrays of arbitrary length, I can't imagine it being any slower than rust. I could be wrong. If you'd care to provide an example program in whatever language you are advocating, I can give you an implementation in C using a bounds checked array ADT, and we can compare notes.
- index checks aren't intrinsic for all cases where it is done, e.g. https://github.com/rust-lang/rust/blob/8f62c2920077eb5cb8132...
- being intrinsic is absolutely not required for the checks to be removed. Bounds checks are just a normal branch with a fairly straight-forward condition, and the always-true nature of those conditions is inferred in the same way for both the built-in indexing and non-built-in indexing. This is true in languages other than Rust.
An iterator is designed to just not do any indexing at all (neither using the built-in [] operator or one of the functions that implements manual bounds checks), because it instead just manually (unsafely) walks a pointer along the array.
It obviously still has to bound-check that it's not about to walk right out of the array. If you want to call this something other than a "bound-check", I think that's being overly pedantic.
But yes, strictly speaking you're right, an iterator does do check when it's reached the end of the array, it just doesn't do any indexing nor does it use the indexing checks built into the compiler (which is what you implied the iterators benefit from).
This is false. The index checks are written in Rust code as an impl of the Index trait on [T].
The checks being removed have nothing to do with this -- LLVM can prove that certain checks are unnecessary. C does the same, if you used a library that provided checked indexing.
What's different is that in Rust indexing is overall used much less often, because iterators are the dominant pattern, which completely sidesteps this pattern.
> I could push chars onto it and I didn't have to worry
And how did you implement this? Are you stitching together chunks of memory, or are you reallocating and copying? If you are reallocating and copying, what do you do when you have a point to the old address space that still exists? Now instead of manually updating memory and knowing you have to take care of pointers, your dynamic implementation might change stuff from underneath you without you noticing.
What's your point? My only argument has been that it's very easy and achievable to avoid buffer overruns in C. The fact that you assert most people won't do it is completely orthogonal to that.
And how did you implement this? Are you stitching together chunks of memory, or are you reallocating and copying? If you are reallocating and copying, what do you do when you have a point to the old address space that still exists? Now instead of manually updating memory and knowing you have to take care of pointers, your dynamic implementation might change stuff from underneath you without you noticing.
I use the standard library function realloc. The dynamic array is written in a fairly standard way, IE, I am not returning pointers to the allocated memory of the internal array. I access values inside the dynamic array by value (eg they are copied), not as a raw pointer to a slice of memory that could be realloc'd. It would never even occur to me to do that, so your example seems strange and far fetched.
X is possible!
X is not probable!
Its not a winnable debate. You are both different kinds of people. The prior an idealist, the latter a pragmatist. Both philosophies are good, both are correct. Before both of you continue your debate, you should both recognize this difference.
Realloc may automatically copy the range to a new memory block if the old block cannot be expanded, and when that happens the old range is freed. Any other pointers you had to items in that original array may become invalid every time realloc is called, and if it's automated by your dynamic array code, that could conceivably be any time you push an item onto that array.
This is the waterbed theory of complexity. You can push complexity down in one part, but that just causes it to pop up somewhere else. You can make array size management easier, but the cost is that when it needs to deal with array sizing it's abstracted away to the point where you can't be sure when it happens and when you need to fix problems it might cause, or you can deal with it up front and manually when needed and then when you are manually dealing with the size changes you should remember that the memory might be freed and you may need to deal with that. GC languages deal with this by having all the information to know exactly how to fix all the references needed, and Rust deals with it by requiring you to not have two references to the memory in that circumstance.
Note that it still prints 42 despite the allocated memory being freed. Line 7 copies the data at a specified index into x - x has no references to the malloc'd memory.
Because I'm not limiting the case to just core types. You don't only ever need arrays of chars, ints, doubles, floats, etc. Sometimes you need arrays of structs.
As a simplistic example, perhaps you have a large array of structs, and you want to iterate through them and add all the items that match your criteria to a shorter array of matches, which will be pointers to the real data. Adding a single item to the original array could cause realloc to invalidate every pointer in the array of matches. Of course there are ways around this, but someone starting work on the code might not necessarily expect that adding an item to an array would cause pointers elsewhere in the code to become invalid, unless they look at the implementation of your array code to understand what it's doing.
Here's how it works - if your array is of malloc'd structs, then when you access the element, you get back a malloc'd struct. At no point can you access the actual memory, only copies of the values contained at a given index.
So yes, I suppose if your bounds checked dynamic array was implemented by a complete novice or someone that doesn't know C, your hypothetical scenario could happen.
None of this changes the fact that it's easy for a competent C programmer (do I really need to specify this?) to completely avoid buffer overflows in his own code, which is the only thing I have argued.
The basic idea is that sometimes you might want a pointer to an array item, if that item is complex, not just a copy of it, as there's no need to be wasteful if it's a fairly large struct. Any pointers to that array might be invalidated if realloc is called on it. Knowing exactly when that happens means you can note that it might be invalid, and do something about it, but if it can happen any time you add items to the array, that means you need to check for whether it was reallocated every time, or assume it's always invalidated whenever you push an item on the array.
In the example here, I'm determining the struct with the smallest num field. As I keep allocating space to the array (which I'm doing explicitly here), I'm doing the incorrect thing, which is assuming I can continue to use the pointer, which may no longer be valid and just checking if any of the new items are smaller than the existing smallest. I should be recomputing from scratch. It's obvious when I'm calling realloc, but if I was just pushing new items to the array, it would not be obvious at what point it reallocated to a new location in memory unless I specifically checked.
What's happened in that case is we've traded the complexity of explicitly controlling memory allocation of arrays for the complexity of either not allowing pointers to array items or having to keep track of the array location with a separate pointer and checking that they are still the same prior to using any pointers to array items we've stored.
https://gist.github.com/anonymous/32df4da4949a6c1067c2be8f20...
so TL;DR, if you don't want to deal with copying structs, malloc them then push them to the dynamic array, and deal with their pointers. Else if you push a struct value on the ray, return struct values.
So, manually manage their memory allocation, but allow dynamic allocation of the array of pointers? Sure, there are some cases where that's useful, but if you're already managing memory for the structs themselves, you can probably just manage the memory for the array at the same time.
> Else if you push a struct value on the ray, return struct values.
So, like I said, "not allowing pointers to array items".
You can do this, but you aren't just making array access a little safer, you're also restricting quite a bit of what you can do for efficiency. If I'm going to throw away the ability to use pointers for efficiency, why am I even using C in the first place? I should just write it in some other language from the start. Presumably I used C because there was a need for that efficiency.
Yes.
Sure, there are some cases where that's useful, but if you're already managing memory for the structs themselves, you can probably just manage the memory for the array at the same time.
... then you have memory bugs. As your example code clearly shows. What you're suggesting (exposing the internal backing array of a dynamic array) is completely unorthodox and fraught with potential bugs, and I doubt if rust even does this internally.
All I can suggest is that you look up how dynamic arrays are typically implemented in C. The technique I describe is almost universally followed. This is also what happens with std::vector in C++ - std::vector doesn't manage the memory of the elements themselves, just its internal backing array.
I think this gets at the crux of people's problem with the idea that you can just work around the problem of manually allocating memory. It's a bolt-on to the language, and the behavior is dependent on the implementation chosen, and it makes the behavior fundamentally different than "native" C arrays, to the point that it might cause problems.
> This is also what happens with std::vector in C++ - std::vector doesn't manage the memory of the elements themselves, just its internal backing array.
Yes, but there's also usage directions for std::vector that specifically state and make very clear what iterators/pointers are invalidated on what actions. Encountering someone's home-rolled array routines may or may not allow you to easily make the same deductions. Are the routines for dynamic arrays, or are they for doing system cleanup at the same time, or have they been combined? Are there comments noting the reason for what's being done, and that certain operations may invalidate pointers, or are you left to intuit that yourself?
These are the problems with having a non-core (and not even a popular implementation to fall back on) way to extend the language. C++ is a step up in that it at least standardizes a bunch of core types so you can learn those and carry your knowledge of how they work around to different projects in the language. C's lack of this means that every project may implement something like this - or not - in their own way, with subtle usage differences.
The main benefit you would get from Rust in a situation like this (ignoring that it would likely either be built in or readily available through a crate), is that on encountering some home-rolled system, you can look for where it uses unsafe to find any problematic behavior you need to be aware of, because otherwise you are fairly protected. Worst case, the whole home-rolled chunk of code is riddled with unsafe blocks, and you know it's definitely something you need to hunker down with to figure out what's going on (assuming you need to use it).
Rust's unsafe is effectively an enforced comment around dangerous code. Put that way, I'm not sure many C programmers would really object.
There is some memory overhead per array with this method though.