Comment by ncmncm | Hacker News Reader

Comment by ncmncm | Hacker News Reader

This article was remarkably well written and reasoned. I would welcome more.

The author is correct that C++'s greatest weakness, and one that can never be fixed, is that other code can do things that violate the model your code depends on. It is not enough to have your own discipline, the rest of the system needs to observe the same discipline at interfaces. And there are many possible, correct disciplines to choose from.

Rust imposes one of the possible disciplines for memory-object management that enforces one set of non-negotiable tradeoffs and costs. You the programmer are relieved from choosing a discipline or discovering what was chosen, and from discovering that two subtly different ones were used in different parts of the system.

Memory-object management is not the only place where systems need a consistent discipline, but others can often be referred to it; and every program needs at least the one, besides any others.

Arguably the reason Common Lisp never took off is that practically nothing can be assumed about any other part of a system. C++ has enough forced consistency to drive adoption, but might better have had more. C++ has a lot more tooling than Rust does to capture semantics in libraries, so the library is the natural boundary for a discipline, and it is not unusual to find more than one safely co-existing, but it is also not unusual to find them not safely encapsulated, and in conflict.

Ultimately there is no substitute for sound design. Rust's limits help ensure a consistent design, but the cost in abstraction power is high.

Personally, I've written a few multi-threading implementations at various places and my accumulated wisdom is that any data received by threads has to be passed by values or be constant. Anything else is a lot of risk for little value.

I've also come to the conclusion that async/future are a bad idea for multi-threading. They're fine for managing mono-thread asynchronous events, but using them as a poor man's thread pool is begging for trouble. Among the problems are:

   - you do not know if the execution will be synchronous or not.
   - you often don't know and have no control on how many threads will be spawned.
   - it makes following the flow of data and execution much harder.

What I've come to rely on is the trinity of:

   - a thread provider (non-committal name, we all know it's a thread pool behind the scene), which provides the threads of execution.
   - a work item provider, which provides the input data to the algorithm.
   - a worker provider, which provide workers that receive a work item, execute an algorithm and produces a result.

Then something gets executed when all three are united. For the curious, the reason to have a work item separate from worker is that it allows the worker to be long-lived and re-used and, most importantly, allows containing heavier data. For example, a cache. The work item are light and are just the necessary data needed to process one... work item.

I've also sometimes divided the work item into two parts: a work item and a context. The work item is the changing data to be processed while the context is the almost constant data, and can be shared. (For example, configuration data.)

I generally prefer to code one active thread per process, with multiple processes when I want parallel work. They communicate through shared memory, generally single-writer enforced by the OS.

So, yes, there can be equally rigorous system-level disciplines that substitute for language-imposed ones, and that can offer much better performance. But it does take a lot of experience to choose well.

That sounds a lot like how the linux kernel manages it with its workqueues data structure, which gets consumed by kworker.

> Rust imposes one of the possible disciplines for memory-object management

Could you give some examples of other possible disciplines?

There are lots. For a single-threaded program, moving around unique_ptrs or, equivalently, value objects, suffices. For parallel execution, maybe you add threads and mutexes, or futures, and possibly shared_ptr or a moral equivalent.

The point is that you have a lot of choices to make that are only peripherally related to the problem the code is supposed to be solving. Those choices might allow you to tune the performance of operations better to the problem, but they may be just a distraction from the goal.

For most systems, nobody knows how fast it should be able to go, or whether you got close. Almost as often nobody even cares very much, or cares at all. When they don't, it is wasted effort to spend a lot of time and skull-sweat optimizing performance. There, a straitjacket language that offers few choices is best. It lets you get on to the next thing.

The thing is anything which can break your code implicitly in messed up ways like memory violations require unsafe code which if possible should be avoided and most important makes it easier to pinpoint what to review if you run into unexpected strange bugs.

Sure reviewing unsafe code isn't perfectly ready often it relies on invariants uphold outside of the unsafe blocks (in the same type).

Still much better then having to review everything ;=)

> Rust imposes one of the possible disciplines for memory-object management

Until you load a shared library not written in Rust into the address space.

Reply on news.ycombinator.com