Scalable C – Writing Large-Scale Distributed C | Hacker News Reader

Scalable C – Writing Large-Scale Distributed C

hintjens.gitbooks.io

193 pointsingve145 comments

>The more C++ you know the worse you become at working with others. First, because your particular dialects of C++ tend to isolate you.

So why is using something non-standard and non-universal like <czmq.h> not perceived as a "dialect" of C?[1] Any non-trivial source codebase beyond "printf("hello world")" will be a "dialect" of the programmer(s). When looking at the C source files of Linux kernel, Redis, and SQLite, etc, the syntax patterns, helper macros, string manipulations, etc do not look the same.

Also the author's example of,

  for (i = List.begin (); i != List.end (); ++i) 
    cout << *i << " ";

is not the same semantics as:

  char *fruit = (char *) zlist_first (list);
  while (fruit) {
      printf ("%s ", fruit);
      fruit = (char *) zlist_next (list);
  }

The C++ loop is multithread safe. The C version is not. For the zlist_next() to work, the "list" data structure needs to maintain mutable state in between subsequent calls. (Think of how something like strtok() works by mutating the string).

[1]possibly because Pieter Hintjens is the programmer & CEO behind ZeroMQ and czmq.h. Therefore, it doesn't feel like a dialect to him?

Also, it's not always a problem of being able to work with other. Having a standard also means that other compilers will compile your C++ code.

We're not talking about a language, we're talking about a programming language, and standards and compilers are the only metrics which make that language "talked" by others.

Isn't a programming language both? Sure the goal is arguably to communicate some process for the computer to perform in a machine-readable way. But I can easily write two valid semantically equivalent programs which produce equivalent code when fed to a standards compliant compiler. However, one of these programs may be much more readable to a human because it follows an established set of coding practices. Perhaps a better word that standards would be "conventions" but either way, most programs also need to be read by humans at some point.

> Having a standard also means that other compilers will compile your C++ code.

As someone maintaining code that must run on VS2012, VS2013, VS2015, GCC 4.7+, clang 3.5+, on both x86 and x86_64, I chuckled hard at your comment.

Let's just say that I'm looking forward to moving on with my career in a few months. Hopefully to something that doesn't involve maintaining C++ code.

The C loop is not multithread safe because of this right in "why this book exists":

> The standard approach for concurrency is POSIX threads that share their state. This is complex and fragile. We know how to do this better, using message passing between threads.

In other words, the code he's presenting is not expected to need to be thread safe because the structures are not expected to be accessed from more than one thread.

>is not expected to need to be thread safe

I'm not "expecting" his example code to be multithread safe either. (By default, I expect all code to not be multithread safe unless it is explicitly engineered to be so.)

I'm pointing out that the code does not work the same way under the covers. The author would know there's mutable state hidden in that "list" object but I don't think it would be obvious to others at first glance. If it was, programmers wouldn't get bitten by bugs because of misunderstanding ctime() and strtok() in single-threaded programs. Likewise, the zlist_next() can also be inadvertently misused to cause bugs in single-threaded programs.

If the author knows how czmq internally works and how it's supposed to be used (don't multi-thread or misuse in single-thread), but outsiders haven't grokked those guidelines yet, can that coding paradigm be labeled as a "dialect"? That's the interesting philosophical question to me. Words like "dialect", "explicit", "syntactic tricks" are relative to the programmer and I think the author's code examples illustrate those subjective biases rather than advise us of objective truth.

I think there is a fundamental difference there in that as you say: You by default expect all code to not be multithread safe. That's a default assumption for most C/C++ developers when encountering a class or library they don't know because it is very common. To the extent that quite a few people argue that libraries should not take extra effort to be thread safe, as is the case here. Besides, it is obvious from the API in this case that there is hidden state to all but extreme beginners.

I think his argument regarding dialects in C++ is sound to an extent. C is quite boring and settled when it comes to structure and interop at the API level. APIs may vary wildly, but e.g. <libraryidentifier>_<classname>_<methodname> or <libraryidentifier>_<functionname> with pointer to an "object" or struct holding generic state is very well established, for example. A lot of the simplicity there comes from the simplicity of the language. The internals of how the code is written varies more, and things like build systems etc. varies wildly. But at the end of it you are usually left with some headers and a bunch of exposed symbols that follows a relatively limited set of styles.

C++ has far trickier issues, in that you are dealing with people who wrap everything in templates; people who try to use functors all over the place; people who want exception safety everywhere; people who ban exceptions in their codebases; smart pointers vs. bare pointers etc. C library code can usually interoperate. C++ code written in different "cultures" can sometimes be like trying to make LEGO and PlayMobil fit toys fit together. Many C++ developers have gone through "phases" - for my own part, before I mostly ditched C++ I'd written codes in styles that appeared totally alien to each other...

Libraries that are not designed from the ground up to be thread safe almost certainly aren't so that's the safe assumption. (And even plenty of libraries that are designed to be thread safe aren't because it is hard to do this right.)

> plenty of libraries that are designed to be thread safe aren't because it is hard to do this right.

Is it? Are we talking about fully thread-safe or only being reentrant?

For the most part, being reentrant is sufficient -- i think there are few cases where it makes sense to be really thread-safe (i.e., lock mutexes on every access as opposed to let the client do the locking as needed) since that forbids composability.

Making a function reentrant is mostly very easy. It's not trivial to assert that external dependencies are also reentrant, but most libraries don't have external dependencies.

Reentrant is more general than thread-safe. Thread-safe can be achieved with hacks specific to a threading model: thread-specific variables, locks and so on.

Reentrant is just about absolute. As in, pretty much, "you can stick this routine into an interrupt context on a randomly selected microcontroller and everything is cool".

Look at this particular zlist example. We can make it thread-safe with thread-specific variable for the iteration cursor. Yet, the function will not still support recursion, or even nested iteration of the list. A reentrant function will.

Making a function reentrant is not easy, if it has arguments which are pointers to various objects.

Only functions that receive everything by value and don't refer to anything global are easily made reentrant.

(Of course, the compiled code for all functions refers to global things: registers, such as the stack pointer. So reetrancy isn't absolute; it depends on machine contexts being correctly saved and restored by all the responsible code, like trap handler entry points. The compiler also can't do stupid things like use static buffers for passing structures (very early C compilers did that) so the source code only looks reentrant.)

You got that wrong.

For example, a fully thread-safe binary tree would be a tree data structure which allows concurrent insert operations on the same tree instance.

Whereas a reentrant binary tree allows concurrent inserts, but not on the same tree instance.

So, your typical reentrant function takes care not to depend on or modify global state. But it might read from and/or write to an implementation-specific datastructure.

Run a "zgrep -l reentrant /usr/share/man/man3/*.gz" which will convince you that's the usual notion. In particular, look at the manpage of qsort / qsort_r.

You can see that making a function reentrant means typically converting static data dependencies into dynamic ones. The straightforward and very often simple approach is using function arguments (plus a contract that the arguments are not shared between "threads").

Making dependencies thread-local is another approach, and "errno" is a good example for that.

No kidding! For instance, the memcpy function is reentrant: but only as long as two contexts don't call it at the same time, and pass pointers to the same memory. It is not unconditionally reentrant. Some functions are; those that don't take pointers to anything as arguments. I covered that in my comment.

An absolutely reentrant function could use memcpy internally (for instance to copy one object stack on its stack to another), so in other words, memcpy doesn't have bad properties that prevent it from being integrated into fully reentrant situation.

An insert function that doesn't handle two threads inserting into the same tree isn't reentrant when called that way. It execution cannot be suspended and re-entered, except with different objects.

The posix _r doesn't mean "absolutely reentrant"; it just means "this function doesn't do stupid things under the hood with globals that would prevent it from being used in reentrant code".

Why do you insist? Please argue with the POSIX people writing these manpages (and correct Wikipedia) if you think that they are all wrong and your nomenclature is the right one.

What you talk about are pure functions.

Only pure functions are absolutely reentrant.

Reentrancy is logically weaker than thread safety.

For instance, malloc is thread safe in POSIX, right?

Yet, you cannot call it from a signal handler, at least not an asynchronous one. This is because it isn't reentrant.

(POSIX would call it "async signal safe", but that's totally POSIX-specific terminology. POSIX uses some terms in POSIX specific ways, unsurprisingly. And "async signal safe" is something less than reentrant because functions can be made async signal safe by manipulating signal masks to avoid being re-entered.)

I'm not the one who made this discussion about POSIX; my original comment wasn't about how POSIX uses "reentrant" (and I don't care, for that matter).

> Reentrancy is logically weaker than thread safety.

It isn't; neither is an implication of the other. If you mistrust manpages, lookup the definition on Wikipedia.

But in many contexts "reentrant" is the better "thread-safe", as in "what you actually want" (give up mandatory locks to get scalability, composability...).

Do you have an example of a function inherently reentrant (pure function), or a function or object used in a reentrant manner, that isn't also thread-safe?

Wikipedia's first paragraph on Reentrancy completely agrees with me; I'm not inclined to change a word.

POSIX doesn't define "reentrancy" or "reentrant", though it uses those terms. They are not defined in the documents it includes by normative reference. ISO C is one of those references and also throws the word around. ISO C refers to a document called ISO 2382, whose Part 1 gives a technical vocabulary; it might be defined there.

By POSIX, of course I don't mean the man pages on a Linux system, but the actual specification. I refer the online version from time to time, specifically this one: http://pubs.opengroup.org/onlinepubs/9699919799/

Exactly.

I would add that this goes deeper than just coding styles. My current project relies on a number of widely-used C++ libraries (Qt5, OpenCV, ITK, VTK and a homegrown framework) and each one of them redefines basic data types (std::string, QString, cv::String).

I could go on about how none of them even acknowledges the existence of <cstdint>. I'm sure there are good excuses why each individual library decides to do this (lack of common ABI, badly designed std::string), but the end result is a heap of forced runtime - and cognitive - overhead.

This is pretty sad considering that C++ is advertised as a language that doesn't add unnecessary runtime overhead.

A lot of it is simply age.

Qt and VTK predates C++98 significantly. OpenCV is slightly newer, but it took a long time for people to get comfortable relying on it, and I don't know whether or not it was based on older, pre-existing code.

But it's probably going to remain a drag on C++ for another couple of decades...

On the other hand, std::string already existed back then, but they still chose to reimplement it.

This points to a more systemic problem in the C++ ecosystem. Has C++ evolved to the point where, say, a new library written today would not have to redefine std::string?

It existed, but if you wanted interoperate with your older code, or wanted to run on various platforms, it often would not be available, or bugfree enough or fast enough. It was well after 2000 before everyone started trusting that they'd have a reasonably conformant C++98 environment. You'd still find people start new std::string alternatives up until then, for seemingly good reasons. Some of them valid, some of hem not so much.

Today I don't think anyone would redefine std::string, but I haven't written much C++ for years.

The standard was finalized in 1998 but the implementations on a wide variety of necessary platforms didn't happen until later. Qt's QString predated 1998.

Also, this SO q&a has more tidbits: http://stackoverflow.com/questions/1618798/why-is-there-a-di...

Not it didn't. The work on Qt standard on 1991, according to Wikipedia. The work on STL only started in 1994 and became a standard with C++98, and even then it took time for compilers to get decent standard support. So every library needed its own string implementation. New libraries generally use C++ string, unless they're have very strong opinions about performance that trump interoperability (e.g. Facebook's folly).

But seriously, isn't that hypocrite to blame old C++ libraries for implementing their own string class, and then go espousing C? C of all languages - the language whose spartan standard library includes almost nothing (including usable string processing code, if you care even slightly about securing your code from memory errors), and each respectable C project ends up implementing their own lists, queues, maps, stacks and of course strings.

No, it isn't "simply age." I know projects started in 2014 that defined their own string types since there are a lot of things that std::string doesn't support. For example, you can't turn a C string into a std::string without copying. There is basically no usable unicode support in std::string. Copies of std::string use copy-on-write, which may create hidden race conditions when sharing data between threads. If you want to fix those problems, you need your own string type.

How does std::string copy on write cause race conditions?

Because if you "copy" a string and the copy really is just a reference to some shared data structure, changing the shared data structure later in another thread without memory barriers or other synchronization causes undefined behavior. Remember that the shared data structure will need a reference count to be updated whenever a reference gets deleted.

Two things to note. std::string is pretty much barred from being CoW as of C++11. QString is CoW, but as of Qt4, cross-thread copies are safe to modify just like normal objects.

> you can't turn a C string into a std::string without copying.

C and C++ use value semantics by default. std::string is a value., a char* is a reference, so this shouldn't be surprising. If you want an immutable, zero-copy, drop-in replacement for std::string, use boost::string_ref (a variant of which will soon be standard). It'll give you a memory unsafe (just like a char\) reference to a C string or* a std::string. Want to wrap a C string safely? Use it in combination with unique_ptr or shared_ptr<char> (for exception safety).

> Copies of std::string use copy-on-write,

Actually that's explicitly forbidden by the standard, since C++11. The interface for std::string always assumed mutation, which made COW really inefficient.

> which may create hidden race conditions when sharing data between threads

The COW versions of std::string, that I'm aware of at least, were always thread-safe.

Thank you for the correction about std::string no longer being copy-on-write. I'm glad they changed this in C++11. string_ref also sounds like a step in the right direction, assuming it is well-implemented.

I'm curious why std::string is badly designed?

I'm also curious why. I wonder if any of the original designers ever shared any insight into their design process.

As for what is wrong with it, multiple things. My personal pet peeves are: 1. the lack of formatting or interpolation support (i.e. QString::number(3.14, 'f', 2) or QString("Hello, %d!").arg("world") 2. the utter lack of Unicode support

I've seen other people complain about how it interacts with <algorithm>, inefficient support for common operations (join, split, copy) and mutability. I don't particularly care about these, given that its lack of Unicode support pretty much relegates it to a glorified std::vector<char> implementation for me.

Yes, you can read "The Design and Evolution of C++" and the "The Annotated C++ Reference Manual" aka C++ ARM.

There you will understand why C++ is the way it is, when you are trying to sell a safer language with higher level abstractions to hardcore C developers at AT&T, with zero friction with existing C toolchains.

Many like me hate the C underpinnings of C++, but if it wasn't for them, most likely no C vendor would ever have adopted it.

I'm tired of people blaming the bad things in C++ on C. Most of the bad things in C++ came from C++ itself, not from anywhere else. The Modula-3 type system with its inheritance anti-patterns can't be found anywhere in C. "Guru meditation" template error messages can't be found anywhere in C, nor do C programs typically take hours to compile. In C, functions and types that are "static" to a file don't appear in header files... C++ "improved" that so that private types and functions must appear in header files, greatly extending compile times. Global constructors are not in C... you know what order your program will run in when using C.

The set of things in C that were "fixed" by C++ is very small, possibly even the empty set for reasonably modern dialects of C. For example, having the compiler implicitly declare functions as returning "int" was on its way out with or without C++.

Probably the worst thing about C was all the implicit type conversions-- between long and int, and char* and boolean. Of course C++ didn't fix this. They even made it worse by declaring that any C++ constructor that took one argument would be a candidate for implicit type conversion. So if you declared Foo::Foo(int i), in the future any time you have an int the compiler would consider promoting it to a Foo. Later they realized their mistake and added the "explicit" keyword so C++ gurus can avoid this error most of the time (when they remember).

The new syntax using curly braces to initialize a variable disallows type conversion and is the recommended method now iirc.

OK, so C++ came out in 1984, and it only took them 30 years to add a way to bypass the implicit conversions they added to the language. Just once, I'd like to hear a C++ advocate say "we made a mistake." Of course it won't happen, so it's time to deny, obfuscate, and redirect blame.

Bjarne has been very open over the years about mistakes he thinks affected parts of c++'s development. There is no perfect language but c++ does keep trying to improve.

Hey, C came earlier and strcat is still in the standard, as well as non-prototyped function calls. Someone told me FORTRAN still maintains backward compatibility with arithmetic ifs, and it's been like, what, 60 years? Strange that.

You are comparing apples and oranges. strcat is a library function which is easy to avoid, whereas the implicit type conversion behavior is part of the core C++ language and very difficult to escape. Non-prototyped function calls are there, but they trigger warnings on any compiler from the last 3 decades. Anyway, thanks for playing the "deny, obfuscate, shift blame for broken C++ design decisions" game.

You're saying directly, C-level style and compatibility at a language-level in early C++ couldn't negatively impact its safety or maintainability vs Modula-2/3, etc? I highly doubt that given most crashes early on were C++ being done in a C-like style, using C runtimes, or calling C OS functions. The C++ style modernists here advocate greatly reduces problems outside compile times.

So, I'm a C++ opponent but it's still obvious C dragged it down a lot. Stroustrup's writings on its origins and why he made specific tradeoffs shows it's a fact. There were often more ideal stuff, from high-level perspective, that wouldn't fit with C, would break legacy, or would take a performance hit C developers would reject. None was allowed as C developer adoption was main goal.

We failed to convince them to use lots of other low-level languages, too. C developers are picky. So, their foundation of quicksand remains a the bottom.

Please think carefully about what you're saying and try to avoid making unfounded generalizations about large groups of people ("C developers are picky" etc.) People were willing to accept a lot of performance compromises to use C++. For example, C++ streams are at least 3x slower than stdio to this day. But they were still widely adopted. Similarly, exceptions caused a performance hit, but were widely used for a while.

There isn't a single "modernist" style in C++, there are dozens of permutations (exceptions vs. no exceptions, boost vs. no boost, quasi-functional vs. C-like).

I was a C++ developer for 10 years. More than half of the really difficult problems we encountered were C++-only, relating to constructs that didn't exist or were simpler in C. The reality is that C++ just has much more undefined behavior than C, which tends to make it less safe and less secure than that language. And some of the most exploitable software out there like Adobe Flash is written in the oh-so-safe C++ rather than the evil old C.

"Please think carefully about what you're saying and try to avoid making unfounded generalizations about large groups of people ("C developers are picky" etc.) "

I did. Over a decade of research, empirical comparisons, non-C-like alternatives to C, C-like alternatives to C, safer subsets of C... literally everything attempted failed to get a huge chunk of C people off C's problems. Even stuff that had most of C's advantages with less of its disadvantages. Tiniest tweaks didn't work. Whereas people on other languages and niches were more likely to experiment and try all kinds of languages for projects. So, I think it's clear the C crowd has some strong, emotional attachment to their language and way of thinking.

So, to get adoption, C++ had to build on that nonsense we were trying to replace. It had to merge very contradictory philosophies. Without the C part, the job would've been easier. We know that's true because it was in a number of safe, system and application languages. Anything with C or C-style at foundation has problems.

Think carefully about what you just wrote and ask yourself if you're being honest, or just shifting blame for your own failures on to "them." The people behind Objective C, Rust and Go aren't sitting on their hands because the oppression of "them." They're trying to make things better for authors of low-level software.

The cult of shifting blame on to other people for C++'s failures needs to end. The language sucks. It's bad and you should feel bad. Period.

I agree with you that C++ streams are rather horrible. This is probably the oldest part of the standard library and the original design predated both templates and exceptions, so it had to do with virtual functions and error states. I wish we could have better library in the standard. If I needed performance, I would rather use a third party I/O library over stdio, which is a security nightmare.

> There isn't a single "modernist" style in C++, there are dozens of permutations All complex languages have several styles, so it's not just a feature of C. Take a look at Scala (mutable vs immutable objects), Haskell (recursion vs. folding, lambdas vs. point-free) or even Javascript (callback-hell vs. promises vs. generators vs. async/await) and it's not so different than C++. If you want one style to rule them all, there's always Go.

> More than half of the really difficult problems we encountered were C++-only, relating to constructs that didn't exist or were simpler in C. Yes, constructs like classes or templates don't exist in C, but their warts are generally born out of the requirement for C compatibility. Excruciating long compile time? Due to compatibility with C's preprocessor-based "module system". Private variables and methods have to appear in header files? Again, wouldn't be an issue if not for the mandated module system inherited from C. Weird template errors? Compatibility with C types made introducing constraints is quite hard (that's one of the reasons Concepts are languishing in the standards committee I guess).

> And some of the most exploitable software out there like Adobe Flash is written in the oh-so-safe C++ rather than the evil old C. This software is more likely written in C with a sprinkle of C++ classes, lots of manual memory managements and, raw C arrays, C library calls and so on. My experience might not be representative enough, but the developers who tend to write this kind of malloc-happy code are the same developers who hate templates, exceptions and RAAI. You might as well call them C developers in a sheepskin.

> I agree with you that C++ streams are rather horrible.

Apparently I belong to that strange group of C++ devs that likes iostreams. :)

I never said C++ streams were "horrible", just that they were slower than stdio and still got adopted. Which kind of negates the idea that among C++ "designers" (I use that term loosely) that you have to make all sorts of terrible semantic compromises to save a tiny bit of performance.

I don't think stdio is "a security nightmare" either (I assume this canard is based on the fact that it still technically includes "gets" or some other such function that nobody has actually used in new code in 3 decades.)

C++ is a uniquely schizophrenic language because you can't even use libraries developed in one style with code developed in another style. Try throwing an exception when -fno-exceptions is enabled in a google code base. Try passing code using a QString to code that wants a std::wstring. Tee hee. You can't say this about Javascript, Haskell, etc. and frankly your comparisons are vapid.

And the C++ sheep keep bleating "it's C's fault, baaaaa" even after I gave the example of Objective C, which has modules and avoids C++ template hell. You even non-ironically blamed C for the C++ concepts feature "languishing in the standards committee." Can you not see that this is the equivalent of "thanks, Obama" whenever something goes wrong halfway across the world? What the hell does C have to do with templates or concepts?

Yes, I'm sure evil C programmers are responsible for all crappy and insecure C++ software. They're in league with the Zionist media conspiracy to smear C++'s good name. We must root out these double agents wherever we find them! Onward C++ soldiers. Never question the decisions of your leaders, which are made for excellent reasons and always make complete sense.

If you cared to follow my posts you would see that I would rather be discussing about Modula-3 and Ada than C or C++.

Yes the blame is 100% on C compatibility.

There is an interview that I can gladly search for you, where Bjarne says he was dismayed the lack of knowledge about other languages and he had to do his thesis in C,swering never to use it again. So when he joined AT&T, he started right away to create his improved C version, which had to be copy-paste compatible with it.

Had C++ had proper modules, it wouldn't work with C linkers.

Those implicit conversions were added to make C++ data types fit into the way C developers were used to write their code.

I can assure you than I worked on a C project in 2000 that took around one hour per platform to compile.

As for enjoying C++, I rather programm in memory safe languages and since Oberon I am big fan of GC enabled systems programming languages.

But C++ was my escape from C, when Turbo Pascal started fading away. Compared with Turbo Pascal, C always seemed a meh language, only thing going for it was better portability as the Pascal dialects.

So, even with its inherited C defects, C++ does provide the tools for the security conscious developers, if one does decide to use them.

Of course, ideally, both should be replaced, but it will take ages until it becomes a reality.

Blaming someone else for your own poor design decisions is cowardly and irresponsible. Better languages than C++ can easily be built on top of C. For example, Objective C is a better language than C++, also built on top of C.

Implicit type conversions could have been eliminated by having a "use strict" mode for new files, as was done in Perl 5. C++ didn't do this because the designers didn't see implicit type conversions as a problem. In fact they dramatically expanded the number of types that could be implicitly converted to include all user-defined types, as I have just described. But I guess the evil C designers somehow forced them to do that? At what point do you start taking responsibility for your own mistakes? It's sounding a lot like "never".

By the way, C++ could have had a module system too. Objective C does, and people are talking about adding one to C++. But then again, Objective C had good designers, not "I just work here" type people who blamed all their problems on the previous engineer.

> For example, Objective C is a better language than C++, also built on top of C.

Somehow I cannot agree that a language that has [] and @ everywhere, relies on the programmer to manually write retain/release, uses dynamic typing and keeps all C defects, has a better design.

Remember Objective-C didn't had ARC in 1983.

> But I guess the evil C designers somehow forced them to do that? At what point do you start taking responsibility for your own mistakes? It's sounding a lot like "never".

Yes, when 100% compatibility with existing code is a design goal.

> Objective-C does, and people are talking about adding one to C++.

Objective-C modules is a work from Apple, relies on clang linker and wasn't part of the original language design.

Likewise, the upcoming C++ modules require the implementation of new linkers.

Both don't work with the UNIX linker model that Bjarne had to use.

Just the C++ compiler was new. The linker had to be the system linker already in place. This is how name mangling came to exist.

Languages that use modules and have their own linkers don't rely on name mangling.

There are lot of "roads not taken" that would have been better than what C++ actually did. Clearly Bjarne could have implemented his own linker-- if Go, Rust, Objective C, etc. etc. can do it in a year or two, then Bjarne could have-- he's a big boy, after all.

Even without rewriting the linker, you could create an informal module system the same way the linux kernel does-- by having well-defined and cleanly separated interfaces between modules which got linked into .a (library archive) files. Of course C++ failed to do this, either at the (nonexistent) module level, or even at the library level. C++ ABIs are a complete mess even to this very day-- it is effectively impossible for anyone but hardcore C++ gurus to know what changes to a class will break library compatibility. The best solution is to wrap your C++ API in a C API. Because C was a better thought-out and designed language, and can support stable APIs, whereas C++ can't.

> There are lot of "roads not taken" that would have been better than what C++ actually did. Clearly Bjarne could have implemented his own linker-- if Go, Rust, Objective C, etc. etc. can do it in a year or two, then Bjarne could have-- he's a big boy, after all.

Then he would have failed to have zero friction with the tools being used at AT&T and no C developers would have "incrementally" adopted C with Classes, as C++ was known back in its early days.

> Because C was a better thought-out and designed language, and can support stable APIs, whereas C++ can't.

C++ was designed to support copy-paste from C code and use the same linkers.

Everything that C can do, C++ can do exactly the same way.

It's not blaming someone else. Bjarne worked at AT&T Bell Labs, and once C++ caught management's eye and became more than his own personal project, 100% compatibility with C++ was a business requirement. Bjarne ended down expanding a huge effort to bring that down to about 99% compatibility, to avoid some of C's most insidious warts such as prototype-less functions, but compatibility was still a goal.

That goal included semantic compatibility (C++ constructs shouldn't feel alien to C) and performance equivalence (C++ abstractions should have the same performance cost as C).

That's why Objective C's good(?) design of strapping Smalltalk's syntax and semantics to C inside square brackets was never an option. Objective C containers not supporting C primitives would have never made it.

And let's not forget that Objective C uses dynamic dispatch and must box primitive types, so it could never claim to be on par with C performance-wise.

It seems you are blaming C++ as a language for the design of the libraries you are using.

I've also never seen a program actually be slow due to string manipulation (excluding grep). My guess is that if you profile, it will turn out that your performance is getting away from you in places you don't expect.

> It seems you are blaming C++ as a language for the design of the libraries you are using.

Precisely. In the same way that Java is criticized for proliferating the IAbstractFactoryVisitor design anti-pattern, C++ should be criticized for its weak standard library which causes an overproliferation of mutually-incompatible reimplementations of basic functionality.

As to whether this is or is not a performance issue depends on the specifics of the application and the underlying hardware.

This is besides the point to this discussion. Why don't C# libraries reimplement System.String like C++ libraries reimplement std::string? That's the real question here.

Because when C++ appeared, its standard library was basically C.

Then you had the early adopters that were basically the same developers that were still writing C with lots of inline Assembly.

So each developer, or team, had to write their own library, because they knew better.

It took all the way up to ANSI C++ (1998) to finally have a proper minimal library. Minimal, because C++ needed to cater to the C mentality that the OS is the runtime.

Then you had almost a decade until the C++ vendors were able to catch up the standard.

Mind you that in the mid-90's there were still quite a few C compilers that were K&R only.

This is why when Java appeared, many of us C++ devs that cared also about safety, jumped into it. As it provided a more portable experience.

Modern libraries don't reimplement std::string. The C++11-14 standard library is actually pretty solid.

I was actually curious if this is true, and it appears that Unreal Engine 4 reimplements std::string (FString) [1] I think this qualifies as a modern library.

My yardstick for "pretty solid" is actually meaningful Unicode support, so I will reserve judgement as to whether the C++11 STL qualifies.

[1] https://docs.unrealengine.com/latest/INT/Programming/UnrealA...

Unreal has been evolving for over 15 years and the engine does lots of preprocessing as well as being built for interactive C++, so I don't think it is the best example.

Death by a thousand strings is a very common performance diagnosis.

"Death by a thousand strings" I love this phrase, I hope you don't mind if I shamelessly copy it for future use!

Glad to help :)

In C++ with small string optimization or in other languages?

I'm thinking C++, but I'm sure other languages have it too.

Is the zlist version even re-entrant?

That's also a good point. You can't use zlist* functions in recursive algorithms in single-threaded code.

Whether such recursive code would be an artificial contrivance or a real-world misuse is something left to the reader's imagination.

I notice that the zlist_copy() function was deprecated so you'd have to do something else if you wanted to pass a copy of a new "list" to subsequent recursive calls. I didn't dig deeper into the czmq documentation to find out what the recommended guidelines would be. (Although we'd guess that any copy operation would be O(n) and therefore thinking in terms of recursion would be an antipattern for czmq.)

You don't need recursion to hit the wall - how about comparing all pairs of elements? I.e. inner loop that iterates over all elements once again.

It's zlist_dup() and you would not use recursion to process these types of lists. The API is designed for iteration. One case for recursion is when you use lists of lists of lists... you can then process such structures recursively. See zconfig for examples of that.

> The C++ loop is multithread safe. The C version is not.

Neither of these loops are multi-thread safe. That's assuming that's an std::list, and even if it isn't you can't implement that C++ API in a fast and thread safe manner.

Yes, the ZMQ list is awkward (why did they put the "cursor" or the iterator inside the list object)? But that's not an inherent property of the C language.

If you want an example of how to implement a linked list in C elegantly and efficiently, look at the intrusive linked list implementation in the Linux kernel (also available separately in the CCAN project).

>Neither of these loops are multi-thread safe.

Yes, as a general rule, all STL containers in C++ are not thread safe. (E.g. Write operations need locks. Read ops also need locks if other threads can write to the container.)

However, I was looking at the specific loop example and only saw .begin(), .end(), and a pointer dereference which are all read operations. These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that.

What if a background thread deletes a node from the list while the iterator is running?

From the cpp reference: "Concurrently accessing or modifying other elements is safe, although iterating through the container is not."

That's the problem with multithreaded code. You can't just see a read loop in isolation and say "this is thread-safe" - it's not.

> These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that.

In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism :)

>In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism

The outer context can affect whether or not an STL begin()/end() loop needs locks. For example, many programs can be coded to work in phases:

  main() {
    setup_lookup_tables(); // phase 1:  fill up (write) to lists/vectors/maps
    analyze();    // phase 2:  spawn multiple threads to read the lists/vectors/maps
  }

The code in the analyze() doesn't need to pay for millions of expensive locks+unlocks that slow things down. The programmer knows there are no writers to STL containers in the analyze() phase. With the assumption of no concurrent writers, the programmer can safely spawn a dozen threads with for(list.begin();list.end();) and get 100% determinism. Explicit sequential ordering can let one avoid unnecessary locks. In this limited sense, the loop is multithread safe for multiple readers.

However, even with the assumption of no writers to zcmq's "list", you still cannot spawn multiple reading threads of zlist_next() because the cursor itself is mutating inside the list. The result will not be deterministic.

That's the type of comparison I was making. You also have a valid point about general multi-threaded code covering all cases (implied concurrent writers).

The CZMQ list embeds the cursor because (a) it is sufficient for all cases we've encountered and (b) it makes calling code simpler. This iteration pattern is widely used in CZMQ and Scalable C applications like Zyre and Malamute.

zlist_first and zlist_next can be made thread safe by using a thread-specific variable for the iteration cursor, rather than a simple member of the list structure.

Of course, that's a big, inefficient hack, and tied to a particular threading model (not generally reentrant).

Anyway, it's probably largely moot because neither example is safe if the list is treated as mutable (there is at least one writer concurrent with the reader).

> So why is using something non-standard and non-universal like <czmq.h> not perceived as a "dialect" of C?

Somehow many fail to acknowledge that many write in compiler dialects, and think "my compiler == C language".

Yet commercial compiler vendors are always criticized for their language extensions or having other interpretations for the parts not defined in ANSI C.

CZMQ is a library of around 30K lines of code. The Scalable C book is in many ways a guide to using that library in real projects.

Of course every programmer develops their own dialects, just as every writer has their own voice. C is however a small language and if you stay away from weird macro magic, any well-written C code is mutually intelligible to other C programmers. Whereas with C++, dialects can have so little overlap they are not mutually intelligible.

FWIW neither of the fragments is safe if you are sharing state between threads. In practice we do not share objects between threads, and each object holds its own state, and thus our C code is 100% thread safe and reentrant.

>The Scalable C book is in many ways a guide to using that library in real projects.

I'm not criticizing the book or czmq. I'm sure it's a fine library. I just found your characterizations of C++ to be strange.

For example: ">, any well-written C code is mutually intelligible to other C programmers."

If you're going to qualify C code with "well-written" to help make your point then can't the same qualification be applied to C++? If so, it means you believe that there is "well-written C++ code" that is simultaneously unintelligible to other programmers. In your opinion, what would be an example of that? (E.g. If you can point to github of unintelligible but well-written C++ source code.)

And your other statement: >, because your particular dialects of C++ tend to isolate you.

I don't know if CryEngine and Unreal game engines are well-written C++ but they seem attract developers. There's also the scuba diving app[1] written in C++ that also has Linus (who you already know hates C++) modifying the source code. I contend there are bigger factors than C++ dialects causing isolation.

The C++ ABI may be more isolating than the C ABI because of C++ name mangling incompatibilities. But I don't see how the C++ dialects (of well-written code) are isolating.

[1]https://github.com/torvalds/subsurface

Given the sheer breadth of approaches and features in C++, any one codebase will only touch some of it. Inevitably this means that if you get used to C++ in one project, you will have more difficulty moving to projects using a different subset.

Since you ask, I'll admit it: I enjoy trolling C++ users because the language has so often thought of itself as superior to C. Beating on the language always gets lots of discussion going, which is fun. I've nothing against the language, or any other language, as such. Good tools, in the hands of good developers.

I find it the other way around. It's usually C hackers who tend to describe C++ as the devil incarnate. C++ developers tend to be more pragmatic - we'd happily write in C if necessary, we just find it limited. And since C++ is almost a superset of C, there's rarely any need to use it.

Thank you your honesty. Now, C++ developers can stop trying to have a productive conversation with yoir new generation of hipster C programmers crafting smal-batch artisinal bytes, and get back to getting work done.

From experience, both unreal and cryengine are perfect examples of Pieter's point. They're large codebase with archaeological layers of many different c++ styles and paradigms.

>perfect examples of Pieter's point.

Please consider the precise argument I was responding to.

Pieter's point is a stronger statement than just the existence of "different styles". (Every significant codebase by multiple people has multiple styles -- e.g. see Linux distribution.) He was stating that there was a cause and effect such that the different styles ("dialects") cause "isolation".

If so, it means Apple's Swift compiler written in C++ on github[1] should be getting near zero forks and pull requests. In his theory, Chris Lattner and his dialect of C++ should be "isolated". It's also possibly unintelligible C++ (although I haven't looked at it yet).

If it was just one of those throwaway sentences to troll people, then fine. My first impression was that he wanted his programming guide to be taken seriously.

I enjoy reading C++ criticisms. The criticisms in the C++ FQA by Yossi Kreinin has interesting points. I'd prefer it if the faults of C++ were accompanied by evidence.

[1]https://github.com/apple/swift

Yes, like I said, speaking from experience, different C++ code styles and paradigms absolutely do cause isolation. Having to climb yet another mountain in order to be effective on a different codebase in the same language means people don't do it, so they become experts in a single codebase -- isolated.

I've never really got the "C++ dialects" argument so popular with C enthusiasts. Sure, C++ is several order of magnitudes more complex than C, and this complexity breeds different dialects. But for an experienced C++ programmer, that's a stylistic problem (you generally end up disliking the dialect of about half of the libraries you use), not an intelligibility problem.

I've read many libraries, both C and C++, and I've never found C++ libraries particularly hard-to-read. There are a few libraries rife with deep template magic (Boost.MPL et al.) but they are hardly the rule.

The real problem with C++, is that truly competent developers are hard to find. I've worked with and interviewed countless C++ developers. Very few used anything from C++11 (except for auto perhaps), and most are essentially program C++ as C with Java-style classes bolted on top. Templates are few and far between, and manual memory management abounds.

So these incompetent C++ developers you hear about in Linus' frequent rants? They're mainly C developers who learned some C++.

Good C++ developers write cleaner code that's easier to read compared to C, but it takes more time to train one than it takes to train a C developers. But if you only care about fast developer training, Java, or even better, Go are much better choices than C.

If one particular language suffers so much from all its programmers being incompetent, it should raise the question of what about the language makes it so hard. You can't reasonably dispute that C++ is a gigantic language full of optional features. To be competent with C++, you really have to learn which features to use or not. It's too bad that not everybody agrees on which features to use! Blaming outgroup C developers for everything is a cop-out.

Oh, if it was only so easy! There are many other languages which abound with incompetent programmers. Even an atrociously simple languages had many incompetent programmers over the years. On the other hand, there don't seem to be a lot of incompetent Haskell programmers.

It's usually practical reasons which determine the influx of newbies with no previous experience and little incentive to learn to a language. Popularity-wise, C++ was the Java of the 90s, and many programmers came during that time and don't want to learn anything new. Many other programmers are essentially not willing to learn C++ features beyond the features they're already comfortable with.

"Which features not to use" is another myth propagated by professional C++-bashers. There are very few features in C++ that you shouldn't use, and most of them are officially deprecated parts of the standard library. The real problem is developers who refuse to learn templates, because "void pointers are simpler", or reject std::array, because it's fancy C+11 stuff and writing "char myArray[100]" is much shorter than "std:array<char, 100> myArray;". They then go on to forget std::array ever exist, and complain that vector is too inefficient, so they're going to use C arrays all the way.

If you're willing to use all these newfangled features like closures, initializer lists, 1-part for or constexprs, its' very hard to shoot yourself in the foot. It's very easy to shoot yourself in the foot by using raw C arrays or manual malloc/free though.

As a Java developer, I have found the source code of well-designed C++ libraries (like libtorrent) actually a pleasure to read. While C libraries need lots of 'macro and typedef' hunting. Doable but a chore.

Unfortunately, the Boost libraries are really arcane - template magic that gives me a headache. So, I guess I fall into 'incompetent C++ developer', but that is not a surprise as it hasn't been my working language for some years now.

> The C++ loop is multithread safe.

Of course, it's not. If another thread removes the item that i refers to while i's thread is in its cout part, then this loop will go out in flames. Either functionally or semantically. And if it will not (meaning that List explicitly supports this sort of re-entrant case), then you basically get here not just a list, but some multi-purpose construct that can withstand a nuclear strike and the fallout - not something you'd actually expect when looking for a basic list container.

However this is exactly the kind of thing you'd typically get in C++. In contrast, with C you get a basic minimum that does what it says on a tin - links things into a list. You want it "multithread safe", you arrange for that externally. You happen to need that frequently, you wrap it into mt_safe_list container.

That's the key difference between C and C++ mentality. C++ does things universally and you can pick a sliver that you want to actually work with (leading to dialects), while C generally favors doing things in small functional units and combining them as needed. The Unix way. You can, technically, call the result of such combining a dialect, but that's quite a bit different dialects arising from the C++ usage.

Exactly the kind of code I make.

I use C for performance.... as an extension for python that have a GC and a GIL. But more than never, I first use numpy (fortran) because it is dazzling fast and has specialized tricks of digital signal processing availables (ifft). And I do C after the profiler says to do so. When needed. If needed.

Most of the dynamic data structure (message sent with zmq) the config, the parsing are better handled in python.

And since I do as much sloc / day in C and python ...

(1 python loc = 6 C loc)

I code 6 times faster.

And I don't have the headeaches of the dependency management

I am totally okay with C, but when doing distributed system more often than never you also multi-threading. And C is not builtin for thread safety, it is harder.

So I C some masochisms at work here.

And I am pretty sure I am not the only coder thinking this book is full of pedantism and of advices not to be followed and it is empty.

How did such a poor news made it to the top?

> more often than never you also multi-threading

This is a Windows-ism that's crept into Unix-likes over the years, and it's not a good trend. Just don't do it. Especially since sharing direct access to state makes it harder to decouple components to scale them further. This book specifically argues to pass state via IPC for a reason.

> And I am pretty sure I am not the only coder thinking this book is full of pedantism and of advices not to be followed and it is empty.

Maybe, but iMatix has been a successful company for more than two decades, and delivered impressive open source applications (e.g. Xitami web server) and code-generation tools (Libero ec.) as well as cross-plaform utility libraries for C (SFL etc.) already two decades ago, and have gone on to develop large-scale C-based distribued systems and in the process developed things like AMQ and 0MQ that's been incredibly successful, so I for one tend to at least pay careful attention as they actually do have a track record.

I know a lot of successful companies with products that are notoriously poorly coded : - "security software"; - "game industry"; - "car industry";

And if you read correctly I think the biggest issue is TIME is MONEY.

What are the advantages of using techniques that:

- are expensive (productivity is constant in sloc whatever the language);

- are notoriously a systemic risks given their domain of work and is hard to audit;

- that can be smoothly achieved by upgrading a faster to build architecture in a scripting language ;

They may have bigger balls of steels than me doing it in C and be the best programmers in the world. My question is business oriented: what is the economic rationale of a full C solution from start?

Beats me.

I know that if I had a large enough project and had to add people, I have a dozen people in my Rolodex who speak 'C' at the expert level and that they will perform. But depending on the problem domain, that might mean C++ or it might mean Python or something else.

But given the level of hostility the language inspires, I have to wonder. To wit "bigger balls of steel" and "best programmers in the world." Both sentiments are quite foreign to the sorts of environments I've worked in, I assure you.

Sorry for you then. Living in such a boring world, and what a disdain-full answer that makes my point.

I have add my share of conferences technical or about FOSS. And I met peoples with a lot of passion ... and code delivered. You probably use their software daily.

I have been using more than 13 langages ranking from C to forth, matlab, vhdl, spice, python, perl and php.

There are definitively cultures associated with languages and beliefs.

Perl community is thinking coding is like speaking/writing a foreign language;

Ruby about you IQ and technical skills are totally correlated with how nice your apple laptop looks like and how expensive it is. They are our hipsters (troll);

Python secretly hides a sect hating braces and everything that looks like C and believe C coder can't make safe thread code, malloc, correct string handling. And they hate braces.

for c++ coder referring to linus torvalds rant would be the spirit.

Java coder believe in the utltimate safe portable VM and the power of GoF. And think people look the wrong way;

Haskell thinks of themselves as alchemists loving to use obscure terms coined by an hallucinated metaphyscian priest that said ET must exists and that no one will notice. They still laugh of their ultimate joke;

And C coders think that only them are the pure programmer, the only one that can see the matrix between the purity of abstraction and the undetermination of hardware/norms due to the imperfection of the humans. But, with their discipline that is above the norm (no noob accepted) they can fight the God of Entropy

FORTRAN coder think that computers are a pain and would just like to have exact figures much more than nice looking interface and wonder when a correct intuitive language will appear (<--- My sect)

They appear maybe because for each language comes a practical field of use and that one computer language cannot fill all the needs.

The need for correction and exactitude in science conflicts with the "ease of use" of numbers.

The need for having cheap workforce conflicts with efficient cheap to maintain code;

The need for preventing embezzlement (origin of SQL) conflicts with creative accountability;

At one moment, at my opinion C is like a middle age corporation. Trying to promote a one best way of CS that always boil down to C.

C community maybe "professional" as opposed to "enthusiasts". But I think it does not always serve them.

And I do not think that recognizing Computer Science is a peaceful uniform land, but an arena full of organic entities in conflicts with logical distinct rationalities for the same resources.

In short, I have the write to mock other cultures.

Nicely put. Very nicely put.

I don't know how you came up with "disdainful"; it's more sort of sad and weary as I read it now. After all, I started with "Beats me" - such a decision would have to be very local. The first rule of 'C' is "don't use 'C'" these days... the people I know DON'T swagger; that was my point.

The "professionals" vs. "enthusiasts" divide is extremely interesting in all fields of endeavor. I'm definitely on the "professional" side.

I... don't think 'C' programmers are "above any norm"; they just sort of know where the rocks are right under the surface of the water. It's more difficult to explain than to do. If a bunch of people misrepresent themselves as ... badass because they sling 'C', I can't help that. The appropriate mentality for it is one of caution. I specifically called that out here...

It also matters less because coding a system is roughly 5-10% of the actual cost of most deployed systems. Language matters much less than mechanism.

Meanwhile, the worst horrors are inflicted using systems like SAP.

Don't feel too sorry for me; I use at least three language systems every day, and have messed with ... dozens ( all resulting in deployed code at some level) , including graphical CASE tools.

If you are building large scale systems, the amount of time spent writing and testing actual code, rather than collecting requirements, designing the solution etc. is typically small.

Imatix also specialise in code-generation. One of their earliest open source releases was Libero - a code generator to generate state machines. The point being that they have spent a couple of decades on solutions based around minimising the sloc, and while a lot their output is in C, a large part of the code they write is not C, but code generators and input to code generators.

In terms of their techniques, a both the code generation and many of the other techniques they advocate are about reducing and isolating risk: Generate code so that you DRY things up and can focus on testin a smaller code base; split code into modules separated by process boundaries and message-passing, giving you smaller modules to test, boundaries across which you can add high availability (distribute message across multiple instances) or handle failures by queueing and so on.

And as much as I love "scripting languages" - almost all my code is Ruby these days - there are many areas still where C is necessary for performance. The approach advocated in this book fits neatly with that: Nothing stops you from interoperating with any language you want - the message passing creates nice boundaries to isolate components in whatever mix of languages you want.

But also keep in mind that they've been doing this for more than two decades. They have amassed a lot of tools and a lot of experience in doing C fast and right. It might very well be that the economic case for/against C or other languages would look very different to them if they were to start today without that experience.

That said: It also isn't all about economy. I regularly reject job offers because it's not what I want to do. If we all made all our decisions based on time being money, most of us would be in different jobs.

I'm in the exact same camp. I call it "two-language programming". Python+C beats the heck out of any single systems language, especially C++. And thanks to the amazing bloated complexity of the C++ language, the cognitive overhead of Python and C combined is way less than C++ (unless you stick to a tiny subset of C++ in which case you're just doing C-only, with slightly cleaner syntax).

I'm still deciding on the best approach for the interface between Python and C. I don't like ctypes/cython etc because they require you to modify python code. I'd much rather create CPython extension but there is too much boilerplate. I'm looking into swig.

You can also #include "python.h" in C and use cpython to handle from C dynamic data structures and manually do the incref, decref.

If you are totally crazy you can also manipulate the frame to bring exception with stack trace in C... I don't do black magic. :)

Very wrongly vulgarized you can integrate python as a runtime for your C code.

IIRC with Numpy you get CBLAS by default, so not Fortran. But I don't believe there's a noticeable speed difference between them

This looks like it's coming from someone who doesn't know C++ well and is just coming up with reasons to fit their bias. The fact that he/she didn't mention any disadvantage to the C code written beside verbosity makes it clear.

For one, it's easy to forget to call zlist_destroy. Who owns what in C can get very complicated and you can run into dangling pointers. At least in the C++ version you can manage ownership easier in their case.

I am not defending one language over the other, I use them both and have experienced the advantages and disadvantages of each.

What's being shown in this book is not how typically you create link-lists. man queue(3) to see how it's generally done.

The C++ for-loop is not how you typically iterate over a list , again the author decided to show a bad example to confirm their bias:

  for (const auto& i : List)
    cout << i << " ";

More likely, someone who hasn't programmed C++ in the last 10 years. Forgetting auto, and using the cumbersome 3-part for loop with iterator boilerplate when you only need value shows age. Initializing the list is also easier now, with initializer lists syntax, so you could just do:

  list<string> lst = { "tomato", "grape", "apple", "orange"};

and cut another 4 lines, making the C++ line count half of C version. Not a negligible difference, as the author claims.

Nice work in progress, Peter. Look forward to seeing more of it given your prior work. I light how you preempt many C-related counterpoints with model-driven development that generates C. Done excellently by iMatix and many others. I'm especially interested in how you'll apply that to distributed C.

I'm a cpp programmer who recently spent a week learning zeromq to replace named pipes in a project. By the end I was disappointed by the cpp language bindings as they only cover the low level library. Had I known from the start I probably would have looked elsewhere. Its a shame cpp is ignored in much of the open source community in favor of c. If nothing else Cpp after all is a safer c.

   http://250bpm.com/blog:4
   
   http://250bpm.com/blog:8

Yes I read those before. I didn't find them convincing. Intrusive lists is an anti pattern that you can also do in cpp if you want. Getting rid of exceptions doesn't mean you get rid of errors. It just means you can more easily ignore errors and continue running a corrupted program. But the big picture though is that cpp I a safer language. A core library might be written in c for whatever reason. But its good to provide a wrapper in a safer language for users to use.

I blame the rise of GNU/Linux for it.

Given how the market of programming languages looked like in the mid-90's, I bet C would have lost many users if it wasn't for the rise of open source.

Remember that the GNU manifesto had for long time C as their main language.

In the early days, this entry spoke only about C.

http://www.gnu.org/prep/standards/standards.html#Source-Lang...

> //Solution: make /usr/local writeable.//

> This is a brutal and effective solution, the best kind of solution

I... uhm, what?

> Solution: grab the latest CZMQ git master from github.

No, you do not want to run your software off of master, and the fact that Master doesn't always build (because of errors) should be a fringe occurrence with CI now being free/cheap and highly flexible.

I'm not sure about CZMQ, but I assume what you eat is that you don't want to run code from a development branch that's rapidly changing. That's not necessarily what master is in all projects. The master branch is sometimes used as the latest stable release.

True, I forgot about that aspect. However in this case that doesn't apply either as a stable branch should always compile.

during the last years, master wasn't building a couple of times, it usually got fixed in a couple of minutes.

It's funny how this centered around ZeroMQ, which is written in C++.

This might be part of the reason:

* http://250bpm.com/blog:4

* http://250bpm.com/blog:8

Interesting, in this case it seems the choice to keep using C++ is just the most practical decision, which is much better than telling people to use C, but developing a library in C++.

Which are weak arguments that point more to the author's dissatisfaction with the architecture of libzmq than with problems in C++ language. This was discussed previously here:

https://news.ycombinator.com/item?id=3953434

It's worth noting this was written by a different author.

It is at heart a guide to using CZMQ, the high level C binding for ZeroMQ. More broadly, how to write scalable C applications like Zyre and Malamute. The fact ZeroMQ is written in C++ is invisible and irrelevant, except that it's easy to wrap in C.

The point is that you are telling people to use C instead of C++ and that the best way to do that is by using a library that is written in C++. I completely understand that it's invisible, but it's not irrelevant -- it's not eating your own dog food (given that you are the author of ZeroMQ).

ZeroMQ is a community of many projects. The old core library is libzmq, written in C++ by hundreds of people, led originally by Martin Sustrik. The library I'm explaining in this book is CZMQ, a high level C binding for libzmq that offers such things as actors, pollers, certificate management, beacons, and containers. CZMQ of course uses libzmq to send messages between processes and threads. In fact libzmq exposes only a C API, not C++ at all.

Github repository: https://github.com/hintjens/scalable-c

I'd be interested in this if it were nearing completion. I think structuring the book around problem-solution pairs is a nice technique. But it would be a much better read if fewer irrelevant statements of opinion were strewn in. Also, a lot of the bizarre statements of irrelevant "fact" should be checked, for example:

> In the Old Times, creating a repository was days, weeks of work.

I can't begin to comprehend what this may mean. "svn create" (or whatever it was called) was always instantaneous. Setting your project up for network access took longer because you had to read docs and write a config file, but the same is true for Git.

> Optimizing compilers (...) may remove assertions.

Bullshit. Using NDEBUG removes assertions, and yes, this indeed means that assertions must be side-effect free. But an optimizing compiler? No. If that actually happened for calls to impure functions (and no, it really doesn't happen), it would be a major compiler bug.

> a nasty reminder of the old days when computers stored data and code on different kinds of rust, and languages enforced that

Code and data do live in different places in memory; nowadays more than ever, for reasons of security. C's original "declarations before statements" rule (the context here) was simply because it makes it much simpler to write a primitive single-pass compiler.

> The standard C library often puts destination arguments first, which is a hangover from assembly language. MOV X, Y.

... or maybe it's an analogy with assignment statements, X = Y?

Note that three out of these four examples are just irrelevant opinions, so they should be removed from the text even if they weren't factually false.

Creating an svn repo was fast, yet you could not use it without a dedicated server, DNS configuration, security configuration, firewall configuration, etc. etc. If this was your only job, sure, a few hours' work. For the rest of us, begging a sysadmin or spending days learning the details.

Whereas with git it's literally "git init ." or clicking on Github.com and we're ready to roll.

I do appreciate the fact checking, and you're welcome to send me more comments. Errors of fact don't survive the editorial process, one hopes. Opinions, that's a different story.

> with git it's literally "git init ."

That doesn't magically give you a shared, network-accessible repository with all the correct access controls.

> or clicking on Github.com

SourceForge has existed since 1999, and after a click you have always been ready to roll.

> and you're welcome to send me more comments

But I probably won't if your strategy is "spread misinformation first, then make others work to point out mistakes, then defend an indefensible position, then maybe change it". That's not how communities are built. That's what you yourself criticize in the section on merging strategies...

> then you know where C stops working, as a language.

He actually makes a really strong argument against using C right in the first two paragraphs.

C is a dangerous language. Assembly is even more dangerous. There are languages that compile to close the same speed and are systems oriented with 0 overhead.

I'm truly curious, if you're working on a new project would you pick C? Or would you reach for something that's going to reduce the bugs that inevitably come from writing even 10 lines of C?

What languages would you personally pick over C?

Rust, no debate.

this seems to be still it is early nascent stage, with a complete toc missing, most likely, in the works. caveat emptor.

Yes, indeed. I've updated the book title on Gitbooks to make this clear. I'm writing and publishing the book piece by piece, to get feedback early on in the process.

I know the basics of C, but stopped short of getting deep into threading and concurrency since it seems like Go and Rust handle that in a more efficient way (although there's no way I would use Rust in production yet). Are there any advantages to using C/C++ for a new large-scale project?

The main advantage is a huge body of mostly functional code that you can attempt to integrate into your build system.

The main disadvantage is that good C++ programmers are very hard to come by.

Good programmers are hard to come by, and that will always be true, regardless of what language you choose.

Very true, and that's why newer languages are designed to limit the amount of damage average programmers can cause, while maintaining the same level of performance. See Rust for example.

Sorry, small rant incoming. Ignore at will!

I would still argue that good C++ programmers are exceptionally rare. I've worked with C++ professionally for roughly a decade (and as a hobby for many years before that), and despite using it and reading about it and fighting it and reasoning with it, I still don't consider myself a good C++ programmer. I've met several self-proclaimed good C++ programmers, but I've only ever met one I'd genuinely call good.

He had experience in a dozen languages, half of them functional, and he produced code that was concise, maintainable, fast, documented, with correct pre- and post-conditions. Perhaps the best part was that he created interfaces that were a joy to use and very hard to misuse. He didn't like C++, but he knew how to make it work anyway. It was a joy to work with him.

Contrast with your average C++ programmer who will overuse templates (hey, it's generic), generate a deep class hierarchy (hey, it's object oriented), of course with tons of setters and mutable state (parallelism? isn't that like std::thread and stuff?) and then waste time micro-optimizing anyway (hey, C++ is fast). Bonus points for avoiding the STL because "the STL is slow" (== game programmer, although this is becoming less common nowadays). Extra bonus points for arguing that GCs are slow, and then littering the code std::shared_pointer everywhere.

These days, I go as far as to say that anyone who only has C++ in his toolbelt is not a particularly good C++ programmer, simply because of the lack of perspective.

Just for curiosity's sake, what specifically would make you not use Rust in production yet?

Not OP but from the perspective of the industrial side: no track record, no formal standard, changes too fast, incomplete documentation, doesn't have an extensive commercial and supporting ecosystem (e.g. Parasoft, Java Path Finder), limited pool of experienced programmers with embedded and regulated environment experience. Arguably, it falls under the heading of "too new".

I'd prefer to deal with known knowns rather than the known unknowns or gasp unknown unknowns of something new. It's a very conservative position, but it's borne out of the expense associated with mistakes and corrections of.

Cool thanks! I'm trying to figure out what blockers are so we can prioritize things; a lot of these are very reasonable, but not immediately actionable things for me. Sounds good. :)

Sorry I can't offer anything more specific and actionable; I guess comparing Rust to a fine wine, something that must be aged to reach full potential, will have to do :)

Hehe, no need to be sorry. It's one of the best answers, actually: it means that there aren't any fires, it's just about playing the long game and letting time pass. I prefer that. :)

I don't care quite as much about the lack of a proven track-record -- I understand the value there, but I love tinkering and trust Rust enough to not have any concerns about core stability (maybe a mistake, but there it is). It's mostly the lack of a stable HTTP library / ecosystem that throws me off; it seems like everything there is changing pretty frequently. That's just my impression though, and I'd love to be wrong :) I mentioned in another thread as well that the last time I played around with Rust, it seemed like most of the useful Cargo packages were dependent on the nightly version, which definitely gave me pause. I can't seem to find any of these packages now, though, so that's either changed or was an incorrect impression.

Thanks :) hyper is pretty much the defacto HTTP library; there are multiple web frameworks, but they're all on top of it.

> While C lends itself to building libraries, it has no consistent API model.

What language has? Wouldn't this require first class modules (which few systems have; JS' hacked together solution is obviously not to be considered a true solution)?

OCaml has a typesystem in which modules are first-class citizens, just like functions. They have a clear separation between interface (they call it "signatures") and implementation. The compiler enforces that you can only write against the interface, making modules with the same interface really exchangeable. The modules are also parametrizable (they call such modules "functors").

http://caml.inria.fr/pub/docs/manual-ocaml/moduleexamples.ht...

https://realworldocaml.org/v1/en/html/functors.html

https://realworldocaml.org/v1/en/html/first-class-modules.ht...

That's all true, but it doesn't mean that OCaml has a "consistent API model", whatever that may mean. Unless "provide a fold and a map for all datatypes", which I guess is consistent across most APIs, is a "model".

Java, C#, heck even PHP has classes support for languages these days. So has JS if you don't mind compiling using traceur.

So far I've only read the Preface and part of Chapter 1. What bugs me is this:

> * Write portable code that runs on all platforms.

Ok, good plan.

> * An operating system you are comfortable with. Linux will give you the best results. OS/X [sic] and Windows are usable if you have no choice.

So... results vary by platform? And then after the "hello world" example:

> And you should see that familiar Hello, World printed on your console. If you are using OS/X [sic] or Windows, it won't be this easy. I'll repeat my advice to install Linux.

Funny, this example works just fine for me on OS X. You do realize OS X is a Unix-like system, right?

> Having said that, remember this rule:

> Linux is the native environment for C development.

Gee, I wonder how people like Dennis Ritchie ever managed to write C code before Linux came along?

those three points in the bullet list near the start all seem to miss the mark for me.

Love it.

I dream of the day when all current system-level fads bite the dust, replaced by new fads, while C is still running as the system layer. (hint: just like it is happening today with the 90s fad called C++, replaced by fads like Go, Rust, D).

lol? C++ is hardly a "fad"!

Well, IMHO it was, and C was too (google books magazines from 80s and 90s). The people left using both languages are usually doing so deliberately rather than because it is trendy.

Reply on news.ycombinator.com