Hacker News Reader

Comment by Steven_Vellon | Hacker News Reader

A lot of C developers I know see value in keeping the C standard library small. I implemented my own `l_string` (length-string) struct and associated libraries for a class (we weren't allowed to use external libraries) and it was not prohibitively difficult. Something like this probably covers most use cases: https://github.com/antirez/sds

View on HN

Sadly, real C standard libraries are anything but small. glibc is pretty huge; statically linking a simple hello world program produces something like 750KB of output (libc.so itself is nearly 2MB in size). A quick peek at glibc's documentation shows just how much stuff there is: https://www.gnu.org/software/libc/manual/html_mono/libc.html.

Unfortunately, I wouldn't call all of it critically useful stuff. For example, "strfry" and "memfrob" are useless beyond toy applications as neither are remotely secure; "l64a" and "a64l" are useless (and possibly even dangerous) because they look like they implement base64 but with a different alphabet (and a much worse API); they have a hilariously large number of random number generators, _none_ of which produce high-quality random numbers for simulation or especially cryptography; etc. etc.

Neither MSVC nor macOS will let you truly statically link a binary (because the system interfaces aren't constant), so it's harder to directly compare, but on macOS "libSystem.dylib" depends on basically everything in /usr/lib/system which is about 5MB of code.

So yeah, I don't think I'd call many real libc libraries "small" by any stretch.

Here is a web page comparing the different C library implementations:

http://www.etalabs.net/compare_libcs.html

For the .a size:

musl: 426k

uClibc: 500k

dietlibc: 120k

If you statically link to these, your final binary will only pull what is actually needed and be smaller than this.

glibc is only so bloated because of locale und unicode support. Also 32bit wchar_t and intl errors.

Adding better string functions or CPU-specific optims is nothing compared to that.

musl is mostly so small because it only does the C locale and basic unicode towlower. Nothing like the monstruous glibc unicode tables or internationalized error messages.

I don't see value in keeping the C standard library small in this regard, since everybody is going anyway to implement this kind of code from scratch, it's like not having stdio.h for FILE* operations. However... the library you linked is surely a good pick according to me since I'm the author myself :-D

I really love his trick of returning a pointer to the inside of his own allocated memory which is then compatible with C's stdlib string functions, while having his own string functions use pointer arithmetic to get the real struct back out and operate on that. Genius! This is the kind of thing that keeps drawing me to C every once in a while. C is almost like a puzzle, to figure out how you can accomplish what would be trivial in JavaScript, but with all these restrictions caused by an almost extreme simplicity. (Granted, C is not actually simple when you start hitting all the weird edge-cases and surprising UB, etc.)

I don't love this trick. To get this to work, they had to typedef the string type to char*, which means the API will also accept any normal string and probably crash at runtime. So to get this little convenience feature, type safety was sacrificed.

Seems to me that it would also prevent tools like the address sanitizer and valgrind from detecting some invalid memory accesses (corresponding to negative indexes). A "more conventional" string library [1] might be safer.

[1] for example: https://github.com/websnarf/bstrlib/blob/master/bstrlib.txt

> A lot of C developers I know see value in keeping the C standard library small.

And what value what that be?

... if you're going to say "easier to fit a small implementation into the memory of some microcontroller" - not a sufficient argument. There's always some smaller system with not enough memory, and larger systems with more than enough.

This is C land. You're not going to make it far by making the case that bloat is acceptable.

People have a range of technical and aesthetic reasons for hating bloat and C attracts a lot of them.

Off the top of my head:

- Smaller memory footprints

- Easier reimplementation

- Lower attack surface

- Less room for breaking changes

- Less baggage for when we inevitably wish we could deprecate things

> - Lower attack surface

wut?

String formatting libraries, and other things that work with buffers are frequent attack surfaces for buffer overflow attacks.

Granted, using C means developers often implement these operations themselves which introduces the possibility of creating more attack surfaces. But it's less likely that the standard library presents an attack surface when the standard library is tiny.

Actually not having dynamic strings and buffers in the library is one of the main reasons people do pointer math and reallocations by hand introducing memory corruption bugs.

I mean, C's attack surface is like that of activated charcoal. I'm not sure that C's small standard library gives it a smaller attack surface, specifically because it means programmers who have better things to do are forced to reinvent the wheel, poorly [1]. But mostly, because C's lack of guardrails means it takes active effort on even trivial operations to be safe.

I've been working with it for nearly two decades, and every year I think more that C programs should be confined to a well-guarded quarantined area with hazard trefoils and a "beware of the leopard" sign.

[1] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule

edit: people reimplementing their "safe" string library isn't something to brag about, but be ashamed of our entire industry for.

> Smaller memory footprints

Nobody said you need to link the entire library.

Plus - with custom-made libraries, the memory footprint is no smaller.

> Easier reimplementation

Implementation begins with some part of the standard library, and gets completed gradually later on.

> Lower attack surface

Are you really arguing in favor of "everyone roll your own library" as a reduction of attack surface?

> Less room for breaking changes

C's standard library was already broken in various significant ways to begin with (e.g. gets() ...) .

Also, are you really worried about all those breaking changes between C99 and C11?

> Less baggage for when we inevitably wish we could deprecate things

Umm, you do realize things usually get deprecated when they've been replaced by something more relevant, right? In our case - new C library code.

There's some nuance to be had here. The important implications of a larger C standard library aren't technical, they're social--i.e. the implications of a larger standard library for teams using C are negative. Let's look at two other languages: C++ and Python.

C++ has a large standard library. This ends up being one of C++'s biggest downfalls: the various libraries don't play nicely with each other, have many different ways of doing the same or similar things, each have complex gotchas--and it's impossible to know them all, so you never really know C++. It's full of dark corners, and the result is bugs, and difficult debugging, and teams that try to enforce only using subsets of the language (and inevitably fail a little). This problem is only getting worse over time. The reason C++'s new features haven't just been backported back into the C standard is that people saw this coming and didn't want it happening. C is already a complex language, but it's still simple enough that you can actually reasonably read the entire C standard library (I have; QED).

A large standard library doesn't have to be this way. Python has a large standard library--this is the "batteries included" of Python. The reason Python can get away with this is that they can deprecate their mistakes. And they are: quite a few modules in the standard library will disappear with the Python 2 sunset. But this is not without downsides--everyone who has been working on a major Python project in the past few years has felt the pain at some point. And C is not Python: it's the backbone of a lot of other applications, so breaking changes are far more costly for C than for Python. C is "once supported, always supported", and for very good reason, so we can't afford to introduce mistakes to the standard.

So, if you don't want the problems of all the cruft in C++'s standard library, and you don't want the deprecations of Python's standard library, your only remaining choice, by process of elimination, is to have a very slim standard library, and make VERY conservative choices about what to add to it.

I think that we should add a new string library to C when strings are a solved problem. But contrary to what people are saying, strings aren't a solved problem. We shouldn't add a string library that doesn't solve internationalization, for example. And in fact, with that problem, the fox is in charge of the henhouse, making the problem worse: we've got people in the Unicode team adding fsking emoji to the standard. Unfortunately, I think we're going to have to wait until Unicode collapses under its own bad decisions and is replaced by a better standard, and that standard becomes mature, before committing to that proven solution. Otherwise we end up with ICU-related crap into the "once supported, always supported" C standard library.

> C++ has a large standard library.

Is it that large? i.e. is it larger than, say, Java's ?

> it's impossible to know them all, so you never really know C++

This is semantic quibbling. You know a language more if you know its commonly-used libraries better - whether they're in the standard or not.

> the various libraries don't play nicely with each other

They may not be in perfect synch, but they play ok-ish-ly with each other - as far as I can tell.

However, not playing nice is much much more of a problem with non-standard libraries - in various languages. I'm sure C programmers have that experience with a vengeance... say, everybody using different implementations of basic data structures like queues and lists and hash tables and so on.

> It's full of dark corners

Yes, yes it is. And they're pretty scary... however, that's not really because of the library size. Also, those dark corners exist, unexplored, for C libraries; it is arguably better that they be mapped once and for all, at least some of them which are somewhat-frequently stumbled upon.

...

More generally - the issue you're describing seems to be more with the combination of "iron-clad backwards compatibility" and a larger standard library:

- C has the former, not the latter - Python has the latter, not the former

C++ has the former (upto minor points like the `auto` keyword), as well as the latter, but with warts.

The warts are there, that's true, but they're not so bad. And use of the standard library does pay off very nicely - often removing warts in your own code.

> Is it that large? i.e. is it larger than, say, Java's ?

Large, yes. Larger than Java's? I don't know.

It's larger than C's, that's for sure.

> This is semantic quibbling. You know a language more if you know its commonly-used libraries better - whether they're in the standard or not.

If it's in the standard library you better bet it gets used, even if only by your dependencies.

> I'm sure C programmers have that experience with a vengeance... say, everybody using different implementations of basic data structures like queues and lists and hash tables and so on.

Eh, it's not too bad in C. Libraries tend not to force you to use their data structures too much, and if they do, you just don't use that library. Some, you use their data structures because that's what you included the library to get. It's certainly a problem, but dependency management is easier than having dependencies you can't make go away.

> The warts are there, that's true, but they're not so bad. And use of the standard library does pay off very nicely - often removing warts in your own code.

So use C++, if you like it.

My post wasn't an attack on C++. C++ has its upsides. I only described the downsides of C++ because I was explaining what C is trying to avoid.

> And what value what that be?

A lot of embedded development still have very tight requirements. We're still talking about things discussed in tens or if generous, hundreds of kilobytes.

As a more mainstream example you might have heard of, the Arduino, I think the baseline Arduino has around 32KB available.

Also, in the embedded space, the compilers used are notorious for not having a complete standard library implementation. Lots of things are missing. Thus, developers who care about porting their code to different chips, are cautious about limiting what standard library functions they depend on.

Again, as a more mainstream example you might have heard of, look at Lua, which is implemented in 100% ANSI C. Because they care about Lua truly being portable to everywhere, they try to constrain their dependence on the standard library, and they supply some #ifdefs in luaconf.h to implement or workaround some standard library functions they know to be often missing.

And as a more mainstream example of an incomplete C standard library, look at Android. Bionic is the name of their C standard library. It is missing a lot of things. (And you have to use one of those #ifdefs in luaconf.h to compile Lua successfully on Android to workaround Bionic.)

Reply on news.ycombinator.com