Hacker News Reader

Comment by vlovich123 | Hacker News Reader

Probably a historical artifact from when the extra 4-8 bytes required posed a serious overhead.

The other advantage is that there’s nothing involved in porting C strings between 8, 16, 32, and 64 bit machines.

Of course, most modern languages will store the length separately (using small string optimization) AND offer a way to convert to a C string (for OS interop).

There’s an argument that taking a substring off a C string a) can be done without allocating a copy b) can be done with pure pointer manipulation. In practice though I’m not sure how critical that case is.

View on HN

> There’s an argument that taking a substring off a C string a) can be done without allocating a copy b) can be done with pure pointer manipulation. In practice though I’m not sure how critical that case is.

Rust supports the same features by storing the length of the string with the pointer. Same thing with other references to dynamically sized memory (slices).

> Probably a historical artifact from when the extra 4-8 bytes required posed a serious overhead.

Variable length encodings like varint [1] allows to use 1 byte for small integers and more for large. Zero terminated string will use less memory for large strings, but the difference between 1 (\0) and 3 bytes (varint length) for a 64k string is negligible.

[1] https://sqlite.org/src4/doc/trunk/www/varint.wiki

Accessing a varint length isn’t free. In fact, one of the main performance advantages for capnproto / flat buffers if avoiding varint encoding. It may not matter in all cases, but it would in the ones that absolutely needed to access the length frequently. Probably better to just eat the four bytes and do a small string optimization instead.

Yeah, developing an OS in the 70s on a PDP-10 was a bad move, but that's what they had.

I suggest looking at the string handling capabilities of 1960 hardware and the high level languages that were used to program them.

This could also be achieved by locating the pointer and the (offset and length) as separate.

This will cause a lot of indirect access which can be a problem in the 70s-80s' because this not only requires more memory but this also requires many more operations (say like LEA) to extract a slice.

This might work fine in a bank mainframe, but is prohibitively expensive in the minicomputers back in the days.

C is written for Unix. Unix is written with minicomputers in mind. Thus C is somewhat influenced by minicomputers and has to work on their setback in the good ol' days.

You have to think in the historic context.

> This will cause a lot of indirect access

It causes strictly less indirect access if you store it as part of the string struct, as C++, Go, or Rust do.

But makes passing around the type challenging to do efficiently. I agree it was probably the wrong choice in the end, but given the information available at the time, it’s not clear to me there was enough data / reasonable way to predict what the consequences would be and which approach would be better.

It’s more clear today but that’s because we’ve had 20 years of data of various experiments and ideas like small string optimizations on modern architectures that didn’t exist 40+ years ago.

> But makes passing around the type challenging to do efficiently.

Doesn't seem to be much of an issue in practice, even when strings are 3 words wide (as is the case for C++ and Rust because their strings are mutable[0], Go's strings are not so only 2 words).

Especially as it's a good tradeoff for safety, reliability, gains you performances elsewhere, and allows slicing.

[0] Rust String specifically, &str is immutable and 2 words, C++17 added string_view but I don't know how large it is, or whether it's used much if at all

> This will cause a lot of indirect access

You can optimize the common case of string data following the header by encoding the fact in a bit of the length field.

E.g. if the length is odd (LSB = 1) then the next word contains the pointer to the string body, otherwise the string body follows the length field. Archs that lack tagged arithmetic would require a shift but at least that skips a memory read for the string data indirection.

I suggest looking at the string handling capabilities of 1960 hardware and the high level languages that were used to program them.

Reply on news.ycombinator.com