Hacker News Reader

Comment by MichaelGG | Hacker News Reader

Why is this even a thing? Copying and the like is such a common operation. Why don't chip providers offer a single instruction that gets decoded to the absolute fastest way the chip can do? That'd even allow them to, maybe, do some behind-the-scenes optimization, bypassing caches or something. It's painful that such a common operation needs highly specialized code. I know you can just REP an operation but apparently CPUs don't optimize this the same way.

This is too obvious an issue, so there must be a solid reason. What is it?

View on HN

http://stackoverflow.com/questions/8858778/why-are-complicat...

counter claims https://news.ycombinator.com/item?id=8683565

Ah that thread is great and answers my question. TL;DR seems to be that rep+mov is as fast as anything now.

The CPU has a limited amount of silicon to spend on instruction decoding. That silicon is also part of every single instructions issue latency. Machines like the Cray with complex internal operations paid the cost in issue delays, and most workloads won't earn them back.

This seems like something that compilers should do and CPU instruction sets should not.

It's really something the stdlib should do, which, when I've seen it implemented, is usually what happens.

The compiler should just provide good inlining support, so that if eg. you include the short-string optimization in your stdlib, the compiler can optimize it down to a couple bit operations, a test, and a word copy. If the test fails and your string is more than 7 bytes, it's perfectly fine to call a function - the function call overhead is usually dwarfed by the copy loop for large strings. And then if new hardware comes out and you vectorize it differently, you can get away with replacing that one function in the stdlib instead of recompiling every single program in existence.

Why? It's a common op that requires internal knowledge of every microarchitecture, isn't it? Seems like something that should be totally offloaded to the CPU so you're guaranteed best performance.

The message you were referring to was talking about code for copying strings. If you wanted an instruction to copy lots of strings, the CPU would need to know what a character is (which could be 7 bits, 8 bits, 16 or 32), what a string is, how it's terminated, what ascii and unicode is, be able to allow new character encoding standards etc etc. Then you would need other instructions for other high level datatypes. That's not what CPUs do, because you're limited by how much more logic/latency you can add to an architecture, how many distinct instructions you can implement with the bits available per instruction, how many addressing modes you want etc.

So instead, this information/knowledge about high level data types is encapsulated by standard libraries and then the compiler below that. Most CPUs have single instructions to copy a chunk of data from somewhere to somewhere else and a nice basic way to repeat this process efficiently, and it's up to the compiler to use this.

In the olden days of x86: "REPNZ SCASB" to get the length of a zero-terminated string and "REP MOVSB" to copy bytes from place to place. But I think more modern CPUs actually work faster with the RISCier equivalents.

See http://yosefk.com/blog/its-done-in-hardware-so-its-cheap.htm..., "It's done in hardware so it's cheap".

Summary: 'Why doesn't "hardware support" automatically translate to "low cost"/"efficiency"? The short answer is, hardware is an electric circuit and you can't do magic with that, there are rules.'

I remember way back in the DOS days there were a lot of clever things you could do with setting up DMA transfers which would then run in the background... With CPU/memory latency and bandwidth being so much more of an issue these days I can't see why this isn't still the case. Maybe it's all done automatically in the background now?

DMA is done automatically by modern hardware, PCIe(/Thunderbolt) and SATA just do it.

Somewhat related is e.g. the sendfile() syscall that's used by web servers/frameworks to pass a file directly from your fcgi application to the outgoing socket.

Reply on news.ycombinator.com