Hacker News Reader

Comment by zamadatix | Hacker News Reader

Network protocols immediately came to mind for me. They love packed structs of weird bit sizes because anything that's not payload is overhead on every message for the rest of time. Fields can also be large, e.g. VLANs are 12 bits, VXLAN ids are 24 bits, MAC addresses are 48 bits, IPv6 addresses are 128 bits so it's not just limited to a couple of small sized bitflag style things.

View on HN

For packed structures, C already has bit fields. Example (from https://en.cppreference.com/w/cpp/language/bit_field):

  struct S {
    // will usually occupy 2 bytes:
    // 3 bits: value of b1
    // 2 bits: unused
    // 6 bits: value of b2
    // 2 bits: value of b3
    // 3 bits: unused
    unsigned char b1 : 3, : 2, b2 : 6, b3 : 2;
  };

This is more aimed at large integers.

I'm pretty sure the exact layout is not guaranteed in this case, so while the above code may work in many cases it's not possible to represent all data structures like this, particularly if the bits are not byte or machine word aligned.

I don't think the standard even guarantees which bits in an integer the bitfields will be stored. That's important for network protocols.

Kinda important part mentioned in the link is

> Adjacent bit field members may be packed to share and straddle the individual bytes.

So it might not be packed. Ofc it usually is, at least on gcc and such.

I think the bigger issue in regards to networking is ordering, not the (lack of) packing. I don't think the ordering is guaranteed and probably depends on CPU architecture (e.g. big vs little endian), not necessarily compiler.

In practice, little-endian machines have the first bit field in the least-significant bits of the data type they fit into (usually but not always an int), and big-endian machines have the first bit field in the most-significant bits.

"so it's not just limited to a couple of small sized bitflag style things."

The examples I gave are integers you may have to compare or manipulate not just bitfields to copy.

I also disagree this is aimed at large integers based on the examples of the syntax in the article commonly being <16 bits, sometimes <8, while the examples I cited had non power of 2 use cases up to 48 bits.

You can already do this in C though, If EVER SO SLIGHTLY wasteful in non-practical terms.

If you need a 23bit object you just structure it to be that. It’s a couple of AND or SHIFT ops when accessing, but so what? Even for 100Gbit networking you aren’t going to max out even a slightly appropriate CPU.

The same argument could be made to get rid of all numeric types except the largest. That's likely how it would compile down to on platforms without 23 bit types though just it would be handled automatically based on target. I think the point of such a feature is to abstract you from thinking about if the machine has a native 23 bit type the same way you don't think if the machine has a native 64 bit type or a hardware float type today. Also when you do this manually you're now responsible for tracking the actual type and such too. Beyond that you also want to do operations on these fields not just store them and ignore them, does the IP match this policy? Have I learned this MAC? Is this TOS in an allowed range or does it need to be bleached? Constantly pulling these out and putting them back in, the above work isn't a one time thing.

100 Gigabit networking eats more CPU than you think, especially if you're actually looking at headers. It's an enormous portion of cloud CPU usage and a big reason networking is still driven by what's easy to put in an ASIC vs running the easier thing in software.

You probably could get the compiler to shout at you for a few more mistakes with explicit types?

Reply on news.ycombinator.com