Hacker News Reader

Comment by ufo | Hacker News Reader

Does anyone know more examples of C APIs where the user is responsible for allocating data and where all the functions only "borrow" pointers? Is getting rid of malloc/free worth the loss of information hiding?

View on HN

Pretty much the vast majority of the API out there? Only relatively new ones like asprintf or getdelim do memory allocation for you.

I don't understand your remark about information hiding. How does the difference between returning an owned pointer and borrowing a pointer concern information hiding?

If your function operates with a pointer to a struct you don't need to put the contents of the struct in the header file. You can simply declare the struct with

    struct mystruct;

and then use `mystruct*` in the function prototypes. I think in C++ this is known as the PIMPL pattern.

I was also asking more about user-defined libraries than about the standard library...

Opaque types do not force you to do the allocation in the library.

my_lib.h:

  struct my_struct;

  size_t my_struct_size(void);

  #define MY_STRUCT_MAX_SIZE 512
  #define MY_STRUCT_VAR(name) \
    char name##_buf__[MY_STRUCT_MAX_SIZE]; \
    struct my_struct* name = name##_buf__

  void my_construct(struct my_struct*, int, float);

my_lib.c:

  struct my_struct {
    int i;
    float f;
  };

  size_t my_struct_size(void) {
    return sizeof(struct my_struct);
  }

  void my_construct(struct my_struct* ms, int i, float f) {
     ms->i = i;
     ms->f = f;
  }

user.c:

  // On the heap
  struct my_struct* ms = malloc(my_struct_size());
  my_construct(ms, 123, 456.0f);

  // On the stack
  MY_STRUCT_VAR(ms2);
  my_construct(ms2, 123, 456.0f);

Do you know of any libraries that do this?

When I tried it here GCC complained about `ms2 = ms2_buf__` being an assignment between incompatible pointer types. That sounds like an easy way to get into trouble and undefined behavior.

I just wanted to convey the idea so I didn't compile or check the code. But now that you bring up UB, I would change the MY_STRUCT_VAR macro as follows:

  #define MY_STRUCT_VAR(name)                          \
      _Alignof(max_align_t)                            \
       unsigned char name##_buf__[MY_STRUCT_MAX_SIZE]; \
      struct my_struct* name = (struct my_struct*)name##_buf__

Ie, changed to unsigned char and set the buffer to have the maximum (strictest) alignment. And added the cast.

It's a pretty widely-used technique and I have satisfied myself in the past that it's not UB (ie, by checking the Standard). I am no language lawyer though so don't take my word for it. The heap example, on the other hand, is pretty much identical to what an allocation inside your library would look like so feel free to disregard the stack example if you'd like.

One example of something based on the same principle is the 'sockaddr' structure in the BSD sockets API. See https://en.wikipedia.org/wiki/Type_punning#Sockets_example.

  _Alignof(max_align_t)

should be

  _Alignas(max_align_t)

  _Alignas(_Alignof(max_align_t))

Take a look at SDL2. It does do some mallocs, but mostly avoids it and is overall a very well designed API and codebase.

Could you give some examples? I looked at a tutorial right now and it is full of SDL_CreateXXX / SDL_DestroyXXX functions. I couldn't find any instances of code that does `XXX foo; SDL_InitXXX(&foo)`, as is proposed in the article.

SDL_Create/SDL_Destroy basically malloc/free under the hood. (SDL has its own custom allocator, so depending on your platform and compile switches, it may use something other than the standard library malloc/free.)

SDL has some value struct types like SDL_Rect which APIs sometimes pass around through pointers. SDL_RenderCopy is one example.

In my above response, it was implicit, but not explicit, that because libraries may use a different memory allocator than your main code, well constructed libraries provide their own Create/Destroy pairs. So to be explicit, you should really never call free() on memory allocated within some library, and that library should provide its own free function. (Windows devs might also remember that crossing DLL boundaries exposes this same issue, so calling free in your program from memory allocated in the DLL is potential hazard.)

I also wanted to state that "information hiding" is typical for C libraries, because it is often useful for maintaining ABI stability while allowing for future changes.

For example, SDL is very careful about keeping the ABI stable for the life of a series. They broke it between SDL 1.2 and SDL 2.0, but that was a long number of years in between.

If they change a public struct that you use, like SDL_Rect, all the memory layouts in that struct may change. That means if your program is dynamically linked, and the SDL that gets used is newer than what you compiled against (and somebody mucked with SDL_Rect), your program is not going to work correctly.

If everything is behind an opaque pointer, then they can change the internals without breaking your binary. So as an example, a running joke is that Apple seems to change the way they do fullscreen every macOS release. This means your game won't work correctly on the next macOS. So SDL will need to add new code to support whatever the new thing is, while still preserving the old code for the existing macOS versions. This could mean a lot of internal changes to SDL's structs. But since they were opaque to your program, it doesn't affect the ABI. So you can just drop in the latest SDL dynamic library with your program (no recompile necessary), and things will work.

If you're explicitly looking for that pattern, the OpenGL API has a lot of it. I'd like some other examples as well.

Reply on news.ycombinator.com