Comment by simias | Hacker News Reader

Comment by simias | Hacker News Reader

Parent Full thread

Linux is a kernel, not a C runtime so that's not really relevant. You say you realize that but then what are you arguing for exactly? Arguably you could complain that the glibc (or whatever libc you're using) is not working around that by, for instance, scrubbing the pages in malloc() to force the kernel to allocate memory.

But even then I'm not convinced that anything here is non-standard, at worse maybe we're in a bit of a grey area. As long as the kernel maintains its smoke and mirrors whether and how it allocates memory is irrelevant from the point of view of the standard. The C language has a rather simplistic memory model, it doesn't impose a lot on the implementation.

Now the problem occurs when the program attempts to access virtually-allocated memory and the kernel realizes that it can't find any physical memory to map it to. In this situation several things can happen but in general the process will be killed. Is it against the standard for the OS to kill a program for arbitrary reasons? I can't imagine why. It could also freeze the program, waiting for more memory to become available. Again, not against the standard as far as I can tell. Or maybe kill some other program to free memory.

If you have some specific part of the C standard in mind please do tell, I always find these language lawyering arguments interesting, somehow.

> If you have some specific part of the C standard in mind please do tell

See here: https://news.ycombinator.com/item?id=20145604

Also note the POSIX standard:

Upon successful completion with size not equal to 0, malloc() shall return a pointer to the allocated space. If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned. Otherwise, it shall return a null pointer and set errno to indicate the error.

In neither case is there any provision for returning a non-null pointer to anything other than an allocated block of memory of at least the given size.

Thank you for your reply but I still don't buy it. As far as the program is concerned it is returned a memory block, what this "memory" is effectively behind the scenes is none of the standard's business. As long as the implementation manages to maintain the illusion it's perfectly fine AFAIK. The problem is when this breaks down and the kernel realizes that it can no longer maintain the masquerade. If at this point it did something stupid like map this block nowhere or remap something that's already been allocated and let the program continue like this, then yes that would be a clear violation of the contract. But it does no such thing, it kills the program instead. If there's no program then there's no problem. At no point did a single C instruction get executed in an environment that did not maintain a coherent memory model. >In neither case is there any provision for returning a non-null pointer to anything other than an allocated block of memory of at least the given size. A pointer to virtual memory is a pointer to memory. The rest is an implementation detail. If, when dereferenced, the kernel issues an order to Amazon for more RAM and waits for it to be installed to resume the execution, that's none of the C standard's business.

> If, when dereferenced, the kernel issues an order to Amazon for more RAM and waits for it to be installed to resume the execution, that's none of the C standard's business.

We're not, and we never were, debating the situation where dereferencing the non-null pointer returned by malloc succeeds but takes long time due to your Amazon order. We've been talking about the situation where it fails. malloc is not allowed to return a non-null pointer to a memory block that cannot be written to. Linux does it anyway, and in doing so blatantly violates the standard.

That is my point, it doesn't fail. Either the kernel finds out a way to map the memory and it succeeds, or it kill the program and the instruction never runs. Code that doesn't run can't violate the standard. When the code is allowed to run all the invariants are guaranteed to be respected. If I write this code:

    char *b = malloc(2);
    if (b == NULL) {
      return 0;
    }
    
    b[0] = 'A';
    b[1] = '\0';
    
    printf("%s\n", b);
    
    free(b);

The standard tells me that if the malloc succeeds then the following code, if allowed to run, will display "A" on stdout. The C standard cannot and does not guarantee that a C program can't be interrupted however. For the sake of the argument we could imagine a kernel that instead of killing the program freezes it indefinitely on disk waiting for RAM to be available. It's functionally the same thing. As long as the kernel doesn't let code run with broken invariants it's fine. This is completely outside of the scope of a language standard to define.

Or, to try one last time from a different direction, if you consider that the C standard mandates that accessing memory returned successfully by malloc has to be successful and I happen to press ^C when that happens in a program, should the kernel refuse to kill the program? This is obviously absurd, but it's effectively the same thing: the kernel reacts to some external state and decides to terminate the program.

Okay this is a far more reasonable argument but I'm not convinced it's right. The standard does define normal and abnormal program termination. It also defines <signal.h>. SIGINT addresses the keyboard case (or the implementation can provide another signal). SIGABRT, SIGTERM, etc. are raised upon termination. For the keyboard case, then the program would be made aware, and it can indeed ignore the Ctrl+C request if it desires. That's perfectly normal and nothing absurd. For other types of termination, the other signals are raised. But in this case the program is terminated before any signal is raised at all. Now, as a practical matter I would argue the hosted environment should still be able to kill the program in the face of user request simply because nobody wants a host that's the slave of the program even if it's against the standard, but this is going far, far, far beyond that. It's happening at the request of neither the user nor the program; it's just happening because the host feels like it. Now that's both a violation of the standard and an uncool one at that!

P.S. I don't think indefinite hold is "functionally the same thing" as termination. A caller system(), for one, would need to return in one case, but not the other.

If a tiny, strictly conforming ISO C program obtains a tiny block of memory from malloc (like a few hundred bytes) and crashes when trying to initialize it, that is almost certainly a non-conforming ISO C implementation. The reason is that the implementation cannot support even one single program that exercises each of the implementation limits.

See 5.2.4.1 in http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1548.pdf

However, it's true that not any old instance of this malloc problem demonstrates such a nonconformance. If it happens in a large program that has allocated gobs of memory, then no.

Basically if the system is low on memory that it can no longer support the execution of a small C program with modest memory use, then it becomes nonconforming.

However, the mere property that memory can be doled out by malloc which might later not be used doesn't make it ipso facto nonconforming.

Moreover, a system with any kind of memory management (including management that earnestly reports null for "out of memory") can be come a nonconforming C implementation if it is low on memory.

Allocated "space", i.e., virtual memory space. Malloc allocates virtual memory space, not physical RAM.

C has no notion of virtual memory. If memory is allocated for you that means you can read and write to it.

There is no guarantee offered by the standard that you must be able to read and write to memory allocated by malloc(). What if the memory was allocated and just a split second later, the physical RAM burnt out due to overheating? How is the standard supposed to guarantee reading and writing to it then?

Both the hardware burning after memory allocation and lack of availability of physical memory after memory allocation are outside the scope of the standard. The standard says nothing about them. A C program can fail in these scenarios without violating the standard.

> Linux is a kernel, not a C runtime so that's not really relevant.

Maybe it's more accurate to say that Linux is a kernel that assumes many features and semantics of the C runtime. But it certainly seems more deeply intertwined than you're willing to address here.

Reply on news.ycombinator.com