thradams

It's been about 10 months since I posted about Cake's ownership and nullability model.

Since then, I've significantly improved Cake's flow analysis.

The new analysis can track the origins of states across control-flow paths, making it easier to understand where a state came from and how it reached a particular point in the program.

Another milestone is that the entire Cake source code is now annotated with ownership and nullability, and checked by Cake itself.

I've documented the flow analysis here:

https://cakecc.org/flow.html

In this model, ownership is checked statically when variables go out of scope and before assignment.

"Owner pointers" must be uninitialized or null at the end of their scope.

Basically, the nullable state needs to be tracked at compile time, and nullable pointers,despite being a separate feature, reuse the same flow analysis.

For the impatient reader, a simplified way to think about it is to compare it with C++'s unique_ptr.

The difference is that, instead of runtime code being executed at the end of the scope (a destructor), we perform a compile-time check to ensure that the owner pointer is not referring to any object. The same before assignment.

So we get the same guarantees as C++ RAII, with some extras. In C++, the user has to adopt unique_ptr and additional wrappers (for example, for FILE). In this model, it works directly with malloc, fopen, etc., and is automatically safe, without the user having to opt in to "safety" or write wrappers. Safety is the default, and the safety requirements are propagated automatically.

It is interesting to note that propagation also works very well for struct members. Having an owner pointer as a struct member requires the user to provide a correct "destructor" or free the member manually before the struct object goes out of scope.

#pragma safety enable

#include <stdio.h>

int main() { FILE *_Owner _Opt f = fopen("file.txt", "r"); if (f) { fclose(f); } }

At the end of the scope of f, it can be in one of two possible states: "null" or "moved" (f is moved in the fclose call).

These are the expected states for an owner pointer at the end of its scope, so no warnings are issued.

Removing _Owner _Opt we have exactly the same code as users write today. But with the same or more guarantees than C++ RAII.

When exploring the design of nullable pointers in C and comparing them with other languages like C# and TypeScript, which have constructors, I realized that C might benefit from a way to represent transient states, the state equivalent of when object is being constructed.

The C++ mutable keyword came to mind as a potential solution.

During the object creation (or destruction), the instance is considered to be in a transitional state, where the usual constraints—such as non-nullable pointers and immutability—are lifted. Once the transitional phase is over and the object is returned, the contract that governs the object (such as immutability of name and non-nullability of pointers) is fully reinstated.

A two-minute video explaining the concepts of nullable pointers and pointer ownership in Cake, using a simple example.

Cake is a open source compiler and static analyzer in development. (Not production quality yet.)

This video shows how cake can help programmers to create safe code just fixing warnings.

https://youtu.be/X5tmkF16UMQ

We copy paste code then we add pragma safety enable

This enables two features ownership and nullable checks. Ownership will check if the fclose is called for instance, also checks double free etc, while nullable checks will check for de-referencing null pointers.

New qualifiers _Opt and _Owner are used but they can be empty macros, allowing the same code to be compiled without cake.

I think at "Keep the language small and simple" it should say avoid "two ways of doing something"

( The sample I have is 0, NULL and nullptr where nullptr is something new. Two ways of doing something makes the language complex. )

Cake disabled checks in a few linked list functions like pop_front pop_back.

Cake is not porting Rust semantics. It works on classical C code, like the first sample using fopen.

    #include <ownership.h>
    #include <stdio.h>

    int main()
    {
      FILE *owner f = fopen("file.txt", "r"); 
      if (f)
        fclose(f);
    }

But comparisons are inevitable, and I also think there are lessons learned in Rust.

C programmers uses contracts, these contracts are part of documentation of some API. For instance, if you call fopen you must call fclose.

All we need is to create contracts that the compiler can read and verify automatically.

(question about rust.. this is not implemented in cake yet)

Let's say I have to objects on the heap. A and B. A have a "view" to B.

Then we put a prompt for the user. (or dynamic condition) "Which object do you want to delete first A or B?" Then user select B. How this can be checked at compile time?

The ownership checks are new in Cake, less than one year. So the answer is no, just cake is using. And there is a lot of work to do..in flow analysis etc.

The cake source itself is moving to another experimental feature that is nullable types.

Similar of C# https://learn.microsoft.com/en-us/dotnet/csharp/nullable-ref...

(I think preprocessor is the place where memory is used and released all the time while expanding macros.)

This "dynamic drop semantics" does not exist in cake.

    int * owner p = malloc(sizeof(int));
    if (condition) 
       free(p);
    free(p); // error p may be initialized/moved.

to fix

    int * owner p = malloc(sizeof(int));
    if (condition) 
    { 
      free(p);
      p = 0;
    }
    free(p);

auto, typeof, _Generic are implemented in cake. Sometimes when they are used inside macros the macros needs to be expanded. Then cake has #pragma expand MACRO. for this task.

Sample macro NEW using c23 typeof.

    #include <stdlib.h>
    #include <string.h>

    static inline void* allocate_and_copy(void* s, size_t n) {
        void* p = malloc(n);
        if (p) {
            memcpy(p, s, n);
        }
        return p;
    }

    #define NEW(...) (typeof(__VA_ARGS__)*) allocate_and_copy(&(__VA_ARGS__), sizeof(__VA_ARGS__))
    #pragma expand NEW

    struct X {
        const int i;
    };

    int main() { 
        auto p = NEW((struct X) {});     
    }

The generated code is

    #include <stdlib.h>
    #include <string.h>

    static inline void* allocate_and_copy(void* s, size_t n) {
        void* p = malloc(n);
        if (p) {
            memcpy(p, s, n);
        }
        return p;
    }

    #define NEW(...) (typeof(__VA_ARGS__)*) allocate_and_copy(&(__VA_ARGS__), sizeof(__VA_ARGS__))
    #pragma expand NEW

    struct X {
        const int i;
    };

    int main() { 
        struct X  * p =  (struct X*) allocate_and_copy(&((struct X) {0}), sizeof((struct X) {0}));     
    }

The idea is to keep cake aligned with C, not a language fork. But Cake itself could have a fork to Cake++. :D

(by the way, embed is not working on web version because of include directory bug - it is an open issue and regression)

Rust needs to add some runtime checks when calling destructors in scenarios where some object may or may not be moved.

In C++ for instance, for smart pointers, the destructor will have a "if p!= NULL". Then if the smart pointer was moved, it makes the pointer null and the destructor checks at runtime for it.

One issue I see with this approach (compiler leaking memory) is, for instance, if the requirements change and you need to utilize the compiler as a lib or service. For example, if the Cake source is used within a web browser compiled with Emscripten, leaking memory with each compilation would lead to a continuous increase in memory usage.

Additionally, compilers often offer the option to compile multiple files. Therefore, we cannot afford to leak memory with each file compilation.

Initially I was planning a global allocator for cake source. It had a lot of memory leaks that would be solved in the future.

When ownership checks were added it was a perfect candidate for fixing leaks. (actually I also had this in mind)

In cake there is no temporal hole, we cannot reuse the deleted object. This prevents double free and use after free.

    int main() {   
       struct X * owner p = malloc(sizeof(struct X));
        
       p->text = malloc(10);

       free(p->text); //object text destroyed

       //p->text is on uninitialized state. 
       //cannot be used (except assignment)

       struct X x2 = {0};

       *p = x2; //x2 MOVED TO *p

       x_delete(p);

Thanks for the rust sample. It looks very similar. Can the allocator be customized?

As I said I am not Rust specialist.

Also, in my understanding is that in Rust, sometimes a dynamic state is created when the object may or may not be moved.

In cake ownership this needs to me explicit ( and the destructor is not generated)

I also had a look at Rust in lifetime annotations. This concept may be necessary but I am avoiding it.

Consider this sample.

   struct X {  
     struct Y * pY;  
   };  
   struct Y {  
     char * owner name;  
   };  

An object Y pointed by pY must live longer than object X. (Cake is not checking this scenario yet)

Also (classic Rust sample)

    int * max(int * p1, int * p2) {  
      return *p1 > *p2 ? p1 : p2;
    }

    int main(){  
       int * p = NULL;
       int a  = 1;
      {
         int b = 2;
         p = max(&a,  &b);
      }
      printf("%d", *p);
    }

This is not implemented yet but I want to make the lifetime of p be the smallest scope. (this is to avoid lifetime annotations)

   int * p = NULL;
   int a  = 1;
   {
      int b = 2;
      p = max(&a,  &b);
   } //p cannot be used beyond this point*

Cake implements defer as an extension, where ownership and defer work together. The flow analysis must be prepared for defer.

    int * owner p = calloc(1, sizeof(int));
    defer free(p);

However, with ownership checks, the code is already safe. This may also change the programmer's style, as generally, C code avoids returns in the middle of the code.

In this scenario, defer makes the code more declarative and saves some lines of code. It can be particularly useful when the compiler supports defer but not ownership.

One difference between defer and ownership checks, in terms of safety, is that the compiler will not prompt you to create the defer. But, with ownership checks, the compiler will require an owner object to hold the result of malloc, for instance. It cannot be ignored.

The same happens with C++ RAII. If you forgot to free something at our destructor or forgot to create the destructor, the compiler will not complain.

In cake ownership this cannot be ignored.

    struct X {
      FILE * owner file;
    };

    int main(){
       struct X x = {};
       //....
       
    } //error x.file not freed

>Can you ask Github Co-pilot to look at C code and answer the question "What is >the length of the array 'buf' passed to this function"? That tells you how to >express the array in a language where arrays have enforced lengths, whicn >includes both C++ and Rust

this is the way you tell C what is the size of array.

    void f(int n, int a[n]) {
    }

mempool does not solve double free, use after free (at least at compile time) or fopen sample. But mempool and ownership can be complementary.

In cake object and memory are two resources. We can for instance, delete the object and reuse the same memory.

For instance, this code is correct.

    #include <ownership.h> 
    #include <stdlib.h>

    struct X {
       char * owner text;
    };

    void x_delete(struct X * owner p)
    {
        if (p)
        {
           free(p->text);
           free(p);    
        }
    }

    int main() {   
       struct X * owner p = malloc(sizeof(struct X));
        
       p->text = malloc(10);

       free(p->text); //object text destroyed

       struct X x2 = {0};

       *p = x2; //x2 MOVED TO *p

       x_delete(p);   

       //no need to destroy x2
    }

the other difference in RAII destructor cannot be turned off. In cake the same object can be a "view"

    struct X x = {0};
    //...
    view struct X x2 = x;
    destroy(&x);
    //x2 does not need destructor

I think gsl::Owner is related with RAII.

The difference with cake ownership and RAII , is that with C++ RAII, the destructor is unconditionally called at end of scope. Then flow analysis is not required in RAII.

Cake requires flow analysis because "destructor" is not unconditionally called.

When the compiler can see that the owner is not owning a object (because the pointer is null for instance) then the "destructor" is not necessary.

To understand the difference.

With flow analysis (how it works today)

    int main() 
    {
      FILE *owner f = fopen("file.txt", "r"); 
      if (f)
        fclose(f);
    }

Without flow analysis (or with a very simple one, where the destroy must be the last statement)

    void fclose2(FILE * owner p) {
       if (p) fclose(p);
    }

    int main() 
    {
      FILE *owner f = fopen("file.txt", "r"); 
      if (f){
      }
      fclose2(f);
    }

the cake implementation cannot be mapped to rust. I am not rust specialist but one concept for instance is that a owner pointer owns two resources at same time, the memory and object. In rust it is one concept.

Owner pointers take on the responsibility of owning the pointed object and its associated memory, treating them as distinct entities. A common practice is to implement a delete function to release both resources, as illustrated in Listing 7:

Listing 7 - Implementing the delete function

    #include <ownership.h>

    #include <stdlib.h>


    struct X { 
      char *owner text; 
   };

    void x_delete(struct X *owner p) {
      if (p) {
        /*releasing the object*/ 
        free(p->text);
    
       /*releasing the memory*/ 
       free(p); 
     }
   }

   int main() {
      struct X \* owner pX = calloc( 1, sizeof \* pX);
      if (pX) {
       /*...*/;
       x_delete( pX); 
      }  
   }

currently cake uses existing msvc and gcc headers. These headers does not have any owner qualifiers.

The temporary solution, is the re-declare the malloc etc when compiling with cake and not complain withe the function signature difference only by owner qualifiers.

if this ownership were standard then gcc and mscv headers would have the qualifiers there enabled or not , but they would be there.

Not sure if I understood. the usage of old and new (checked and unchecked) is a challenge.We may have the same headers used in both codes. The other challenging is that same source may compile in compiler with or without support.

Ownership Feature Strategy (Inspired by stdbool.h)

If the compiler supports ownership checks and qualifiers such as _Owner, _View, _Obj_view, etc., it must define __STDC_OWNERSHIP__.

However, even if the compiler implements ownership, it is not active by default. The objective is to have a smooth transition allowing some files without checks. For instance, a thirty part code inside your project.

For instance, when compiling this file, even if the compiler supports ownership we don't have errors or warnings because the checks are not enabled by default.

    #include <stdlib.h>

    int main() {
      void \* p = malloc(1);
    }

A second define __OWNERSHIP_H__, is used to enable ownership. This define is set when we include <ownership.h> at beginning.

    #include <ownership.h>
    #include <stdlib.h>

    int main() {
      void \* p = malloc(1); //error: missing owner qualifier
    }

The other advantage of having a <ownership.h> is because owner is a macro that can be defined as empty in case the compiler does not support ownership, allowing the same code to be compiled in compilers without ownership support.

This code will not compile because we cannot return owner as non owner.

To fix we need to add qualifier owner at return type.

    FILE* owner open_file(const char* p) {
        FILE* owner f = …
        return f;
    }

Then when returning f , f is moved. Because f was moved the compiler will not complain at the end of scope of f.

https://thradams.com/cake/playground.html?code=I2luY2x1ZGUgP...

It works in both ways. We can just tell the compiler to ignore some function. We can be also creative writing the code that at same time is good and makes the static analysis happy. A good sample is linked lists.

    #include <ownership.h> 
    #include <stdlib.h>
    #include <assert.h>
    #include <stdio.h>
    #include <string.h>

    struct node {
    char * owner text;
    struct node* owner next;
    };

    struct list {
    struct node * owner head;
    struct node * tail;
    };

    void list_append(struct list* list, struct node* owner node)
    {
    if (list->head == NULL) {
        list->head = node;
    }
    else {
        assert(list->tail->next == 0);
        list->tail->next = node; //ZERO OVERHEAD
    }
    list->tail = node;
    }

    void list_destroy(struct list* obj_owner list)
    {
    struct node * owner p = list->head;
    while (p) {
        struct node *  owner next = p->next;
        free(p->text); 
        free(p);
        p = next;
    }
    }

    void list_print(const struct list* list)
    {
    const struct node * p = list->head;
    while (p) {
        printf("%s ", p->text);
        p = p->next;
    }
    }

    int main()
    {
    struct list list = {};
    struct node  * owner p =  calloc(1, sizeof * p);
    
    if (p) {
        p->text = strdup("item1");
        list_append(&list, p);
    }

    list_print(&list);
    list_destroy(&list);
    }